What exactly does the letter say? Does it mention prosecution under any specific wording or law, or does it just ask for your comments/version of events?
Did you pay for a new undiscounted ticket when you were inspected, or were you issued with a Penalty Fare, or did you merely have your details taken?
Unfortunately, if the matter was not regularised there and then through the purchase of a new ticket (or the issuance of a Penalty Fare), there would be no defence to a prosection under Byelaw 18(1) of the
Railway Byelaws 2005, which creates a strict liability offence of boarding a train without a valid ticket.
As you have a valid Railcard, prosecution under the more serious
Section 5(3)(a) of the Regulation of Railways Act 1889 (RoRA) would not be successful, as you would clearly not have intent to avoid payment of the fare (a prerequisite to conviction under said Section).
Much though I can appreciate that forgetting the Railcard is itself frustrating, and it is even more frustrating to then suffer financial or other penalties for forgetting it, it is important to keep a cool head and to think carefully about the best way forward. Trying to defend a prosecution under Byelaw 18(1) on the basis that you had a Railcard but could not produce it will not be effective: that is not a defence because, as previously mentioned, it is a strict liability offence (similar to speeding - intent doesn't matter).
One saving grace is that a conviction under Byelaw 18(1) would not be a recordable conviction (unlike Section 5(3)(a) of RoRA), and it would be immediately 'spent' under the Rehabilitation of Offenders Act 1974. Therefore it would not generate a criminal record on the Police National Computer, and it would not need to be declared for the likes of insurance policies, job applications etc. - not for anything except high-level security clearance and the like.
The only question, really, is whether you would prefer to settle the matter out of Court for an administrative payment (usually the train company's costs of investigation plus any fare outstanding), or whether you would prefer to be prosecuted and taken to Court.
It is not always possible to settle matters such as this out of Court, and there is no
right to such a settlement, but where it is a first time offence consisting merely of an accidental mistake rather than intentional fare evasion, it is often possible to negotiate a settlement with the likes of GWR.
I hope that helps, and as said, if you could provide more details on the contents of the letter (perhaps a redacted photograph of it), that would help us to give further advice. It's obviously not a pleasant situation, but it is certainly not the end of the world.