When it is your money and your time then you are more than welcome to deal in probabilities and likelihoods.
Since, in the case, it is neither please don't describe the 'best' course of action as one that can result in significant loss (financial and time) to the OP when there is a simpler and cheaper course of action available, which is guaranteed to resolve the matter.
Paying £20 is neither cheaper nor simpler than paying the amount actually owed, which is nothing at all. We have seen many valid appeals rejected in the past on this forum, (and cases where the amount of a 'refund' still leaves the successful appellant out-of-pocket relative to what they should have paid) so there is no guarantee the OP will get his/ her money back if he pays and appeals; in other words, you are incorrect in stating that paying and appealing is guaranteed to resolve the matter in a satisfactory manner.
I think that the chances of the TOC prosecuting are very slim indeed, given that the ticket presented was valid for the journey made. Even if they did, the OP would certainly not need to spend hundreds, let alone thousands, on a solicitor when he/she could get an acquittal representing him/ herself.
I would appeal anyway, and promptly, but, in contrast to a case where the passenger is, or even might be, in the wrong, I would advise against paying pending the determination of the appeal, since any so-called 'administration fees' for late or non-payment would be wholly unenforceable in this specific instance.
If the appeal is rejected, the OP could still choose to pay at that point in order to avoid the possibility of prosecution, however remote, and then pursue various avenues to recover the money paid.
If the appeal is upheld, then the OP has avoided giving the TOC an undeserved interest-free loan of £20.