I would strongly advise you to pay the £20 here. You might be able to get the train company you got the ticket from to refund it because of the problems but that is a separate matter. On the Penalty Fare notice you have received what is the reason stated for issuing the notice?
If you pay the penalty fare first and then appeal, I think it would be correct that the worst that could happen then is you loose the appeal. It would be very bad form for them to cancel and otherwise paid Penalty Fare and refund you just because you appeal and they don't like it. I don't think the rules allow them to do this. Penalty Fares can be canceled though under certain circumstances. If you are going to appeal though you have to have valid grounds for appealing.
Thanks for the advice - the weird thing is that the inspector who issued it did not even complete the form properly. He didn't put anything in the reason for issuing, got my details and the due date wrong too. The customer service was very bad! You are correct, I feel that my grounds to appeal isn't strong, as others have suggested, the penalty fare was issued in line with the rules and regulations. Part of me thinks if I don't try I would never know, but a larger part of me is worried that it gets to prosecution.