I am pretty confused as to what your position is at the moment. So in your opinion you have paid whichever organisation the £3.20 you think you owe, and you consider that you have done everything right. Is that a fair summary?
Ignoring all the other side-discussions, if that were indeed the case, then surely you have nothing to fear. You can very confidently allow them to take you to court, should they decide to do that, and after presenting your side of the arguments, you believe the judge will find in your favour. From what I can see, a large number of posters also appear to agree with you.
So in essence, nothing else needs to be done from your part, apart from preparing a pack for the court for when/if needed. You can confidently get on with life and ignore all future communication from whoever, until the day of the hearing.
I am not as confident as some other posters, but largely because I am not sure where you stand legally, but if it were indeed so clear-cut as some others have suggested, there is no more you need to do except what I outlined above.
I cannot see myself agreeing with it, for the simple fact that there is no guarantee you will definitely win should it go to court, and by your actions trying to sort it all out, I get the impression that you are not 100% sure either. All of the above would have only made sense had you been absolutely confident that you were correct, but if that were the case, there surely would be no need to have any further involvement with either SET and IPFAS. In the event that you are taken to court, you can grab them by their goolies comprehensively.
All of this means that it would have made much more sense to have simply sent £3.20 to the registered address or relevant personnel as DaveNewcastle pointed out, or, since you requested assistance from Transport Focus, to have followed what they agreed with SouthEastern. I find it unfortunate that you did neither.
Obviously I wish you luck in your endeavours, and am sure this will be resolved one way or another, but I have big questions over whether all this hassle for the effort you invested in it is all worth it. That, I suspect, only you can answer. I hope that expands on my post four upthread, and explains my take on this whole matter.