Well, sticking to places that are roughly town sized i.e. avoiding villages and large urban settlements which should really be cities:
Hastings and to a lesser extent Folkestone - these places have perhaps fallen on hard times but I still have a big soft spot for them.
Rye, Winchelsea and West Malling are lovely.
Lancaster and Canterbury - OK these are cities, but they are town sized.
(I lived in Lancaster as a sudent and it is always a pleasure to go back, Canterbury has good shopping etc)
I love Penzance, St Ives Cornwall, Looe, Newton Abbot, Holt and Sheringham due to good holidays.
Skipton, Huddersfield, Whitby, Howden, Barton upon Humber are all nice places up here.
Oh yeah, and I forgot to say Lewes and Eastbourne (brighton would be way up in the list but I count that as a full city anyway !)