Pacers are fairly derided because the ambience is objectively poor. Most trains, however, the experience is mostly subjective. Cosy to one is claustrophobic to another. Rock hard seating to one is supportive to another. I actually don't mind travelling on voyagers (when I'm seated, sadly most of my experience with them is standing in a vestibule!) but I know many people really dislike them.
It's very rare that the general public cotton onto a particular type of train because of its reliability (or lack thereof) - it's usually down to the travelling experience. To be fair, Pacers aren't actually that reliable either, but they're no worse than Sprinters and a few other unit types that evade such criticism.