The majority of passengers I find will pay when challenged, and a penalty fare scheme doesn't challenge anybody especially if they're aware that enforcement is sporadic and if they're onboard for 3 minutes they all just need to be at the opposite end as the guard to avoid paying their fare.
Unfortunately over the years the number of Assistant Ticket Examiners (ATE) nationwide has been reduced in favour of TVMs and penalty fare schemes and as far as I can see all this does is achieve a reduction in the headcount and salary cost. I don't feel a penalty fare scheme has any degree of success unless its enforced every time, on every train.
I think the problem with ensuring fares are collected is a manpower issue, a single guard on a 3 or 4 car full and standing service isn't going to physically be able to see all passengers, a guard and ATE stands a better chance, and a guard, ATE and selection of TVMs and/or booking offices at the busiest stations stands the best chance.
The only winners from penalty fare schemes to me seem to be Schiedt & Bachmann and their salesmen.