The steam train runs every 45 minutes on peak days. It might just be possible with a DMU to increase frequency to half hourly, though it would mean a tight turnaround time at Kingswear to be able to run Churston - Kingswear - Churston in 30 minutes - unlike St Ives the branch does need some breathing space, as it needs to interact with the mainline for through running to Exmouth. Even if that were possible it is still fewer coaches per hour.
From my experience, most people jump on the ferry to Dartmouth, and around 30% probably do the round robin. Dartmouth is a very obvious day out whilst staying in Torbay, but if the steam train wasn't there I think that most people would drive. The hassle of parking in Dartmouth (which does have a P+R) would probably be perceived as no worse than the combined hassle of parking in Paignton, catching the train, and probably needing to get a separate ticket for the ferry.
There is also the marketing side. Every hotel, service station, Tourist info, pub etc has stacks of brochures for the steam railway, and they have kiosks all over Torquay. There are Great Scenic Railways brochures around, but they cover all of the Devon and Cornwall lines, and so don't do such a good job of marketing any of them. A steam railway is seen as something to do on holiday, I think most people wouldn't think of a NR train in the same light, so wouldn't go and investigate