Did it in the week with a Wayfarer ticket and didn't have any problems. XC ticket check both ways didn't raise an eyebrow.
Interesting timings for the diversion as we left Chesterfield late and then the train arrived significantly early in Burton. Coming back got slower and slower for some reason but not a problem on that score as I was only doing it for the ride.