So what times am I allowed to travel on each of the tickets?
Monday to Friday, the first train from Durham you can take with an Off-Peak Return to Twickenham is the 07:42. You can take this train or any train after it. With a Super Off-Peak Return to Twickenham it's the 08:38 that's the first valid service.
On the way back to Durham, from Monday to Friday, in the morning with an Off-Peak Return the only trains you can't take are the 07:00 (changing at York) and the 07:30. In the morning with the Super Off-Peak ticket the first train you can take is the 09:30.
In the evening, from Monday to Thursday, with the Off-Peak ticket you can't take any train between (inclusive) the 16:00 and the 17:30. And with the Super Off-Peak ticket you can't take any train between (inclusive) the 15:00 and the 18:30.
In the evening, on Fridays, with the Off-Peak ticket there are no restrictions. And with the Super Off-Peak ticket you can't take any train between (inclusive) the 15:00 and the 17:30.
On Saturdays, Sundays and bank holidays you can take any train in either direction with either the Off-Peak or Super Off-Peak ticket.
Note that the times I've quoted are for non-overtaken trains or itineraries; there may be other trains which your ticket is or isn't valid for, but I've disregarded those as they're not relevant to a journey to/from Durham.
The easiest thing, to be honest, is simply to put it all into a journey planner and see what it spits out.