Another complication with time restrictions on Railcards:
Imagine you (a family) are going on holiday by train to somewhere relatively far away from where you live. Suppose the hypothetical 0700 departure from your local station gets you to your destination at 1300, you then have maybe four/five usable hours at the destination before needing to buy food/go to your accommodation. This is plenty of time to, for example, visit a local museum or castle.
If you are restricted to off-peak times, which for your local station is 0930 onwards, and the next suitable departure to your destination is 1000, you will arrive at the destination at around 1600, leaving only an hour or two of useful time. This may mean that you need to stay another night at the place order to fully explore the area, which may mean the holiday becomes unaffordable.
Whilst I can understand some of the reasoning for off-peak restrictions, they shouldn't apply for long journeys. The problem is, where do you draw the line between short and long how do you stop people using longer tickets for shorter journeys?