The point here is that there is also a lot of demand to Oxford from these places. Not everything is focused on London, and much like Cambridge - Oxford also has a lot of inbound demand, high-paying jobs, leisure, cultural and recreational attractions etc.
And overall on this thread - that Chiltern does a lot of things. End to end, London - Birmingham, sure. But also commuting to both of those from points all along the route, and into Oxford from points along the route (Birmingham-Oxford was planned). That requires a few stopping patterns. I would say speed London-Birmingham is the least important thing (as options already exist and HS2 will mop this up - and have plenty of seats) - but fast journeys from Bicester, Leamington and Banbury to London do also need to factor.