You will get a lot of probably conflicting answers to a question like this

but here's my take, as a fairly recently qualified freight driver..
Passenger pros:
Cleaner working environment, a roster that allows you to see far ahead and plan your life a bit more, fewer night shifts (it's all TOC dependent)
Freight pros:
No passengers to deal with, especially compared to DOO TOCs, they're less bothered about being a few minutes late, possibly greater route/traction variety (again FOC-dependent)
With Freight most people state the main disadvantages as, lots of night shifts, often don't get your shifts for one week until the week before, so hard to plan your life - but again that's FOC-dependent; I work for one of the big FOCs where we do see our shifts 13 weeks ahead, we work very few weekends (but still a lot of nights), do no engineering trains and although our base pay may not be as high as most others, there's a whole load of little extras in our T&Cs that we book, which goes a long way to making up that shortfall.
At my depot we've had 2 drivers move over from the local TOC in the past few months, similarly we've apparently had drivers move to TOCs, some of whom stays and some of whom came back!
It's all down to preference...