Because post Brexit, Ireland will have different customs rules to the UK.
Therefore if the customs union is left, there would have to be customs checks on every border with Ireland.
Ireland will also have FoM with the EU, whereas the UK won't, so there will be different immigration laws.
This is conflating people with goods and trade again. What is the market for smuggling goods by road between Northern Ireland and the Republic and vice-versa? Negligible, not worth worrying about. It's an easier problem to solve than at the EU's other land borders because both Ireland and Britain are islands, it's impossible to transport goods between either the islands themselves or any other EU countries without going through a port with customs checks, be it a ferry port or airport.
Immigration is a separate matter, Ireland right now could be seen as a back door into the UK. It's no big problem though as Ireland isn't in Schengen so does immigration checks on everyone except, err, arrivals from the UK who also aren't in Schengen so do as well.