I'm not quite sure why a 'comedian' who tells 'jokes' about such topics as cerebral palsy, suicide, disabled children, Downs syndrome, Jimmy Savile and the kidnapping of a child is worthy of any praise whatsoever, to say that it's just his act is a pretty poor excuse
Perhaps he should use some of his comedy genius to tell a few side splitters about gays or muslims, now that would bring some headlines I'm sure
That reaction is the reason why Boyle is so infamous; he's only popular through infamy.
Some people find that insidious and repulsive, others find that humorous. Live and let live is what I'd say. Besides, you have no proof he meant any of it.