By and large probably the same people who jump for the lift call button when the doors are nearly closed (usually with a false apology when the doors are reopened) rather than waiting for the next one and pressing it once closed. In some buildings the call button doesn't reopen the doors if closure has started unless they are only closing because there was no call for the lift, I wish all lifts were like that.
Yes that is irritating especially when you are already in the lift and think you have got the lift to yourself.