Anger over lollipop man's high-five ban