Mr Holbrook was cautioned by police after the incident in Blackburn but the much-loved local clairvoyant said he had never hit anyone before in his entire life and, although he was embarrassed to be taken into custody, said he stood up for his beliefs.
He said: "The man was swearing, was incredibly rude and had been upsetting the audience all night with offensive remarks.
"There were people in the audience visibly shaken and in tears as they received messages from loved ones and he kept interrupting with disrespectful remarks. He was clearly distressing everyone there."
The incident happened when the audience were leaving the venue.
Mr Holbrook, 39, who has been described as Britain's most accurate medium, added: "I'd just had enough at the end of a very long night."
Mr Holbrook has raised thousands of pounds for Wakefield Hospice alone, and collects for local charities at every performance.
He added: "I am not a violent man, but it's very difficult when you see someone causing so much distress to people going through an emotional experience."
To comment on this story, click here