Charlottesville restaurateur threatened after he boots patrons for Nazi salute

Charlottesville, Virginia (CNN) Brian Ashworth didn't want to deal with the mess descending Saturday on Charlottesville, Virginia , so he shut the doors of his popular breakfast-and-barbecue joint -- only to get a taste of the hate on Sunday.

He was in the office when one of his employees walked back to tell him that a woman and four or five men, who appeared at least sympathetic to the previous day's "Unite the Right" rally, were in the dining room of Ace Biscuit & Barbecue, less than a mile from where Saturday's rallies began.

Annoyed, Ashworth walked out to survey the situation.

Some members of the groups had what Ashworth described as low-quality prison or gang tattoos. One wore a "Make America Great Again" shirt, and another had a shirt promoting the British white nationalist punk band, Skrewdriver. A third man sported a shirt that said, "Pinochet Helicopter Company," a reference to former right-wing Chilean dictator Augusto Pinochet, accused of tossing Communists and other political opponents from aircraft.

There was only one other couple dining at the time, outside, and the group was behaving, minding its own business. They didn't appear to be bringing any of the rally's violence to his eatery.

"One employee said to me, 'Brian, be cool. Let them eat,'" the restaurant owner recalled. "We served them against my personal taste."

Ashworth even engaged in a benign conversation with one of the men who went outside to smoke. The man complimented Ashworth on the sausage Ace uses in its biscuits and asked if it was from a local farm. Ashworth replied that it was made in-house.

JUST WATCHED Videos show car crash into protesters Replay More Videos ... MUST WATCH Videos show car crash into protesters 01:13

The group finished up, paid and walked outside, where they began "loitering and smoking" by their cars in front of his store, apparently waiting for a friend, Ashworth said. By then, the restaurant was full of customers and there was a line out the door.

Suddenly, the young man in the Skrewdriver shirt threw up a Nazi salute, which the others reciprocated, he said.

"That was it. Oh my God, are you kidding me? 'Get out of here. You've got to go,'" Ashworth told them, admittedly in unkind words. "I could've trimmed off some of the f-bombs ... but I was pretty heated."

"It was all I could do to just not explode on somebody," he said.

He witnessed them give a Nazi salute to one another, then immediately told them he doesn't accept that here and that they must leave. — Alexi Garrett (@AlexiGarrett) August 13, 2017

They countered that they had rights, and Ashworth conceded they had rights but said, as a business owner, there are federally protected groups his restaurant must serve, and they're happy to, but he reserves the right to deny service to other groups. His employees don't have to serve them, he said.

"They're on the 'don't' list," said Ashworth, who has been in business almost five years.

The woman in the group called him "limp-wristed" and another pejorative for gay men. And then came the threat: "This isn't over. We'll be back."

Ashworth's customers appreciated him taking action. Still, he's concerned.

"I got a round of applause from the customers who saw me throw them out," he said. "A round of applause is good, but it doesn't keep anybody safe."

Ashworth called the police to request a squad car post up outside his restaurant, but was told the police department's resources were too thin, which is understandable, he said.

He decided to close for the day, for the safety of his staff and customers.

'I just want to sell some food'

JUST WATCHED On GPS: Bridging America's racial divide Replay More Videos ... MUST WATCH On GPS: Bridging America's racial divide 04:17

Charlottesville, like many American cities, has both good and bad history, Ashworth said, and he's angry that people have chosen to exploit his city for their own twisted aims.

"This isn't Charlottesville. These people are not representing us," he said. "The bad history is behind us, and we're trying to keep it there."

Ashworth worries that people like Sunday's customers will keep popping up in Charlottesville and around the country because today's tumultuous political climate provides them a platform, he said.

More worrying, though, is the threat, the husband and father said. One of the guys in the group was "dressed like an everyday dude," which is concerning because Ashworth wouldn't be able to pick him out as easily as he did the others, he said.

Ashworth brought his two .45-caliber pistols to work Sunday because of Saturday's violence. He supposes he'll keep bringing them to work until he feels safe again.

"I just want to sell some food and make a few bucks," he said. "I don't want to be the guy who got shot by a f**king Nazi."