“We are a bookshop. Bookshops are all about ideas and tolerating different opinions and not about verbally assaulting somebody, which is what was happening," he said.
Mr Bannon, the former executive chairman of far-right website Breitbart News, grew up in Richmond, attending a private Catholic military secondary school there.
Mr Bannon is just one of a number of political figures associated with Donald Trump who has been confronted by members of the public in recent weeks.
It comes after Mr McConnell’s wife stepped in to defend him last month after he was confronted by protesters over the Trump administration's family separation policy at the Mexican border.
Elaine Chao, who serves as transportation secretary, demanded demonstrators leave her husband alone as she pointed at the group.
Mr Bannon - who was abandoned by his wealthy backers Robert and Rebekah Mercer in January - left his White House post last August after rumours of in-fighting in the West Wing over the summer.
He returned to be head of Breitbart, with the publication proclaiming the return of their “populist hero” on its homepage hours after his departure became public.
Headlines published on Breitbart under his watch over the years include “Does feminism make women ugly?” and “Birth control makes women unattractive and crazy”.
Mr Bannon described Breitbart as a “the platform for the alt-right” - a political movement which has been accused of racism, antisemitism and misogyny - in 2016.