Several people were injured after a car rammed into a crowd of people who were against the rally in Charlottesville.
A state of emergency has been declared to enable police to mobilise resources.
The “Unite the Right” march is against plans to remove a statue of a US Civil War general. President Donald Trump condemned the violence.
On Twitter, he said: “We ALL must be united & condemn all that hate stands for.
“There is no place for this kind of violence in America. Lets come together as one!”
Video posted on social media showed a car ploughing at speed into several slow-moving vehicles, which were surrounded by a densely packed crowd. Emergency services were pictured treating a number of people who were injured.
A witness said one girl got “tore up” after the car “backed up and hit again.”
Earlier, police fired tear gas against demonstrators and said that arrests had been made after a declaration of unlawful assembly at Emancipation Park.
The state of emergency allows local authorities to request additional resources if needed, the police department said.
The far-right protesters, some waving Confederate flags, carrying shields and wearing helmets, are angry about the planned removal of a statue of Gen. Robert E. Lee from Charlottesville.
Gen. Lee commanded the pro-slavery Confederate forces in the US Civil War of 1861-65.
The New York Times reports that some of them were chanting “You will not replace us,” and “Jew will not replace us.”
Anti-racism organisations such as Black Lives Matter have also held marches.
Sources: CNN, BBC.