As a southerner who's ancestors on both sides of my family fought for the Confederacy you might say that I have a bit of a different perspective on this. Yes, some of my ancestors owned slaves, but that was only one part of my family tree. And yes, slavery was wrong, very wrong. Most of my ancestors were dirt poor and went to fight when their government told them to. The idea that extremely poor people would march off to war so that rich men could keep their slaves is laughable. They went out of a sense of duty and to protect what was theirs. And when they lost the war they actually lost everything, the rich ones AND the poor ones.
The thing that pisses me off the most is that most of the people bitching about the flag and about statues are white, and almost always from somewhere else. They don't see that thinking black people need white people to speak for them makes them FAR more racist than most southerners.