Virgil Bryson was a free Black man who helped register voters in Jackson County after the Civil War in 1868. He was threatened by the early version of the KKK and supported by the Freedman’s Bureau.
Although much about Bryson remains unknown, BPR talked with local historians and community members about his story – and what we can learn from him today.
Ellerna Bryson Forney is a native of Jackson County, who is now 58. She has looked deeply into her family history, but she’d never heard of Virgil Bryson.
“I didn't know anybody named Virgil. I'm trying to get a hold of my uncle. He's 90 something years old. His memory's still pretty good,” said Bryson.
Through family research and working on the book, “When All God’s Children Get Together: A Celebration of the Lives and Music of African American People in Far Western North Carolina,” with Cherokee County historian Ann Miller Woodford, Forney has learned a lot about her family tree.
“It was interesting, but the ones that really knew the history, some of them really wouldn't even talk because you need to leave skeletons in the closet. I'll say good, bad or otherwise, that's what made us who we are. But some of the older ones did not get it,” said Forney. “They did not mind giving the family tree, but true history, you know, the things that you heard and some things I heard, I couldn't say because I was not there, you know.”
Virgil Bryson is mentioned in letters written by western Freedman’s Bureau representative Lt. George Hawley. The Freedman’s Bureau was installed after the Civil War to improve access to food, education and work for freed Black people and others. An office was set up in Asheville in 1866, explains historian Steven Nash.
“For a short time after the Civil War, Hendersonville and Asheville were occupied by United States forces, but then those forces were gone by the end of 1865. The Freedman's Bureau comes in slowly because the United States government, especially under Andrew Johnson, viewed the Freedmen's Bureau as unconstitutional. He believed it was an overreach of federal power,” said Nash. “And there was no mechanism put in place when the bureau was created to pay for it. It was only supposed to survive for the duration of the war and one year after.”
In 1860 before the Civil War, there were almost 13,000 people in Buncombe County – about 16 percent of the population was Black. That includes about 2,000 enslaved people and 100 free people. The numbers vary across Western North Carolina’s other counties from 28 percent Black in Burke County (2,371 enslaved people, 221 free) to 3.7 percent in Watauga County (104 enslaved, 81 free).
Nash explains after the Civil War, Western North Carolina had few roads and the railroad into Asheville wasn’t yet built.
“I think that that's one of the biggest problems that the United States military and Freedman's bureau experienced in Appalachia after the war was the absence of roads. In the case of Western North Carolina, the absence of a railroad meant that you have this sort of very small military presence, basically one cavalry regiment, one infantry… and they were responsible for Morganton west,” said Nash.
In 1867, a bureau was set up in Macon County in Franklin to cover the region West of Asheville. This was Lt. George Hawley’s office.
“What I see of Hawley in that area was that he was incredibly proactive. He was very much afraid that African-Americans were going to be essentially subverted back into a condition equivalent to slavery,” said Nash.
“He gets very involved in local court cases. He gets very involved in helping organize schools for African Americans in that area. Hawley, he basically writes at one point to his superior and it says more or less ‘the people in my district are going to be crushed.’ That's sort of paraphrasing, like they're going to be overwhelmed, like we need to help them,” said Nash.
“The bureau, particularly in the areas where there's no real good roads and communication networks, they're almost on an island and they have to adapt to the situation that they find locally. And when Hawley came to Macon County, he worked to try to help African Americans establish themselves as best as he could. It was not easy. There was violence and pushback in that district,” said Nash.
Local historian Barbara McRae researched Hawley’s work in Franklin too. McRae passed away in March 2021. She spoke with BPR in February
“One of the first things he had to do was to get orders for an office furniture for a desk and a table and some chairs, very bare bones,” said McRae.
Hawley’s letters are part of the Smithsonian Museum’s digitized records of the Freedman’s Bureau. Here’s more information about the Bureau. The letters are available online and detail the needs of community members such as efforts to organize schools for Black students. Hawley also reports violence.
“He did a lot of work in Jackson County because there were the stirrings of what would become the KKK, over there and especially in the Webster area. And some of the people that he was involved with like Virgil Bryson a former slave. He was a Freedman and he was attacked. He had come to Hawley's attention,” said McRae. “Apparently they were appointing people to register Black people to vote, Black men, and also as inspectors of the elections to make sure that nothing went wrong and this upset some of the more rabid white people.”
McRae is referring to the time before the 1868 election when Black men were able to register to vote before the passage of the 15th Amendment. McRae – who loved to read through old census documents – said there aren’t clear numbers for Black people in the area during this time.
“I have a feeling that in 1870, the census taker drastically under counted the Black people. I mean, there are people I just can't find and I've gone through the whole census,” said McRae.
“That decade, that decade was so traumatic. So turbulent, there there's so much movement, you know, there was the war. And then there was trying to figure out what to do after the war,” said McRae, referring to the time period after the Civil War.
Virgil Bryson was appointed to register those men to vote. But Bryson and people close to him were repeatedly assaulted and harassed, according to Hawley’s letters.
Here’s one example in 1868. Hawley writes: “ States outrages perpetrated on Virgil Bryson Freedman, living near Webster. Jackson. Co. by John Allison and William. Bumgarner. The former carried off his gun and threatened his life and the latter assaulted him and took away his horse.”
McRae explains that the men Hawley identified in his letters were influential – including the Jackson County Sheriff.
“Poor Lieutenant Hawley had a time because , I mean, one man could not possibly deal with four highly placed, prominent white men in Jackson County. They were never charged by any civil authority. And, Bumgarner himself was a former sheriff and there just was nothing that Holly could do except make reports,” said McRae.
Hawley also reported a potential lynching in Jackson County that he could not prove of a Black man named Henry Matthews.
His report archived in the Smithsonian reads:
I have the honor herewith to State, that it is reported that a number of citizens of Jackson Co. Supposed to be members of the "Ku - Klux - Klan", on the night of April 28" 1868, took from the Jail at Webster, a freedman who had been confined on charge of Stealing. It is stated that the crowd beat the freedman to such an extent that he has not since been heard of, and it is not known whether he is alive. It appears that it is the aim of all parties to prevent an investigation, as I have not thus far been able to learn the details of the affair, or the names of the parties.
He interviewed men in Jackson County about the incident including E.R. Hampton, who testified the crowd was pursuing Matthews. Hampton said Matthews had “broken out of jail” and that they chased him to the “Tuckaseige River.”
Barbara McRae dug into these stories and tried to track Virgil Bryson’s genealogy. McRae said in her research she found Bryson was the child of a man named Peter Gray. When he was freed, he changed his name to Bryson.
While McRae wasn’t able to learn everything about Bryson, the Women’s History Trail in Macon County is organizing a library of her research so that scholars will be able to pick up where McRae left off.
The people and events reported by George Hawley are still felt today. The names remain local. Bryson, Allison and Bumgarner can both be found on street signs across the region - and voting rights continue to need protection.
Ellerna Bryson Forney found other twists and turns in her family genealogy research.
“For instance, like there's supposedly two sets of Brysons in Sylva. There's a darker side and a lighter side, you know what that means, you know? But then when I went through the death records at the courthouse, there was two David Brysons, who had two women pregnant at the same time. And both of those women named their sons, David and once I brought that up, everybody clammed down, you know, everybody clammed down quickly on both sides,” said Forney.
“I am curious to know more about this man myself. I pray and hope I can find somebody that knows, and that maybe even Richetta my cousin, she did our family tree. She says there's one of the grandfather's brothers that had, I think she said seven children. But she could never find the names of any of them or his great-grandfather,” said Forney.
“He had seven children by two wives, but she could never find the names of any of them. And some of them, it was said through the family that they had passed as white. So they lived their life as such, you know what I mean?,” said Forney.
Overall, Forney feels not enough people know about the history of Black people in Western North Carolina from slavery to local civil rights efforts. She recalls a gathering to talk about Jackson County history where people said there were never any racial issues in the county.
“I had to shut it down because they said that in Sylva you did not know that stuff like that was going on. Because Sylva was okay. Sylva was fine. It was okay, you know, for them to call Mr. Jim, N** Jim. It was okay for people to walk miles and miles to clean her house and take care of the children when their children are at home and this and that, but that you didn't know that civil rights stuff was happening based on Sylva,” said Forney.
“The interesting thing I find is that the folks that the Black people that were here before integration are closer even now with each other, than the ones that were [here] after integration. And I believe that's because they had to stick together and they were together, you know, in school, church and play. Cause even if they come into town, they find each other, but my generation, I might see y'all. I may not and that's okay,” said Forney.
Forney says she believes the community is closed off in a way because it is safer. During the time of the Freedman’s Bureau, early members of the Ku Klux Klan attacked people like Virgil Bryson in order to stop him from registering others to vote.
In 1868, their efforts did not work. Steven Nash says that Lt. Hawley and Virgil Bryson’s work registering Black men to vote – and others like them across the state - is what won the election for Republicans.
“He got involved as the bureau did and helping to create the boards of registration for voters in 1867 and 68 when Black men were allowed to vote for the first time, even before the 15th amendment, they were registered to vote in 1867. And then they voted in the state elections of 1868, which swung the state to the Republican party,” said Nash.
Republican William Holden won the gubernatorial election that year and the Republican party also controlled the state legislature.
The results convinced national policy makers that Reconstruction had succeeded and prompted the closure of the Freedmen’s Bureau outposts in the mountains – taking away a line of defense for Black people.
For a while the government held and Black rights expanded. In part of North Carolina, a “fusion governments,” of populist and Republican parties lead Black men to be elected to positions of power.
Here’s Steven Nash again:
“African Americans continue to sort of play a role somewhat through the 1880s. Then in the 1890s, you get another crazy North Carolina electoral cycle with Republican and populist fusion, which brings African Americans back into the heart of the debate and leads to Wilmington and across the state is part of the cycle of violence is geared against African Americans and a result of Republican Fusion that drifts to disenfranchisement,” said Nash.
Nash says its complicated but the changing political landscape during the 1870s ultimately lead to another rise of violence and voter suppression in the state which culminated in 1898.
The Wilmington Massacre that year during which as many as 250 people were killed. There were also incidents across the state. The Ku Klux Klan and other paramilitary groups like the Red Shirts began a campaign of violence to deter Black people from voting. Their tactics included lynching and propaganda such as in the News and Observer newspaper.
In Franklin, the same year Mitchell Mozeley was lynched. Local historians in Franklin recently held a memorial on the 123rd anniversary of his death. At the event, the connections between violence and voting righters were explained.Nash agrees.
“The violence of Reconstructions is political. A lot of local historians want it not to be – and I’m painting with a broad brush but that is an observation that I’ve had,” said Nash.
In 1898, the racist groups were successful in destroying the rights that had been won by Black politicians and community leaders – and instead pushed the country toward voting literacy tests and other segregation laws.
“One of the things that the Ku Klux Klan was very good at during reconstruction was essentially to convince the world that they did not exist,” said Nash. “The claim was that [Black people and others] were doing these things to themselves to garner sympathy from the federal government to preserve military presence and reconstruction. Nowadays people will call it fake news, right?”
For Nash the tactics of violence and misinformation are the same tactics still being used today such as in the January 6th insurrection.
Ellerna Bryson Forney agrees. She sees the January 6th insurrection as a repeat of that violence.
“There is just no surprise. You know, the thing that happened at Washington, that's not a surprise to most of this. What is a surprise is no one is being held accountable,” said Forney.
“You know, it's not a surprise that there's Klansmen. There's still some skinheads around here. I thought they were gone, but they're still here. We know that they are here [and] we know who some of them are. And I thank our last president for his service. Cause he took the sheets off a lot of them,” said Forney.
She says there was a reason that there weren’t many local Black people at the Reconcile Sylva marches against the Confederate monument in the summer of 2020. Forney spoke at a rally in September after Jackson County commissioners had already decided not to remove the monument.
“The sad part about it is it's not new and we knew about it, but our parents, mostly our grandparents were so protective that they, like I said, they wanted us to stay in our place,” said Forney.
Forney wants to highlight everyday racism of name-calling and discrimination that has worked it’s way into the systems that organize and govern society. That’s called systemic racism.
“And that's why it's so important that right now to have, white people helping because I don't think anybody would listen to otherwise. I know the political machine, I don't think they would listen to just people of color or poor people,” said Forney.
While the violence toward Virgil Bryson isn’t surprising for Forney, his prominent place in the community also doesn’t come as a shock.
“I think it's awesome. And I never doubt it because I think every era has a powerful black person and we may not know who they are. It doesn't surprise me, but it's nice to know,” said Forney.
“I believe God puts people different people in different places at different times. And, and it's just so nice to know that there was somebody that was bold enough to get out there and got out there as much as they could. I think that is just fantabulous,” said Forney.
While the full history of Virgil Bryson may be unknown, thanks to Forney, McRae, Nash and others the history of this region has been documented to help better understand how we got here.
