Race, History, and Memories of a Virginia Girlhood

We stopped first at the cemetery. My brother had picked me up at the Philadelphia airport, and we had driven south and west from there—to Baltimore and Frederick, then down through the hills of the Blue Ridge, past the confluence of the Shenandoah and Potomac Rivers at Harpers Ferry and into the Valley of Virginia. Civil War country. The route of the Antietam and Gettysburg campaigns. The site of John Brown’s incendiary attempt to foment a slave uprising. The place where we grew up.

Apart from one brief drive-through, I hadn’t been back in nearly two decades—not since a visit the year after my father died. Now we could see next to his grave the dirt already unearthed to make a place for my stepmother’s ashes the next day. We had come for her funeral and in my father’s memory.

I had attended many burials here. The family plot houses uncles, aunts, grandparents, and great-grandparents, but no graves nearly as old as those dug soon after the nearby chapel was built in the 1790s. Edmund Randolph, the U.S. secretary of state and the nation’s first attorney general, a Virginia governor and a member of the Constitutional Convention, is a few yards away, surrounded by a crowd of Randolphs, Pages, Burwells, and Carters—members of the First Families of Virginia who had migrated to this northern end of the Shenandoah Valley when the children of the 18th-century Tidewater gentry began to seek new lands and new opportunities. Nestled behind what has come to be known as the Old Chapel, in the quiet of an isolated crossroad, the beautiful little cemetery is so small that no one is much more than a stone’s throw from everyone else.

My brother and I, the only visitors, wandered, reading epitaphs that called up Virginia’s storied past or reminded us of figures from our childhood: the leader of my Girl Scout troop; a teacher from our elementary school; a classmate’s mother, who was an extraordinary horsewoman; and my father’s drinking buddy and his long-suffering wife.

But I wanted to be sure my brother saw one plaque in particular. I remembered its words dimly and had perhaps even tried to forget them altogether. But now here I was again, and I needed to remind myself. I knew it was at the back of the oldest part of the cemetery, and there I found it, partially hidden by leaves and vines, and covered with lichen that nearly obscured its inscription. But I could still read:

I.T.G. was my grandmother. In 1957, I was 10 years old. We both lived here.

There is a monument to the Confederate dead in this cemetery; there are markers for unknown Confederates killed in skirmishes nearby. That is complicated enough. But what is to be made of this invocation of slavery offered during my own lifetime? Of this tangible link between who we are now and who we were more than a century and a half ago? Between attitudes and practices that were taught to me as a child and the person I could or would become? Between the Virginia of 1957 or 1857 and the one that—on the very day in February 2019 when I stood in the Old Chapel cemetery—was confronting the crisis of a governor whose medical-school yearbook page had just been discovered to have included a repugnant, racist photograph of a man in blackface and another in Klan robes?

Explore the August 2019 Issue

Check out more from this issue and find your next story to read.

View More

What had my grandmother been thinking? The language and tone are right out of what is sometimes called the “moonlight and magnolias” version of the pre–Civil War South—the romanticization of plantation culture, the erasure of slavery and its brutalities. Here on the plaque we have not slaves but “servants,” “faithful and devoted” rather than subjugated against their will. The words describe affectionate and appreciative masters—a benign domination, not the cruel system of physical brutality, lives stolen, and human beings bought and sold that we know slavery to have been. The marker declares a nostalgia for an era, a “way of life” that is “gone forever.” Gone, we might say, with the wind. Margaret Mitchell’s book, published in 1936, and the movie that followed in 1939 were still exerting their influence in 1957, as they have well into our own time. And I think, too, of the “Mammy” memorial approved by the U.S. Senate in 1923, and the numerous “Mammy” monuments proposed at that time across the South.

But this wasn’t 1923 or even 1936. It was 1957. Why did my grandmother go to the trouble—and expense—of erecting the plaque at this particular time? It was more than just an expression of views that had persisted since the ideology of the Lost Cause and the idealization of the Old South had solidified among white people in the years after Appomattox, views that she had been indoctrinated to embrace from the time of her birth in Tennessee in 1894.

The year 1957 was a crucial time in Virginia and in the South generally. Three years earlier, the Supreme Court had struck down school segregation in Brown v. Board of Education, and the implications of that decision were beginning to become clear. In September 1957, nine African American students entering the previously all-white Central High School, in Little Rock, Arkansas, were greeted by a segregationist mob supported, per the order of the governor, by the state’s National Guard. President Dwight Eisenhower was compelled to mobilize the 101st Airborne Division to enforce integration and uphold the law. Closer to home, Senator Harry Byrd—who lived just a few miles from the Old Chapel graveyard—had called for “massive resistance” to the Supreme Court’s ruling. Engineering a plan to close rather than desegregate schools, Byrd and his aroused followers were transforming the 1957 Virginia governor’s election into a referendum on race and, in a broader sense, on the morality and legitimacy of the white South’s discriminatory assumptions and practices. In the face of such controversy and opprobrium, the plaque invoked a redemptive narrative of the Southern past, one designed to reassure a society under siege that it was not just right but righteous. It proclaimed a virtuousness fashioned out of a fantastical history, a virtuousness to be reinforced by the generous act of noticing and remembering that the plaque was meant to be.

But why did my grandmother choose this graveyard and this statement to subdue her unease about the challenges to her taken-for-granted world—her unease, I imagine (and hope), about that very world itself? Why a plaque? It wasn’t filling a gaping need. In both its language and its very existence, it protests too much.

Local circumstances had generated an additional motivation. The far end of the cemetery—the land beyond the plaque—had housed graves of enslaved people, though their locations and markers had all but disappeared. As a child, I remember hearing discussions among the adults in my family about how growing demand for graveyard plots had led to a consideration of extending the white cemetery into the area the slave cemetery occupied. This was not understood as sharing—and certainly not as integrating—the space. Instead, the older graves would essentially be erased from the landscape and from the minds and memories of the white church and its members.

But not from my grandmother’s. The puzzling plaque represented her discomfort with (though not, significantly, any overt objection to) a plan to so callously disrespect the dead. She meant to remember and memorialize them with a permanent stone marker that would not rot and disappear. But as a white southern woman imbued with a conventional understanding of the past and its racial practices, she was memorializing a world that had never been. And she was perpetuating a narrative about race that has continued to poison Virginia and the nation more than a century and a half after slavery’s end.

Virginia has a long history to confront. Our nation’s experience with slavery began there, when some 20 captive Africans arrived on a warship in Jamestown in 1619. Black bondage existed in Virginia for close to a century longer than black freedom has. Slavery made colonial Virginia prosperous, creating a plantation society founded on tobacco production, social and economic stratification, and unfree labor. It also produced a class of white owners whose daily witness to the degradations of bondage instilled in them a fierce devotion to their own freedom. They were determined to be the masters not just of their households, their estates, and their laborers, but also of their society, their polity, and their destiny. George Washington, Thomas Jefferson, James Madison, James Monroe, George Mason—slaveholders all. That so many of the Founding Fathers, including the leaders of the Revolution and the authors of the Declaration of Independence, the Constitution, and the Bill of Rights, were slaveholders is both an irony and a paradox. As Samuel Johnson remarked with scorn for the revolutionaries across the Atlantic: “How is it that we hear the loudest yelps for liberty among the drivers of negroes?”

But in another sense, as the historian Edmund Morgan argued so powerfully nearly half a century ago, slavery and freedom were not at odds, but integrally intertwined, even mutually constitutive. It was the unfreedom of 40 percent of Virginia’s population that made the liberty of the rest imaginable as well as materially possible. The economic viability of both the colony and the new nation depended on slave labor. And so did the viability of the Revolution’s political experiment and the Founders’ republican vision. The Virginia gentry could countenance the extension of freedom to some men because it was withheld from others; the exclusion of a portion of the population from the polity, their subjugation and control, made possible the advocacy of equality for the rest. The nation conceived in liberty was also the nation conceived in slavery. The state of Virginia and the country it did so much to create were born out of a set of conflicting commitments that have destabilized the republic ever since. Yet the presence of this paradox at the heart of the Founders’ vision is perhaps the good news, for freedom has had its own driving logic, has claimed its own agenda, has propelled us over time toward better angels.

In the earliest years of colonial Virginia, the nature and extent of bondage remained undefined. While most Africans were unfree laborers, some exercised liberties that later would be unthinkable. Distinctions between white indentured servants and bound black laborers became firm and rigid only over time. As the system of slavery was established during the 17th century, perpetual inherited unfreedom gradually became the exclusive fate of Africans and their descendants.

A century separated the legal codification of slavery in Virginia and the beginnings of the revolutionary movement. The Founders had no memory of a society without bondage and no experience of a world where blackness and degradation had not been conjoined—where white supremacy and black inferiority had not been enshrined in both law and culture. The racial definition of American slavery placed yet another contradiction at the new nation’s heart, one that transcended the political difficulties of reconciling slavery and freedom. What did it mean to be human? This question posed a fundamental challenge to the execution of the laws: Was a slave a person or property? Could slaves be seen as having free will and thus legal accountability for their actions? Antebellum Southern judges struggled with these inconsistencies. “A slave is not in the condition of a horse,” a Tennessee judge insisted. “The laws [cannot] … deprive him of his many rights which are inherent in man.” But a North Carolina judge disagreed, baldly underscoring the ultimate logic of slavery: “The power of the master must be absolute, to render the submission of the slave perfect … Such obedience is the consequence only of uncontrolled authority over the body.” Commitment to a republican form of government was incompatible with the absolute power that defined the system of slavery.

This dilemma was more than a problem of law or government. It was about human identities, emotions, and values. A rallying cry of the 19th-century abolitionist movement would capture it well, with the words of a slave imploring: “Am I not a man and a brother?” The racial definitions of slavery required white Southerners to resist that entreaty every day. Yet denial was not always possible. The force of common humanity could at times reach across the chains of the color line to generate compassion, guilt, doubt, and of course desire. Racism muted these human instincts, even as laws banning interracial sex, prohibiting manumission, and outlawing criticism of slavery acknowledged their existence. Thomas Jefferson’s attraction to Sally Hemings, with whom he fathered five children, embodied the tragedy present at the very creation of American freedom.

White Virginians struggled with the idea of slavery even as they exercised its required cruelties and reaped its benefits. Jefferson observed that the existence of the institution was like having a “wolf by the ears.” The threat it posed not just to the ideals of white Virginian society but to its very security and survival became vividly real in August 1831 when Nat Turner, an enslaved preacher from Southampton County, asserted his claim to freedom. The uprising he led resulted in the deaths of 55 white people as well as uncounted—some estimates indicate more than 100—black people attacked by both mobs and the militia called up to quell the revolt. More than a dozen captured rebels, including Turner himself, were executed.

The revolt provoked near-hysteria among white people not just in Virginia but across the South. Could there be a Nat Turner in every household? Could black people, subjugated and assumed to be inferior, share their masters’ longing for freedom and possess the fearlessness to try to attain it? When the Virginia legislature convened just weeks after Turner’s execution, the lawmakers took up the future of slavery in unprecedented debates that reflected the tensions at its core. In the words of one delegate advocating slavery’s elimination, “It is ruinous to whites—retards improvement—roots out an industrious population.” Another added that it threatened to bring about more such “melancholy occurrences” as “the tragical massacre at Southampton.” But nearly all agreed that emancipation would somehow have to be linked to deportation and African colonization, because living with a population of freed black people was unthinkable. The debates were never about black justice, but about white safety and prosperity.

Slavery did not end in Virginia in 1832. It would take three more decades and hundreds of thousands of lives lost to bondage, and later to war. Indeed, the debates in the legislature yielded new laws intended to strengthen slavery’s hold. Yet these discussions demonstrated that white Virginians felt profoundly and persistently uneasy about a way of life the state had known for two centuries.

In the aftermath of the Virginia debates and the doubts and divisions they exposed, voices in the state and beyond began to defend slavery more vigorously. A proliferation of “positive good” arguments insisted that the institution was not just acceptable and justifiable but the best possible arrangement economically, politically, and even morally. Virginia could no longer afford the kind of ambivalence Jefferson had exhibited, proclaiming slavery an evil yet living on its fruits. As the historian Eugene Genovese put it: “One generation might be able to oppose slavery and favor everything it made possible, but the next had to choose sides.” Nat Turner had made that choice seem more urgent. By the 1830s, slavery was in many ways weakening in Virginia, with the decline of the tobacco economy and the sale and forced migration of thousands of enslaved people from Virginia to the cotton economy of the Deep South. Yet Virginians played prominent roles in slavery’s emerging ideological defense.

And it would not be long before Virginians would be asked to choose sides again. As secession fever mounted across the South in the wake of Abraham Lincoln’s election in November 1860, Virginia remained cautious. South Carolina rushed headlong into separation from the union, voting to secede before year’s end, but the Virginia convention called to consider the issue resisted strenuous appeals from its neighbors to join a proslavery Confederacy. Virginia’s decision was all-important, for without the Old Dominion and its human, industrial, and agricultural resources, the new nation had little chance of success. Virginia’s white population of 1.1 million was the largest of any Southern state, and it ended up supplying the most soldiers to the Confederacy. But some advocates for secession both within and beyond the state saw another element at stake in Virginia’s position: They worried that if Virginia did not emphatically embrace the slave republic, it was likely to abandon slavery altogether in the decades to come. These observers noted the emergence of mixed agriculture in northern Virginia, an economic system increasingly like that of the Middle States rather than the Deep South. And they noted as well the decreasing proportion of slaves in the population as Virginia masters sold their unprofitable bondsmen south to supply labor for the cotton plantations of Alabama, Mississippi, Louisiana, and Texas.

Yet after initially voting against secession in the early spring of 1861, Virginia responded to the firing on Fort Sumter and Lincoln’s call for an army of 75,000 men to march against the South by declaring itself all in for the Confederate cause. And Virginia did not just join the Confederacy; it became the capital and the chief battleground. The decision would come at enormous cost. The war would destroy lands, lives, and the world white Virginians intended to preserve.

The Civil War destroyed the legal foundations of slavery, but the racism that had reinforced it for so long persisted. Emancipation meant black Virginians were no longer property, but white people pushed back forcefully against change, bringing an end to Reconstruction and retaining control over social, economic, and political arrangements in the state. The North’s commitment to overturning the old order of the South waned in the face of white intransigence, and a new system of subjugation and injustice emerged to take slavery’s place.

White Virginians of the late 19th and early 20th centuries congratulated themselves on the harmony of race relations within the state. Political and business leaders disavowed the race-baiting that emerged farther south and endeavored to maintain a system of paternalistic control in order to ensure the perpetuation of racial supremacy. Force could be held in reserve if black people accepted their assigned roles within an enduring racial hierarchy. Yet in its essence, this was a system that rested on spoken and unspoken threats of coercion and violence. Between the end of Reconstruction and 1950, 84 lynchings occurred in Virginia—significantly fewer than in any other southern state, but more than sufficient to transmit the message of racial terror.

Extralegal violence was one guarantor of order, or at least the order the white South deemed necessary. But Virginia’s white elite preferred to uphold its control with the seemingly more legitimate and defensible instruments of the law. In Virginia, as across the South, newly devised Black Codes regulated postwar social arrangements, and vagrancy laws preserved white power to coerce labor. The public whipping post—used overwhelmingly for black offenders—marked disturbing continuities with slavery’s practices. Tellingly, it provoked not just strong black opposition across the state, but debate among white people who worried that its highly visible brutality undermined the idealized narrative of racial tranquility. In 1898, the Virginia legislature at last voted to abolish the whipping post.

Yet just as one instrument of violence and coercion was abandoned, others took its place. A modernized form of unfreedom, the penitentiary imprisoned black people at rates dramatically higher than the rates for white people. In 1893, one in every 5,000 white Virginians was incarcerated; the figure for black Virginians was 7.5 out of every 5,000. And African Americans were almost exclusively the victims of the emerging convict-leasing system, which rented prisoners to owners of Virginia’s quarries, mines, canals, and railroads. It has been called slavery by another name.

For a time during Reconstruction and the years that followed, freedmen in Virginia voted and even served in the legislature, but by the turn of the century, a variety of measures, including literacy tests and poll taxes, worked to exclude African Americans from the ballot box. A system of racial separation that came to be known as Jim Crow was set firmly in place, segregating schools, transportation, and public entertainment, and forbidding interracial marriage. In 1896, the Supreme Court in Plessy v. Ferguson proclaimed its approval. Yet this new “separate but equal” doctrine as the supposed logic and justification for segregation represented just another in a long line of distortions and deceptions the white South embraced. “Separate” was—and was designed to be—unequal. That was the point.

But elite white Virginians created a narrative of an invented past and a distorted portrait of their own time to reassure themselves of the justice of their social order and of their own benevolence. The cult of the Lost Cause embraced an apocryphal history suffused with nostalgia for a world of valorous Confederates, kindly masters, and contented slaves. And it mischaracterized the present, extolling the “Virginia Way,” a distinctive form of Jim Crow in which blacks and whites lived peaceably together in lives of “separation by consent,” in the words of Douglas Southall Freeman, a Richmond newspaper editor and renowned Robert E. Lee biographer. Freeman acknowledged that this was a social order designed to perpetuate “the continued and unchallengeable dominance of Southern whites,” who, he told his readers, would work to provide assurance of safety and security to black Virginians in return for their acquiescence in the status quo. “Southern Negroes,” he explained, “have far more to gain by conforming than by rebellion … by deserving rather than demanding more.” Elite white Virginians had inherited a legacy of gentility accompanied by the imperatives of noblesse oblige; Virginia’s black people, in turn, were “inherently of a higher type than those of any other state.” Nowhere else, Freeman insisted, “are the Negroes more encouraged through the influence of friendship for and confidence in them, on the part of whites, to be law abiding and industrious.” But never to claim equality.

I grew up in that Virginia, on a 500-acre farm a mile and a half from the town of Millwood, home in the 1950s to about 200 people, most of them black. My childhood friends—all white—lived on surrounding farms like ours. Almost all the properties had names, and although our house dated from the early 19th century, many had stood since the time of the American Revolution. Saratoga had been built by Daniel Morgan in the 1780s to commemorate the 1777 battlefield victory. Carter Hall, Pagebrook, and Long Branch were all erected after the Revolution by scions of the Tidewater gentry. In Millwood itself, most African Americans lived in dwellings that lacked running water.

This was a world in which silences distorted lives, and falsehoods perpetuated structures of power rooted in centuries of injustice. This was still the Virginia of the poll tax and of segregated schools. Every adult black person I knew worked for white people as either a laborer or a domestic. Nevertheless, this was not the Deep South. The myth of “consent” required that white people be able to claim—and convince themselves—that black people happily accepted their assigned places. Daily life did not include colored and white signs on water fountains or in waiting rooms. In my small rural community, people just knew. Or learned. Even my own house was segregated. The African Americans who cooked and cleaned ate in the kitchen. We ate in the dining room, except for Sunday supper, when the workers had the evening off and my mother, who could scarcely cook at all, contrived to produce a meal, while we all longed for Monday and the cook’s return. Behind the kitchen was a separate servants’ bathroom. When I used it once, my mother reprimanded me for invading their privacy.

I never witnessed physical cruelty toward black people in my community; I never heard the N-word. Prejudice was hidden beneath a surface of politeness and civility that scarcely masked the assumption of superiority, of greater intelligence, of entitlement. Amused condescension, mockery cast as patronizing affection, often inflected white attitudes toward “the colored people.” Yet as I think back, there was a nervousness about the laughter, a need for mutual reassurance as the adults around me recounted tales of people so close at hand yet so mysteriously and frighteningly different from themselves.

Racial custom was carefully yet obliquely taught. It encompassed all the contradictions that had confronted white Virginians for centuries. We grew up in the constant company of human beings who were central to our lives, yet we somehow came to understand that an unspoken hierarchy required our distance—both physical and emotional—from them. An African American man who worked for my family for decades did everything from shining shoes to mowing the lawn to driving us around the county—to school, to piano lessons, to scout meetings. He was as present in my childhood as my brothers and my parents. He quizzed us on state capitals and the order of the presidents, made sure we remembered our lunch boxes and homework, and told us jokes and riddles. He always spoke not of “driving” us here or there, but of “carrying” us, a usage that to my child’s ears communicated a kind of concerned protectiveness. But I scarcely knew anything about his own life. He had a daughter not far from my age, but I rarely saw her, because she of course went to the segregated black school. I never even knew where it was.

We had—and were taught we deserved—better houses, better education, a better future. Yet at the very same time, we were learning in school that our nation was founded on the belief that all men are created equal; we were hearing in our all-white church that we were all the same before God. “Join hands, disciples of the faith,” the hymn commanded, “what’er your race may be. / All children of the living God are surely kin to me.”

For many white southerners of my generation, a life-defining question has been how long it took us to notice. When did the contradictions become troubling? When did they become unbearable? What was the moment of epiphany, the circumstance that made the inconsistencies undeniable? When did it become imperative to confront the legacies of slavery and segregation, to be honest with ourselves and one another and purge the untruths that, like malignancies, had permeated our society and our lives? “It’s that obliviousness, the unexamined assumption, that so pains me now,” the photographer Sally Mann has written about her 1950s Virginia childhood. “How could I not have wondered, not have asked.” For her, going north to school and encountering the writings of William Faulkner

For many, the civil-rights movement and the racist pushback to it served as a wake-up call, forcing an end to silences, exposing the violence on which Jim Crow rested, and removing the veneer of timeless inevitability that whites had strived to create. And a growing assertiveness by black Virginians made it ever more difficult to maintain the fiction of separation and subjugation “by consent.”

I was 9 years old when the news reports about “massive resistance” and battles over segregation made me suddenly realize that it was not a matter of accident that my school was all-white. I wrote an outraged letter to President Eisenhower—outraged because this wasn’t just, but also outraged that I only now understood, that I had been somehow implicated in this without my awareness. I have wondered whether I was motivated in part by my growing recognition of my own disadvantage as a girl whose mother insisted I learn to accept that I lived in a “man’s world.” I resented that my three brothers were not expected to wear itchy organdy dresses and white gloves, or learn to curtsy, or sit decorously, or accept innumerable other constraints on their freedom. I was becoming acutely attuned to what was and wasn’t fair. And because my parents seemed to take for granted that this was both a white world and a man’s world, I took it upon myself to appeal—without telling them—to a higher power: “Please Mr. Eisenhower please try and have schools and other things accept colored people,” I wrote. “Colored people aren’t given a chance … So what if their skin is black. They still have feelings but most of all are God’s people.” And I acknowledged the accident of my own privilege: “If I painted my face black I wouldn’t be let in any public schools etc.” I seem to have figured out “etc.” before I recognized the realities of the racial arrangements that surrounded me. And, curiously, I framed what I had recognized as the contingency of race and the arbitrariness of my own entitlement by invoking blackface.

Many moments in the years between Brown and the Civil Rights Act of 1964 and the Voting Rights Act of 1965 imposed new truth on white Americans. It was a fundamental strategy of the civil-rights movement to create such occasions. As Martin Luther King Jr. explained, peaceful demonstrators would force the poison out and make it visible. The beating of the Freedom Riders in 1961, the jailing of child protesters in Birmingham in 1963, the clubbing and gassing of John Lewis and other marchers on the bridge in Selma in 1965—all were vividly displayed to the American public in a manner TV had made newly possible. These horrors challenged complacency and compelled many Americans to ask themselves, “Which Side Are You On?”

But we know now, half a century later, that none of this was even close to enough to overturn centuries of racial injustice. By the time of King’s murder, in April 1968, little sense of consoling moral clarity remained. King himself had embraced controversial new positions—taking his protests into the North, insisting on confronting economic inequality, and speaking out against the Vietnam War. Many African American activists had broken with King, advocating Black Power rather than racial reconciliation, abandoning nonviolence, and denouncing King as an accommodationist. The nation has forgotten that a poll taken in 1966 revealed that nearly two-thirds of Americans had an unfavorable opinion of the civil-rights leader.

His death muted voices of criticism and opposition and punctuated the end of an era: Montgomery to Memphis. Like King himself, those years were transformed into a myth of national redemption—and, dangerously, of work complete. Now that the era seemed safely over, no longer a threat to the status quo, we could celebrate it. The story of the civil-rights movement became, as Julian Bond commented with no little bitterness: “Rosa sat down, Martin stood up, then the white folks saw the light and saved the day.”

Bond was right that the movement compelled many white people to begin to see the light. He was also correct that there remained a great deal we had scarcely begun to see, much less do something about. But the emergence of the oversimplified and consoling narrative of the civil-rights movement gave many white Americans permission to not look further. The nation retreated into a kind of post-’60s fatigue, as Richard Nixon and Ronald Reagan ascended to office with the support of a not-so-silent majority that believed black people had progressed quite far enough. Many white Virginians were comforted to return to the traditional narrative of racial harmony, now imbued with far greater legitimacy after the drama—and the accomplishments—of the civil-rights movement.

This was the world in which Governor Ralph Northam came to political consciousness. He was born in 1959 to a prominent family on Virginia’s Eastern Shore, the son and grandson of circuit judges and the great-great-grandson of slave owners. Northam was 12 when the schools in his county desegregated. “We didn’t see color,” he has said.

Others growing up in that time and place, particularly African Americans, did. Bryan Stevenson is a lawyer, a social-justice activist, and the founder of the National Memorial for Peace and Justice, commemorating the more than 4,000 lives lost to lynching. His great-grandparents were enslaved in Caroline County, Virginia. He was born on the Delmarva Peninsula, just two months after Ralph Northam, and just 90 miles to the north.

Southern Delaware in the late 1950s was nearly as much a part of Dixie as Virginia was, and Stevenson did not have the luxury of not seeing color. “It seemed,” he has written, “that we were all cloaked in an unwelcome garment of racial difference that constrained, confined, and restricted us.” His school was integrated when he was in second grade, but white and black children kept apart. Housing in his town was sharply segregated, and many African Americans lived in “tiny shacks” without indoor plumbing. Stevenson was subjected to regular racial humiliations—told, for example, to enter through the back door at the doctor’s office to receive his vaccinations, and to wait while all the white children received their shots first. When he once went swimming at a motel, he was threatened by white parents who pulled their children from the pool. Color and race have defined his life and his life’s work as a death-penalty lawyer: African Americans are present on death row at three times their proportion in the national population. Not to see color, its legacies and its enduring effects on our society, is not to see. “What’s struck me about the Northam matter,” Stevenson wrote to me recently, “is how much of the white South has been acculturated to not see or think about the victimization of black people, their humiliation through Jim Crow, their terror in the face of lynching and racial violence and the constant degradation as human.”

Virginia voted for Barack Obama in 2008. His victory brought me to tears on Election Night as I remembered a time when it was difficult in Virginia to cast a ballot as a black person and completely impossible to imagine the state voting for a black person. A great deal had changed, but so much had not. Only a little more than a decade earlier, Virginia had made a gesture to confront its complex history when Richmond’s city council voted to add a statue honoring the African American tennis star and native son Arthur Ashe to the parade of Confederate heroes memorialized on Monument Avenue. The city erupted in controversy, heralding what would become a widening conflict over Confederate symbols. It was a conflict that would produce clashes across the South, but pose special challenges in Virginia, where a gradually emerging blue majority confronted persisting numbers of conservatives, traditionalists, and out-and-out reactionaries—as well as countless memorials to the Southern soldiers who had fought more than 100 battles on Virginia soil. The state’s vote for Obama, like that of the nation more generally, only briefly obscured deep and lingering racial divisions.

At the same time Americans were congratulating themselves for having become a “postracial” nation after Obama’s victory, the Secret Service was responding to unprecedented numbers of threats, many explicitly racial, against the new president and his family. Obama’s arrival in the White House in important ways ratified the progress that the nation had made, and Obama himself embraced that story. When Congressman John Lewis, the civil-rights hero, asked Obama to autograph a photo on Inauguration Day, the new president wrote: “Because of you, John.”

But the accession of a black man to power invited backlash and produced outbursts of racial hatred that made clear America was neither postracial nor color-blind: the deaths of unarmed black men in cities from Ferguson to Baltimore, the shooting of nine church members in Charleston by a rabid white supremacist, and the reversals of signal civil-rights achievements through the evisceration of the Voting Rights Act and assaults on affirmative action. In Virginia there was also the horror of Charlottesville, where white supremacists and Nazis invaded Mr. Jefferson’s Academical Village in a stunning juxtaposition of his highest ideals with the consequences of his worst sins. The state’s long history of endeavoring to cover its racial inequities with a surface gentility, of repressing the contradictions between power and justice, made Virginia particularly vulnerable to the revelations of an era of sharp polarization—and an era of social media and all but obligatory transparency. The supposed benevolence of paternalism could no longer shield its fundamental assumptions of hierarchy and supremacy. The obfuscations that had buttressed the myths of Virginia’s racial tranquility began to be exposed as the deceptions and equivocations they had always been. In Virginia, as in the nation, there could be little foundation for complacency about racial progress.

We still have not addressed the roots of our racial divisions. We still perpetuate the denials that enable these injustices. That is one lesson of the Ralph Northam episode. Here is a man who was elected governor with strong minority support, who has urged the removal of Confederate statues, who seems to want to do the right thing on matters of race. But like so many white Virginians before him, he has been caught up in the contradictions of the world of white privilege and its obliviousness to black injury. He professed surprise and bewilderment at the revelation that his yearbook page included a racist photograph. This single photograph simultaneously invokes the histories of racial violence and racial degradation, cruelly dismissing their gravity by casting them in the guise of comedy and youthful foolery.

What was most striking to me about the incident is that it had not seemed of sufficient significance to Northam in 1984 for him even to remember whether he was one of the men pictured—though, on second thought, he said he believed he was not. To associate yourself with the Klan or the performance of black humiliation was evidently neither noteworthy nor memorable. He seems, dismayingly, to be right about that. A survey undertaken by reporters for USA Today of 900 yearbooks from the 1970s and ’80s found dozens of photos of students from across the nation in blackface, in Klan garb, and even acting out mock lynchings. Perhaps this is one reason polls have reported that African Americans oppose Northam’s resignation. They were not all that surprised.

And they understand that even though this is about Northam and his inexcusable actions, it is also about something much larger, something that cannot be solved by a single resignation, or even multiple resignations. This is about a long and troubled history that must be understood and confronted.

We have, thankfully, moved beyond the giddy and unfounded assumption that we are postracial. But I worry that we are still avoiding the most fundamental work. The media frequently report accusations that one or another public figure is a racist, and usually the circumstances or actions described are deeply concerning and worthy of condemnation. It is good that we are noticing. But name-calling and shaming seem to me too often expressions of a certain smugness and self-righteousness on the part of the accuser, acts that too often simply seek to separate us into saved and damned, sheep and goats. And, of course, accusations supply endless gotcha moments to generate clicks and feed social media.

This pattern is also dangerous. It situates the issue of race in individuals and their personal morality or choices, rather than focusing on the broader, structural, historical forces that perpetuate inequality and injustice in the United States—inequality and injustice for which we all, sheep and goats alike, bear responsibility. These are, as King emphasized long ago, “evils that are rooted deeply in the whole structure of our society.” Ralph Northam is the product of four centuries of Virginia’s—and America’s—contradictions and blindnesses, and the complex, discriminatory racial order they have created. Black families lost far more wealth in the Great Recession than white families did. Our system of incarceration imprisons one in three African American men over the course of their lifetimes. Black American women are three times as likely to die from pregnancy-related causes as white women are. The long history of oppression and prejudice has shaped everything from attitudes to housing patterns. Only by understanding this history can we hope to at last transcend it.

In its long pursuit of a more genteel white supremacy—a unique Virginia Way of oppression—the state has nurtured the denial, the failure to see, that Ralph Northam represents. The story of Virginia compels us to recognize how important it is that we open our eyes and actively resist the assumptions and traditions that would obscure our vision. To imagine we are or can be color-blind is to render ourselves history-blind—to ignore realities that have defined us for good and for ill. The Founders embraced both slavery and freedom. We have inherited the legacy, and the cost, of both.

Bryan Stevenson frequently remarks that we are all more than the worst thing we have done. Perhaps Ralph Northam is still in office because enough Virginians share that view. Perhaps somehow we as a nation can acknowledge our worst things and finally overcome them. “The past will remain horrible,” James Baldwin wrote, “for exactly as long as we refuse to assess it honestly.” And it will poison the present as well.

This article appears in the August 2019 print edition with the headline “Carry Me Back.”