Main Content

“Whatever Andrew Johnson may be, he certainly is no friend of our race.”

— -Frederick Douglass

Andrew Johnson’s close association with Abraham Lincoln, as both his vice president and his successor, often disguises Johnson’s own slave ownership. He is a complicated example of a southerner who simultaneously supported the Union and gradual emancipation while perpetuating slavery through the bondage of others—perhaps even fathering children with his enslaved servant. Some of these enslaved individuals were later freed and brought to work at the White House during the Johnson administration. While Andrew Johnson was loyal to the North and pompously referred to himself as the “Moses of the colored men,” his legacy, largely measured by his mishandling of Reconstruction politics after the assassination of Abraham Lincoln, remains marred by racial prejudice.1 Click here to learn more about the household of President Abraham Lincoln.

Unlike many other slave owning presidents, Andrew Johnson was not born into the practice, although as a Tennessee native, he would have been surrounded by others who exploited enslaved labor. Instead, Andrew Johnson was born into poverty and his mother, Mary “Polly” McDonough, was widowed when he was only three years old. He and his brother, William, became tailor apprentices, and Johnson worked as a tailor before running for local and state government positions in the 1830s and 1840s.

Andrew Johnson’s successful election to the Tennessee House of Representatives and Tennessee Senate in 1835 and 1841, respectively, provided increased income and status that led him to purchase an enslaved teenage girl named Dolly and her younger half-brother Sam. These purchases were his first venture into slave ownership and a deliberate demonstration of his increased wealth and prominence; in fact, historian David Warren Bowen suggests that “the servants were procured for what might be best described as cosmetic purposes, as they were clearly not an essential part of the family’s economic support.”2

Andrew Johnson's tailor shop, later home to Sam and his wife, Margaret

Library of Congress

According to the bill of sale, Andrew Johnson purchased nineteen-year-old Dolly for five hundred dollars.3 He also purchased Sam, about thirteen years old, for $541.4 Dolly later gave birth to three children—Liz, Florence, and William—who inherited their mother’s enslaved status and also became the property of the Johnson family; the father of Dolly’s children is unknown. Though the exact number of enslaved individuals owned by the Johnsons is unclear, there were four listed enslaved individuals in the 1850 slave schedules and five in the 1860 slave schedules.5 These individuals were Sam, Dolly, Liz, Florence, and William—though there may have been more.6 Further complicating these numbers was Johnson’s purchase of a thirteen-year-old boy named Henry for $1,015 in May 1857, although he was not listed in the 1860 slave schedule.7

In any case, the enslaved individuals owned by the Johnsons worked in a domestic capacity rather than on a plantation. William’s own recollections demonstrate his duties at the Johnson homestead at the young age of “five or six”:

At four in the morning I had to be up. I went up and made the fire in ‘mawster’s’ room, shined his boots, and then made a fire in the kitchen stove. I stood by his side at the table and saw that all his wants were fulfilled. Then I washed all the dishes. After that I made up his room…by the time all this was done there was plenty else to do, such as working for ‘company,’ plenty of which he had, and helping everybody else around the place.8

Records also show that on occasion, the Johnsons hired Sam out to chop wood for neighbors, sometimes allowing him to keep the wages, though at other times they collected his pay. Evidence suggests Sam resisted the forced labor, and Charles, Andrew Johnson’s son, complained that Sam was too headstrong and should be sold away. Allegedly, Sam once told Eliza Johnson that he would “be damned” to work without pay.9 Martha Stover, daughter of Andrew Johnson, later commented that “Old Sam boasts that he was my father’s servant; but the fact is, my father was Sam’s servant.”10

Bill of sale for Sam, purchased by Andrew Johnson in 1842

Courtesy of the Andrew Johnson National Historic Site, National Park Service

The Johnson family, like many other southern slave owners, claimed that they treated their enslaved property with benevolence—something historically misconstrued as compassion. This tactic, of course, was designed by slave owners to keep the enslaved population subservient. “Fair” treatment sought to improve work ethic while minimizing resistance and risk of escape. For example, there is overt evidence of Johnson’s paternalistic attitude toward the young children of Dolly; in 1854, he wrote to his son Robert and told him that in addition to the gifts he had sent for his young son Andrew, he had also sent “a little chair for Liz and Florence.”11

There may also be another reason for Andrew Johnson’s paternalism. Some historians have speculated that Johnson’s “fatherly” treatment of Dolly’s children, coupled with slave schedule data indicating that her children were “mulatto,” though she was listed as “black,” point to the possibility that Andrew Johnson himself may have been the father of Liz and Florence.12

Additional information may illuminate William’s biological father, as well: Johnson’s son, Robert, was close in age to William’s mother, Dolly. Furthermore, after William’s death in 1943, Robert Johnson was listed on the death certificate as his father.13 William’s second cousin and granddaughter of Sam, Adrian McGhee Boyd, filled out his death certificate; thus, this information about Robert’s paternity was shared family knowledge across almost a century.14

Andrew Johnson Stover, son of President Andrew Johnson, is pictured here with Dolly

Courtesy of the Andrew Johnson National Historic Site, National Park Service

It was not unusual for slave owners to enter into sexual relationships with their enslaved servants, the bulk of which were non-consensual. While there is no DNA evidence of these relationships in the Johnson household, Johnson’s fatherly treatment of Dolly’s children, their recorded racial complexion, and William’s death certificate all illuminate the complex and often predatory nature of male slave ownership.

Johnson was proud of his status as slave owner, often mentioning it in political speeches. In 1858, he bragged that “I have not got many slaves; I have got a few; but I made them by the industry of these hands…What I own cost me more labor and toil than some who own thousands, and got them because they were the sons of moneyed people.”15

Still, Johnson was a proud Unionist—a controversial position in Tennessee as the American Civil War loomed. When the state voted on secession in 1861, Eastern Tennessee, home to the Johnson family, overwhelmingly voted to stay in the Union; however, Western and Middle Tennessee voted to leave, thus leading to the state’s secession.16 Tennessee joined the Confederate States of America during Johnson’s tenure in the United States Senate, but he chose to remain a sitting senator for the Union. During this period, he was separated from his family and their enslaved domestic servants, who remained in hostile Confederate territory.17 In 1862, following the recapture of the state by federal troops, President Abraham Lincoln appointed Johnson to the role of Military Governor of Tennessee as a reward for his loyalty to the Union.

William's death certificate lists Robert Johnson, son of Andrew Johnson, as his father.

Tennessee State Library and Archives

During his time as military governor, Johnson began to support emancipation—not due to his own ideas about racial equity, but for military expediency. His primary concerns were ending the war and crippling the Confederacy. Johnson’s ambivalent position on slavery is most clearly demonstrated by his successful attempt to convince Lincoln to exempt Tennessee from the Emancipation Proclamation in 1863.18 However, as the war continued, Johnson expanded his support for emancipation, perhaps for two reasons: first, to remain in favor with northern politicians (particularly President Lincoln) and second, he probably realized that as the Civil War continued, the preservation of slavery in the South became increasingly unlikely. As a result, Johnson appeared before a gathering at the Tennessee State Capitol in 1863 and stated:

The system of negro Slavery [has] proved baleful to the nation by arraying itself against the institutions and interests of the people, and the time [has] clearly come when means should be devised for its total eradication from Tennessee. Slavery [is] a cancer on our society…19

The Nashville Union also reported that Johnson “was for immediate emancipation, if he could get it.”20 This sudden change of heart came just one month after an important event transpired in the Johnson household. According to Tennessee folklore, Andrew Johnson freed Dolly, Sam, Liz, Florence, and William on August 8, 1863. Today, the state of Tennessee celebrates emancipation each year on August 8, a tradition that began eight years after Johnson freed his enslaved individuals.21 According to the Knoxville Daily Chronicle, Sam was the “1st Officer of the Day” in the emancipation parade on August 8, 1871 and helped to plan the festivities; former President Johnson also joined the parade and spoke at the celebration.22

After he emancipated his enslaved individuals, it appears that Johnson hired a few of them as wage-earning servants during his time as Military Governor and Brigadier General. In 1864 and 1865, he claimed pay toward wages, rations, and clothing for three servants: Henry, Florence, and Elizabeth (Liz).23

The Johnson home in Greeneville, Tennessee

Library of Congress

On October 24, 1864, Johnson finally extended freedom to everyone enslaved in the state of Tennessee, boldly asserting: “I, Andrew Johnson, do hereby proclaim freedom to every man in Tennessee!...I will indeed be your Moses, and lead you through the Red Sea of war and bondage, to a fairer future of liberty and peace.”24

Andrew Johnson’s loyalty to the Union paid off. That year, President Lincoln selected him as his running mate for reelection, hoping that Johnson’s alignment with the Democrats might help to balance the ticket and court southern voters in an uncertain election. They were successful, but six months later Johnson was thrust into the presidency following Lincoln’s assassination and death on April 15, 1865. Ill-equipped to handle post-war negotiations and the rebuilding of the American nation, Johnson’s actions illuminated his prejudiced and often outright racist ideas. Johnson vetoed numerous bills that attempted to promote civil rights and equality for African Americans and generally ignored the implementation of “Black Codes” and other racist policies in the American South that infringed upon the rights of newly-freed individuals.25

Sam Johnson, former enslaved servant of Andrew Johnson, went on to work for the Freedmen's Bureau, an organization that assisted newly-free African Americans after emancipation.

Courtesy of the Andrew Johnson National Historic Site, National Park Service

Johnson’s actions also reinforced beliefs in white supremacy, stating in an 1865 letter to Benjamin French, Commissioner of Public Buildings, that “Everyone would and must admit that the white race was superior to the black.”26 Generally, historians consider Johnson’s maladministration of Reconstruction to have significantly worsened race relations in post-war America, rather than easing them.

To combat these racist policies, Frederick Douglass, a prominent abolitionist and formerly-enslaved man, and other African-American leaders met with the president in 1866 at the White House to discuss racial equality. Douglass, a vocal critic of Johnson, hoped to convince him to extend full voting rights to African-American men across the country but was unsuccessful. In fact, Johnson made insensitive statements regarding slavery as a practice, telling the group: “I might say, however, that practically, so far as my connection with slaves has gone, I have been their slave instead of their being mine. Some have even followed me here, while others are occupying and enjoying my property with my consent.”27

Indeed, Johnson brought some of his formerly-enslaved servants to the Executive Mansion as paid employees and sheltered others in his Greeneville home. William became the president’s valet, and Florence worked in the White House as a maid.28 Johnson also sent Florence to culinary school during his presidency to improve her domestic skills.29 After the presidency, Florence was hired as the Johnson family’s household cook, where she utilized this education.30 In 1869, the Cincinnati Commercial Tribune reported that Dolly, Sam, and his wife Margaret lived at the Johnson homestead, where Johnson did not charge rent.31 Sam and Margaret lived in Johnson’s old tailor shop, while Dolly had moved into the former residence of Eliza Johnson’s mother, “a small one-story, wood colored old house” a few feet away.32

This 1866 political cartoon depicts Johnson's veto of the Freedmen's Bureau Bill. It was originally published in Harper's Weekly Magazine.

House Divided Project at Dickinson College

Unlike many of the other enslaved individuals that labored in the White House, we know the fate of many of the men and women enslaved by Johnson. After emancipation, Dolly, Sam, William, Florence, and Liz used the Johnson family name, which was a common practice. Interestingly, Samuel Johnson became a commissioner for the Bureau of Refugees, Freedmen, and Abandoned Lands (the Freedmen’s Bureau), an organization created by President Lincoln to aid newly-free individuals. The Bureau’s expansion was vetoed by President Johnson. Still, the determined Sam seems to have maintained a pleasant relationship with the president. In 1867, he wrote to his former owner and requested to purchase land for a school for the “colored children of Greeneville,” which he was granted.33 Sam later wrote to Johnson: “I…have not changed any in Politics still being for you as much as ever. I would like to see you all very much.”34

After Andrew Johnson’s presidency ended, he returned to Greeneville, and was reelected to the U.S. Senate in 1875. However, his term ended prematurely when Johnson died of a stroke while visiting his daughter in Tennessee later that year. Interestingly, in the last letter ever written by the former president, he mentions two formerly-enslaved individuals—William and Liz. To his daughter, Mary, he describes his upcoming trip to visit, stating “William is very anxious to come and perhaps I may bring him as he is… desirous to see Liz and the children.”35 Johnson did, in fact, bring William, who remained at Johnson’s side until the moment he died.36

William Johnson, former enslaved servant of Andrew Johnson, is pictured with the Capitol Building following his visit to President Franklin D. Roosevelt. He is holding the cane gifted to him by F.D.R.

Library of Congress

William also attended school after emancipation, where he learned to read and write.37 After the deaths of Andrew and Eliza Johnson, William lived with his sister, Liz Forbey, and her family and worked as a domestic servant and cook. In 1937, after being interviewed by journalist Ernie Pyle, William Johnson gained national recognition as the last surviving individual to be formerly enslaved by an American president.38 As a result, he was invited to the White House to meet President Franklin Delano Roosevelt. President Roosevelt gifted Johnson a silver-headed, engraved cane, and he embarked on a tour of Washington, D.C.39 During their meeting, Johnson described his time serving the president both in and out of bondage, learning to cook with Eliza Johnson, and living as a free man in Greeneville.40

Andrew Johnson’s self-imposed proximity to his formerly-enslaved servants, by bringing them to the White House, sharing his home in Greeneville, and referencing them in personal correspondence, further emphasizes his perception of them as family, perhaps initiated by a blood connection to those that he held in bondage. Though Johnson’s actual relation to his enslaved servants can only be corroborated by DNA evidence, he certainly had a unique relationship with Dolly, Florence, Liz, William, and Sam. Still, Johnson’s actions worsened the lives of many African Americans in Reconstruction America by attempting to stop government programs and legislation designed to help them. His antipathy toward African-American civil and political rights revived the racial hierarchies that had allowed slavery to exist in America in the first place. In fact, William stressed in the last years of his life that no matter how fair the treatment, “[A]ny man would rather be free than be a slave.”41

Thank you to the Andrew Johnson National Historic Site for providing documents used in this article, including the personal scrapbook of William Johnson.

This article was originally published March 5, 2020

Footnotes & Resources

  1. Frederick Douglass, The Life and Times of Frederick Douglass: From 1817-1882 (London: Christian Age Office, 1882), https://oll.libertyfund.org/titles/2007#Douglass_1349_620; “The Moses of the Colored Men Speech,” in David Warren Bowen, Andrew Johnson and the Negro (Knoxville: University of Tennessee Press, 1989), 80.
  2. Bowen, Andrew Johnson and the Negro, 51.
  3. John W. Gragg, “Bill of sale to Andrew Johnson for a slave girl named Dolly,” January 2, 1843, The Gilder Lehrman Institute of American History. Secondary source literature often cites that Dolly’s age at purchase was fourteen years old, but this primary source lists her as a nineteen-year-old. Due to the lack of record keeping regarding the births and deaths of enslaved individuals, it is not unusual that these details are incongruous.
  4. “Bill of Sale for Sam,” November 29, 1842, Andrew Johnson National Historic Site, National Park Service.
  5. During the seventh and eighth census years—1850 and 1860—enslaved individuals were listed separately from the head of household’s census data. Instead, they were recorded on “slave schedules,” which listed the sex, age, and skin color of each enslaved individual without including their names.
  6. United States of America, Bureau of the Census. Eighth Census of the United States, 1860. Washington, D.C.: National Archives and Records Administration, 1860. M653, 1,438 rolls. Historians vacillate on the true number of enslaved individuals owned by Johnson, especially because Johnson himself alluded to having upwards of ten men and women in bondage during his lifetime, without corroborating documentation. This was likely said to emphasize his status to other southern slave owners.
  7. Ibid, 471. The impetus for this expensive purchase of a young man is unclear—in 1857, Johnson was Governor of Tennessee; perhaps an increase in salary and responsibility led him to pursue more enslaved help. Johnson also may have purchased Henry for another owner. In any case, his fate is unknown.
  8. These recollections, reported by newspapers almost fifty years after the emancipation of Johnson’s enslaved servants, may not be absolute fact, but are evidence of the way that William remembered his time as Johnson’s enslaved valet. “Personal Slave and Bodyguard of President Andrew Johnson Recalls Service with Famous Tennessean,” December 30. 1929, from William Johnson Scrapbook, Andrew Johnson National Historic Site, National Park Service.
  9. Charles Johnson, The Papers of Andrew Johnson, Vol. 3, 1858-1860, Leroy P. Graf and Ralph W. Haskins (Knoxville: University of Tennessee Press, 1972), 404-405.
  10. James S. Jones, Life of Andrew Johnson: Seventeenth President of the United States (Greeneville: East Tennessee Publishing Co., 1901), 30.
  11. Andrew Johnson, “To Robert Johnson,” The Papers of Andrew Johnson, Vol. 2, 1852-1857, Ed. Leroy P. Graf and Ralph W. Haskins (Knoxville: University of Tennessee Press, 1970), 230.
  12. Andrew Johnson’s own family history also reinforces this theory. Andrew and Eliza Johnson had five children between 1828 and 1852. The first four children, Martha, Charles, Mary, and Robert, were born in succession between 1828 and 1834, while Andrew Johnson Jr. was born much later, in 1852. Liz and Florence, children of Dolly, were born in the large gap between Johnson’s children, in 1846 and 1848. The timing of Florence and Liz’s births, between Johnson’s other children, further evidences that he may have been their father.
  13. “William A Johnson,” Tennessee Death Records, 1908-1958. Nashville, Tennessee: Tennessee State Library and Archives. According to census data from 1870, Dolly never married.
  14. Ibid; “William McTeer Boyd,” Ancestry.com, U.S., Find a Grave Index, 1600s-Current, Provo, UT, USA: Ancestry.com Operations, Inc., 2012.
  15. “Popular Sovereignty and the Right of Instruction,” The Papers of Andrew Johnson, Vol. 3, 1858-1860, Leroy P. Graf and Ralph W. Haskins (Knoxville: University of Tennessee Press, 1972), 60.
  16. Jean Choate, Eliza Johnson: Unknown First Lady (New York: Nova History Publications, 2004), 69-71.
  17. Ibid.
  18. Bowen, Andrew Johnson and the Negro, 106-110.
  19. “Slavery in Tennessee; An Important Speech by Gov. Johnson,” Nashville Union, September 6, 1863.
  20. Ibid.
  21. Alice Clark, “Eighth of August: Clarksville celebrates 156 years of freedom from slavery,” Clarkesville Leaf Chronicle, August 7, 2019.
  22. “Celebration at Greeneville,” Knoxville Daily Chronicle, August 9, 1871.
  23. “Pay and Allowances for Two Months,” The Papers of Andrew Johnson, Vol. 7, 1864-1865, Ed. Leroy P. Graf (Knoxville: University of Tennessee Press, 1986), 129.
  24. “The Moses of the Colored Men Speech,” in David Warren Bowen, Andrew Johnson and the Negro, 80.
  25. Bowen, Andrew Johnson and the Negro, 115-120. “Black codes” passed by states throughout the South restricted the movement, independence, and freedom of formerly-enslaved individuals and saw to the continued oppression of African Americans following emancipation. Various legislative attempts by Congress to combat southern “Black Codes,” including the expansion of the Freedmen’s Bureau and the Civil Rights Act of 1866, were vetoed by Andrew Johnson but later overridden by Congress.
  26. Ibid, 118.
  27. "Transcript, Meeting between President Andrew Johnson and a Delegation of African-Americans, White House, February 7, 1866," House Divided: The Civil War Research Engine at Dickinson College, http://hd.housedivided.dickinson.edu/node/45144.
  28. “Slave of Andrew Johnson Defies Father Time,” The Journal and Tribune, December 24, 1922. Records confirm that Dolly remained in Greeneville during Johnson’s presidency; some newspapers report that William stayed in Greeneville, too, but William himself claimed to have gone to Washington, D.C. with the president.
  29. Ibid.
  30. Year: 1870; Census Place: District 10, Greene, Tennessee; Roll: M593_1531; Page: 277B; Family History Library Film: 553030
  31. Avery, “Andrew Johnson: His Old Home in East Tennessee,” Cincinnati Commercial Tribune, March 20, 1869.
  32. Ibid.
  33. “From Samuel Johnson,” The Papers of Andrew Johnson, Vol. 12, February-August 1867, Ed. Paul H. Bergeron, (Knoxville: University of Tennessee Press, 1995), 183.
  34. Ibid.
  35. “Andrew Johnson to Mary Stover,” Andrew Johnson Papers, Box 1, Folder 3, Tennessee State Library and Archives, Nashville, Tennessee. According to census data, by 1875, Liz had married George Forbey, a local farmer and formerly enslaved laborer, and the two had nine children together.
  36. Ernie Pyle, Home Country (New York: Amereon Ltd., 1947), 98-99.
  37. Year: 1870; Census Place: District 10, Greene, Tennessee; Roll: M593_1531; Page: 276A; Family History Library Film: 553030.
  38. William Johnson is the last documented individual to have been enslaved by a former president, but research is ongoing.
  39. “Ex-Slave Who Visited the White House in 1937 Dies,” Evening Star, May 17, 1943.
  40. “Roosevelt Chats With Ex-Slave Of President Andrew Johnson: Brought to Washington From Knoxville Home; Describes Last Illness of Successor to Lincoln,” The Washington Post, February 18, 1937.
  41. David Nichols, Ernie’s America: The Best of Ernie Pyle’s 1930s Travel Dispatches (New York: Random House, 1989), 306.

You Might Also Like