Originally, the building's only exterior doors led to an interior courtyard rather than out onto the street. This design conveniently allowed the Gadsbys to have greater control over the movements of their slaves and kept the slaves' presence and activities more hidden from public view. Given the lavish parties the Gadsbys threw, it is easy to conclude their household would be tightly restrained in order make sure things ran seamlessly. We know from Nathan Sargent's 1824 description below that John Gadsby ran his hotels in a militaristic and regimented fashion:

"Gadsby conducted his hotel in a sort of military style,... this was observed at his long dinner table. The guests being all seated, and an army of colored servants standing behind chairs, Mr. Gadsby, a short stout gentleman, standing at the head of the table, the guests silent with expectation, the word was given, "Remove Covers!" ... all the servants moved like automata, each at the same moment placing his hand upon the handle of cover, .... lifting it, stepping back in line and facing the head of the table, and at a sign of Mr. Gadsby, all marching and keeping regular step....."

It was highly likely he governed his own household with a similar approach. The first floor of this building contained the household's kitchen, laundry, and a dining room for the enslaved people. The second floor was divided into three small chambers where the enslaved men, women, and children made their homes. However, we have some sense of their roles in and sometimes outside of the house due to the 1862 freedom petitions. For instance, we know Henry King was an excellent carpenter, waiter and market man. Additionally, Nancy Syphax was a nurse, house servant and laundress. Whether they were allowed worked exclusively for the Gadsby family or were even allowed to keep some of their wages if they were hired out is unknown.

Most of the people enslaved here by the Gadsbys were members of the King and Williams families. Some may have worked at Decatur House or at Gadsby's nearby National Hotel at the corner of Pennsylvania Avenue and 6th Street, while others may have been confined on the property prior to being sold. Margaret Syphax, the daughter of Nancy Syphax, may be have been sold out of Decatur House or just prior to John Gadsby's purchase of his retirement home. We know John Gadsby and his son William Gadsby participated in the slave trade using their hotels. There is some controversy over whether he operated a jail during his time at Decatur House. There is little evidence to suggest this, with exception of Sarah Vedder, a longtime resident of the neighborhood who mentions in her 1877 memoir that John Gadsby used the quarters as a holding cell until enslaved people were sold to Georgia. Further research is needed to determine the veracity of these claims.

However, the Slave Quarters was also the site of a significant act against enslavement—Charlotte Dupuy and her family resided in here in 1829. Charlotte would courageously sue Henry Clay for her freedom; she was unsuccessful, but her case continued the fight against slavery on another battleground—the courts. Later in the 1870s the Slave Quarters also housed free domestics, both African American and immigrants from European countries like Hungary, Austria, Ireland and Italy. Many of these domestics would work for the Beale family; for Edward Beale, an entrepreneur and ambassador and his wife, Mary Beale to Marie Oge Beale, an influential society maven. In fact, the Slave Quarters were used as a living space up until 1961 by the Sclarandris family, Italian immigrants who were caretakers of the home.

Today the Slave Quarters are an educational space and have benefited from major preservation efforts as part of the co-stewardship with the National Trust for Historic Preservation. Although most of the interior architectural features were removed by the National Trust in the mid-1960s, a chimney and hearth with the ghostmarks of a mantle are still visible and the building's original wall timbers form the backdrop. In 2010, the White House Historical Association embarked on major conversation project to preserve the Slave Quarters, completing the project in 2012. On May 13, 2013, American Express provided the Association with a generous grant of the one million dollars which will further research, and preservation efforts as well as fund educational programs.

Floor Plan of the Rear Wing of Decatur House, 1937

Elevation of the Rear Wing of Decatur House, 1937