James Moore, a prominent Natchez merchant, built The House on Ellicott Hill about 1798.  This is the last remaining 18th Century merchant's house on Canal Street, and the oldest building exhibiting high-style architectural details such as fanlights in Natchez.

Among the noteworthy tenants were Samuel Brooks, later mayor of Natchez, and Dr. Frederick Seip of Philadelphia, a founder of the Natchez Hospital.

Interesting features include three bridges across the English basement at the back of the house, and a metal ship's lantern-dome adding height to the drawing room ceiling.

In 1797, Andrew Ellicott, sent by President George Washington to mark the boundary between the United States and Spanish Louisiana, set up his camp on this ridge, raising the American flag for the first time over the new territory.  The United States flag of 1797 flown on Ellicott's Hill today commemorates this act.

 

The Natchez Garden Club acquired the House on Ellicott Hill in 1934, with the intent to restore it to serve as their headquarters.  The structure was in a dilapidated condition, as it had for many years been used as tenant housing for the workers in local cotton mills.  The club retained New Orleans architect Richard Koch to assist them in the restoration.  This was the first restoration project in Natchez.