Vintage News says that the Fairbanks House in Dedham, Massachusetts is the oldest:
The west wing of the house was built around 1654, and the east wing was added in the 18th century. After its construction, a chimney was built at its top, and around 1800 expansion of the parlor was made on this side. At the same time, a new wing with two rooms was added to the west side, and behind this side, a toilet was made as the last addition to the house.
The early inhabitants of the house carved hex signs into the mantle in order to protect themselves from witches. Also, shoes have been found in the attic to chase away evil spirits. In 1895, one of the heirs, Rebecca Fairbanks, was in a difficult financial situation, so she sold the house to John Crowley who after the selling allowed her to live there.
Later, Rebecca sold many of the family items including a unique wooden chest which was made by John Houghton in 1658. This item was purchased back by the family in 2003.
When Crowley wanted to tear down the house in 1897, it was immediately purchased by Mrs. J. A. Codman and her daughter. In 1904, the newly established Fairbanks Family took the house over, and in 1905 it became a museum. In 1960, it was declared a National Historic Landmark and was listed on the National Register of Historic Places.
Two buildings, the La Maison Puisseaux and La maison des JÃ©suites in Quebec City, date from 1637, but it unclear if they were wood-frame houses.