The Gold Coast mansion built on a Revolutionary War fort

On a bluff overlooking Cold Spring Harbor and Long Island Sound, a sprawling Gold Coast mansion occupies the site of Fort Franklin, a British military stronghold built during the American Revolution. Known as Fort Hill, the Lloyd Neck estate was constructed in 1879–1880 and substantially remodeled around 1900. Remarkably, when the National Park Service documented the property in 1984, its architectural survey recorded old earthworks enclosing the mansion’s courtyard, preserving a physical connection to the site’s Revolutionary War history.

The mansion’s history connects several periods of Long Island’s past, from British occupation and Patriot raids to the development of the great North Shore estates. Its architectural history may also involve Stanford White, the prominent architect who designed Nikola Tesla’s Wardenclyffe laboratory in Shoreham and whose murder in 1906 became one of America’s most notorious scandals.

A British stronghold on Lloyd Neck

Following the British victory at the Battle of Long Island in August 1776, British and Loyalist forces occupied much of the Island and established military positions along its northern shoreline. The harbors around Huntington and Oyster Bay were strategically valuable, providing access to Long Island Sound and routes for supplying British troops and conducting raids against Patriot communities in Connecticut.

One of the most important British strongholds on Long Island was Fort Franklin, perched on the western side of Lloyd Neck overlooking Cold Spring Harbor and Oyster Bay. Its namesake was none other than William Franklin, Benjamin Franklin’s own son, who had sided with the British Crown against the American Revolution and served as the last royal governor of New Jersey. While his famous father became one of America’s Founding Fathers, William remained a staunch Loyalist, lending his name to a fort built to defend British control of Long Island.

Photo: Portrait of William Franklin (1731–1813), Benjamin Franklin’s son and the last royal governor of New Jersey. The British fort on Lloyd Neck was named in his honor. (New York Public Library, Emmet Collection of Manuscripts)

The Lloyd family, which had owned much of the peninsula since the seventeenth century, found itself divided by the war. Joseph Lloyd, a supporter of American independence, fled to Connecticut during the British occupation of Huntington. He was accompanied by Jupiter Hammon, an enslaved writer who became the first published Black American poet.

The Lloyd family’s connection to this stretch of Long Island survives today at Caumsett State Historic Park Preserve, which occupies part of the family’s former vast holdings. Henry Lloyd, Joseph’s father, established the family residence on Lloyd Neck in 1711, and his original saltbox-style manor house still stands within the park, restored and preserved as a reminder of the peninsula’s colonial past. The property eventually became part of Marshall Field III’s enormous Caumsett estate in the twentieth century, linking the Revolutionary War landscape surrounding Fort Franklin to one of Long Island’s best-known public parks.

According to a historical account published by the Town of Huntington, British forces began constructing Fort Franklin in 1778 on Lloyd family property. A separate fortification was also established near Target Rock on the eastern end of the peninsula.

Huntington residents were forced to build the fort

The construction of Fort Franklin came at a considerable cost to local residents, many of whom were compelled to provide labor, tools, animals and building materials for the British military.

Research published by the Huntington Historical Society in 2026 draws on surviving town records documenting the work. In May 1778, British authorities required Huntington militiamen to construct defensive works on Lloyd Neck. A later accounting recorded hundreds of days of forced labor during May and June of that year.

Construction and improvements continued in 1779. That July, seventeen local men were ordered to report to Lloyd Neck with three days of provisions, horses, wagons, oxen and digging tools. On August 10, Loyalist Lieutenant Colonel Richard Hewlett issued another order requiring Huntington militiamen to appear with axes, shovels and a wagon.

The men received no compensation for their labor or the materials they supplied. Those who refused military work could face fines and, in some cases, the seizure of their property.

By September 1779, Fort Franklin had been completed and was occupied by a garrison of approximately 500 troops. The fortifications consisted of substantial earthen defenses, with additional obstructions intended to make an enemy assault difficult.

The fort’s location made it particularly formidable. Attackers approaching from the water would have to land along the shoreline and climb the bluff, while the relatively level terrain to the east offered the most practical approach to the defenses.

A Revolutionary War map reveals the landscape of Lloyd Neck

A surviving military reconnaissance map from 1781 offers a rare contemporary view of Lloyd Neck during the British occupation of Long Island. Preserved in the Library of Congress’s Rochambeau Map Collection, the hand-drawn document identifies the homes of several members of the Lloyd family, surrounding waterways and features of military interest. Drawn from reconnaissance conducted by American forces, probably shortly before the Yorktown campaign, the map provides a valuable record of the landscape and British defensive positions that American commanders were studying during the Revolution.

Photo: A 1781 American reconnaissance map of Lloyd Neck identifying Lloyd family residences, waterways, a British blockhouse and an inlet where boats could be concealed. Although Fort Franklin itself is not depicted, the map documents the surrounding landscape and British military presence during the Revolutionary War. (Library of Congress, Geography and Map Division, Rochambeau Map Collection)

Among its most interesting details is a structure labeled “fort block house built by ye enemy,” referring to a British defensive position. Another annotation identifies an inlet where whaleboats and barges could be concealed, illustrating the importance of the peninsula’s shoreline to military operations across Long Island Sound. Such information would have been useful to American commanders evaluating British positions and potential approaches to the occupied territory.

Although the map documents British military activity on Lloyd Neck, the Library of Congress specifically notes that Fort Franklin itself is not shown. The drawing should therefore not be interpreted as a plan of the fort or its earthworks. Its historical value lies in recording the surrounding landscape, family properties, waterways and defensive features at a time when British and Loyalist forces controlled the peninsula.

The map eventually became part of the collection associated with Jean-Baptiste-Donatien de Vimeur, Comte de Rochambeau, commander of the French expeditionary army during the American Revolution. It provides a valuable contemporary companion to the later architectural records of Fort Hill, connecting the Gold Coast mansion’s location to the military geography of Revolutionary War Long Island.

Benjamin Tallmadge and the raids on Fort Franklin

Fort Franklin became an important base for Loyalist forces operating along Long Island Sound. Its troops protected vessels and raiding parties that crossed to Connecticut, where both sides carried out attacks, seized supplies and captured prisoners during the conflict.

The fort also attracted the attention of Benjamin Tallmadge, the Setauket-born Continental Army officer who organized the Culper Spy Ring for George Washington. Tallmadge knew Long Island well and sought opportunities to weaken British positions along the North Shore.

Photo: Benjamin Tallmadge (1754–1835), the Setauket-born Continental Army officer who organized George Washington’s Culper Spy Ring and led a September 1779 raid against British forces at Fort Franklin on Lloyd Neck. Portrait by John Trumbull. (Three Village Historical Society, via Wikimedia Commons)

On the evening of September 5, 1779, Tallmadge led approximately 130 men across Long Island Sound from Connecticut to Lloyd Neck. His force included Continental soldiers, boatmen and refugees who had fled British-occupied Long Island.

The raiders captured Loyalists quartered in houses and huts outside the main fortification, seizing prisoners and supplies. The shooting alerted the British garrison, however, and Tallmadge withdrew without capturing Fort Franklin itself.

Tallmadge continued gathering intelligence about Fort Franklin after the unsuccessful 1779 raid. In August 1780, Abraham Woodhull, one of the principal members of the Culper Spy Ring, supplied Tallmadge with a detailed plan of the British fortifications. The intelligence helped American officers evaluate another possible attack, although concerns about the reliability of certain informants ultimately prevented that operation from proceeding.

In 1781, Tallmadge again pursued plans to attack the position, this time seeking assistance from French naval forces. George Washington supported the effort, provided the British fleet did not make the operation too dangerous.

A French-American expedition finally approached Lloyd Neck on July 10, 1781, with approximately 450 troops aboard eight vessels. Benjamin Tallmadge, who had previously sought French naval assistance for an attack, was not present for the operation. The expedition landed on the eastern side of the peninsula and advanced toward Fort Franklin, where Loyalist commander Lieutenant Colonel Joshua Upham had prepared his defenses.

As the attacking force approached, Upham’s troops opened fire with two twelve-pounder cannons loaded with grapeshot. The French-American force withdrew to its ships after suffering three casualties, leaving Fort Franklin in British hands.

Fort Franklin remained in British hands throughout the conflict, despite repeated American attempts to capture it. British forces evacuated their position on Lloyd Neck in October 1782, as the war moved toward its conclusion and the Crown began reducing its military presence on Long Island.

From Revolutionary War fort to Gold Coast mansion

After the Revolution, Lloyd Neck remained associated with the family whose name it still bears. The former military lands eventually passed through different owners as the peninsula gradually developed into an area of substantial country estates.

In 1879–1880, a large residence was constructed on the former Fort Franklin site. According to the Historic American Buildings Survey, the original house was designed by the architectural firm McKim, Mead and Bigelow in the Shingle Style, an architectural approach associated with the large country houses of the late nineteenth century.

Around 1900, the house underwent extensive alterations following its purchase by William J. Matheson, an industrialist identified in the federal survey as a founder of Allied Chemical Company. The original residence was remodeled in brick, and a substantial east wing was added.

Photo: South elevation of Fort Hill mansion on Lloyd Neck, showing the Gold Coast residence constructed on the former site of Fort Franklin. (Historic American Buildings Survey, Library of Congress, HABS NY-6108)

The resulting mansion combined Gothic architectural details with the expansive accommodations expected of a wealthy Gold Coast household. Its exterior included decorative brickwork, stone trim, elaborate chimneys, arched windows, a glass-roofed conservatory and a carved stone entrance.

Photo: Detail of the north elevation and covered carriage entrance of Fort Hill mansion on Lloyd Neck, photographed for the Historic American Buildings Survey. (Library of Congress, HABS NY-6108)

Inside were a large living hall, library, formal dining room and multiple bedrooms. The east wing contained an enormous fireplace, while the grounds included gardens, several cottages, a garage with living quarters and a squash court with a gallery for spectators.

The estate once encompassed approximately 400 acres, according to the federal survey, and included 400 feet of private beach below the bluff. Its approach was lined with sugar maples that replaced an earlier planting of elm trees.

A possible connection to Stanford White and Nikola Tesla

The mansion’s remodeling also carries a possible connection to one of the most prominent architects of America’s Gilded Age. In its 1984 documentation of Fort Hill, the Historic American Buildings Survey identified Stanford White as a possible architect of the extensive alterations completed around 1900. The attribution is qualified, and surviving documentation has not established that White personally designed the work.

White was a partner in McKim, Mead & White, the celebrated architectural firm responsible for numerous important buildings and private residences. His confirmed work on Long Island included the laboratory and transmitting tower constructed for inventor Nikola Tesla at Wardenclyffe in Shoreham beginning in 1901. The Tesla Science Center at Wardenclyffe identifies White as the project’s architect and documents his professional relationship with Tesla.

Photo: Architect Stanford White (1853–1906), photographed around 1900. White designed Nikola Tesla’s Wardenclyffe laboratory and tower in Shoreham and has been tentatively linked to renovations at Fort Hill mansion on Lloyd Neck. (Library of Congress, New York World-Telegram and the Sun Newspaper Photograph Collection)

White’s career ended under extraordinary circumstances on June 25, 1906, when he was shot and killed by Harry Kendall Thaw during a performance in the rooftop theater of Madison Square Garden in Manhattan. Thaw, a wealthy Pittsburgh heir, was married to Evelyn Nesbit, a model and actress who had previously been involved with White. The murder was especially notable because Madison Square Garden itself was one of White’s architectural achievements.

The killing and the subsequent trials became a national sensation, with newspapers extensively covering the private lives of White, Nesbit and Thaw. Although White’s possible involvement at Fort Hill remains unproven, his documented work at Wardenclyffe offers an interesting connection between the Gold Coast estates of western Suffolk County and Tesla’s ambitious scientific project on Long Island’s North Shore.

The Revolutionary War earthworks survived the mansion’s construction

Perhaps the most unusual detail about Fort Hill is that construction of the mansion did not entirely erase the military landscape that preceded it.

In June 1984, architectural historian Eleni Silverman completed a Historic American Buildings Survey record for the National Park Service. Her description specifically noted that the mansion occupied the former Fort Franklin site and that old earthworks still enclosed its courtyard.

The survey does not establish exactly how much of the eighteenth-century fortification remained intact or whether every visible embankment dated to the British occupation. Nevertheless, the documentation establishes that elements of the site’s military landscape had survived more than two centuries of changes in ownership, construction and landscaping.

The federal survey included twenty photographs of the property, documenting its exterior elevations, architectural details and surrounding grounds. The images are preserved in the Library of Congress as Historic American Buildings Survey NY-6108.

The survey also recorded changes made to the property over the years, including a dining room with Georgian paneling added in 1931 and adaptations of the former servants’ quarters. By the time of the 1984 documentation, portions of the estate’s original acreage had already been separated from the mansion property.

The present condition of the Revolutionary War earthworks has not been definitively established. Historian Richard F. Welch, writing about Fort Franklin in the Journal of the American Revolution in 2015, suggested that an embankment wall might still survive, although its identification as an original fortification remained uncertain.

A place where two centuries of Long Island history meet

Fort Hill occupies an unusual position in Long Island history. Its grounds served as a British military installation during the American Revolution before becoming the setting for an elaborate country mansion during the Gold Coast era.

The fort was associated with Loyalist soldiers, British occupation policies, forced local labor and repeated American attempts to dislodge its garrison. The mansion that followed reflected a very different period, when wealthy families transformed sections of the North Shore into private estates designed by some of the country’s leading architectural firms.

The Library of Congress photographs preserve the appearance of Fort Hill as documented by the federal government, while the 1984 architectural survey provides a particularly important record of the older landscape. Whatever may remain of the fort today, the survey confirms that traces of the British stronghold were still part of the mansion’s grounds more than 200 years after its construction began.