From Mount Misery to Pirate’s Cove: The Real Story Behind This Port Jefferson Spot

If you live or play on the North Shore then you know Pirate’s Cove. It’s one of those places that feels like it’s always been part of Port Jefferson. What’s easier to miss is that the cove itself actually relatively recent, at least in its current form. Before it was Pirate’s Cove, it was something else entirely: It was known as Mount Misery.

Photo: Clipping. Source unknown.

Early photographs from the early 1900s show a very different landscape along the east side of Port Jefferson Harbor.

  • Mount Misery was a raised bluff, not a shoreline inlet
  • The area that is now the cove sat as elevated land above the harbor
  • One of the few known images from around 1910 shows the hill intact, before any large-scale excavation began

If you like this story, click here to read “From Gilded Age Palace to Abandoned Campus: The Untold Story of Idle Hour”


The name “Mount Misery” was already in use by that point, long before the land was altered. By the early 20th century, that landscape began to change for a very practical reason.

  • The site was purchased and excavated by the Seaboard Dredging Company as part of a sand and gravel operation
  • The material removed from Mount Misery was transported out through the harbor
  • Much of that sand and gravel was used in construction projects in New York City

Photo: Back of photograph. Irma and Paul Milstein Division of United States History, Local History and Genealogy, The New York Public Library. View of Mount Misery. East side Port Jefferson Harbor. Port Jefferson, Brookhaven Retrieved from here.

At the time, demand for building materials was high, and Long Island’s North Shore became an important source.

  • One operation in the area produced roughly 13 million cubic yards of sand
  • The output was valued at approximately $26 million annually
  • The work involved hundreds of laborers, along with dredges, boats, and on-site rail systems used to move material

What looks like a nice stretch of shoreline today was, for a period, an active industrial site tied directly to the growth of the city. The physical transformation was gradual but significant.

  • Excavation removed large portions of the bluff
  • The site took on a bowl-like shape as material was extracted
  • Over time, water filled the excavated area, forming what became a sheltered cove

Photo: Newspaper clipping. Source unknown.

Photographs from the period show rail tracks extending into the pit and equipment working below the original grade of the land. The shoreline that exists today is the result of that process, not a naturally formed inlet.

As the land changed, so did the name.

  • In the 1930s, it was often referred to as “Seaboard Cove”
  • From the 1950s through the 1980s, many locals simply called it “The Cove”
  • The name “Pirate’s Cove” became more widely used later on and eventually stuck

You’ll still hear different versions depending on who you ask and how long they’ve been in the area.

The current name comes with a familiar piece of local folklore.

  • Stories have long suggested that Captain William Kidd may have used the area as a meeting point
  • Some versions of the story include plans to hide treasure nearby
  • Others connect the tale to activity around Setauket

There isn’t historical evidence to support any of those claims in Port Jefferson Harbor. Most documented accounts of Kidd’s activities in the region point instead to Gardiners Island. The name, however, has endured.

Today, Pirate’s Cove, part of McAllister Park, feels removed from its early history. It’s a place most people recognize, even if the earlier name, Mount Misery, has largely faded from everyday use.

Places like this tend to feel like they’ve always looked the same, but along this part of Port Jefferson Harbor the landscape was reshaped within the last century by excavation and later by natural settling, and once you know that Pirate’s Cove used to be Mount Misery, it becomes much easier to picture how different the area originally was.

Cover photo: The copyright and related rights status of the item has been reviewed by The New York Public Library, but they were unable to make a conclusive determination as to the copyright status of the item. Irma and Paul Milstein Division of United States History, Local History and Genealogy, The New York Public Library. View of Mount Misery. East side Port Jefferson Harbor. Port Jefferson, Brookhaven Retrieved from here.