If you live or play on the North Shore then you know Pirate’s Cove. It’s one of those places that feels like it’s always been part of Port Jefferson. What’s easier to miss is that the cove itself actually relatively recent, at least in its current form. Before it was Pirate’s Cove, it was something else entirely: It was known as Mount Misery.

Photo: Clipping. Source unknown.
Early photographs from the early 1900s show a very different landscape along the east side of Port Jefferson Harbor.
- Mount Misery was a raised bluff, not a shoreline inlet
- The area that is now the cove sat as elevated land above the harbor
- One of the few known images from around 1910 shows the hill intact, before any large-scale excavation began
If you like this story, click here to read “From Gilded Age Palace to Abandoned Campus: The Untold Story of Idle Hour”
The name “Mount Misery” was already in use by that point, long before the land was altered. By the early 20th century, that landscape began to change for a very practical reason.
- The site was purchased and excavated by the Seaboard Dredging Company as part of a sand and gravel operation
- The material removed from Mount Misery was transported out through the harbor
- Much of that sand and gravel was used in construction projects in New York City

Photo: Back of photograph. Irma and Paul Milstein Division of United States History, Local History and Genealogy, The New York Public Library. View of Mount Misery. East side Port Jefferson Harbor. Port Jefferson, Brookhaven Retrieved from here.
At the time, demand for building materials was high, and Long Island’s North Shore became an important source.
- One operation in the area produced roughly 13 million cubic yards of sand
- The output was valued at approximately $26 million annually
- The work involved hundreds of laborers, along with dredges, boats, and on-site rail systems used to move material
What looks like a nice stretch of shoreline today was, for a period, an active industrial site tied directly to the growth of the city. The physical transformation was gradual but significant.
- Excavation removed large portions of the bluff
- The site took on a bowl-like shape as material was extracted
- Over time, water filled the excavated area, forming what became a sheltered cove

Photo: Newspaper clipping. Source unknown.
Photographs from the period show rail tracks extending into the pit and equipment working below the original grade of the land. The shoreline that exists today is the result of that process, not a naturally formed inlet.
As the land changed, so did the name.
- In the 1930s, it was often referred to as “Seaboard Cove”
- From the 1950s through the 1980s, many locals simply called it “The Cove”
- The name “Pirate’s Cove” became more widely used later on and eventually stuck
You’ll still hear different versions depending on who you ask and how long they’ve been in the area.
The current name comes with a familiar piece of local folklore.
- Stories have long suggested that Captain William Kidd may have used the area as a meeting point
- Some versions of the story include plans to hide treasure nearby
- Others connect the tale to activity around Setauket
There isn’t historical evidence to support any of those claims in Port Jefferson Harbor. Most documented accounts of Kidd’s activities in the region point instead to Gardiners Island. The name, however, has endured.
Today, Pirate’s Cove, part of McAllister Park, feels removed from its early history. It’s a place most people recognize, even if the earlier name, Mount Misery, has largely faded from everyday use.
