X Marks the Spot

Uploaded for the Vintage Photos Theme Park theme of "x marks the spot".
At first glance this looks a little disturbing - yikes, what is the "x" indicating?- but since it was glued next to the photo below, I'm pretty sure it was showing the place where that photo was taken.
(See below.)
The album's photos seems to be taken in New York City and New Jersey, with trips to Niagara Falls, the Hudson river and possibly Coney Island.
The photo below shows a river or bay, two castle-like towers, and a possible church next to them - but I don't know where this is.
These photos were taken sometime in the early 1920s.