So I have walked by this place and driven by it more times than I could count. I decided to explore
the area more and see what I could find. There was a lot of dog walkers and fisherpeople on the grounds.I only went within about 20 feet of the actual jail because of the cars in the lot. The wardens house was really cool and the security man's friend that I met said it was safe to poke around, so I did.
The actual area around the house is very "secret garden" like. It is quite overgrown but you can see there is still some care done on the grounds.
I didn't get very far because I was alone, I would say bringing someone with you is a good idea!