Divinity 2 Quarry Area

Hey I'm in the quarry, and through this gold door.
There are two guys named Sidell and Stroud with hammers, they are arguing about Damiens orders and the prisoners, when they notice me. Basically I get my rearend handed to me pretty quick, any help.

Yeah, come back MUCH later!! They were WAY too tough for me when I first went there!!

Indeed, they are very high level. The warning you get near the quarry about "someone being too tough" refers to the people inside the tower, not the ones outside. You'll come back later, don't worry.