Getting out of harbour town before choosing side ?

The tunnel out of town can be found in the brothel. It's in the left room with the bathing girl behind the picture on the right wall if you enter the room (you may have to finish certain quests before the passage becomes available… I don't think that the picture can be moved before a certain point but I'm not sure exactly what quest triggers the unlocking of the passage. Just keep doing quests in the harbor city until it works ). Just stand in front of the picture and hold down the left mouse button just like you might have already done in the Don's residence or like when you dig for treasures in the wild. This will open the secret passage. Now there's two locked doors in the tunnels. The key can be found on Scordo (the guy who hides in the tunnels). Just use pickpocket on him, take the key, and enjoy your newly found freedom .

scale the tower at the back of the commandant's place and take a swan dive … you will think you are dying but then will wake up on the beach. Getting back in the town is easy - another 100 gold at the same place you got in the first time.

You can also climb down. That's what I did the first time. Standing in front of the commandant's place (do not enter, do not pass the guard) go left, to the corner, be careful, you can climb down. Beware of the guards when you're down.

If you have the acrobatics skill you can jump off the walls behind the forge and land near the secret exit taking no damage.

1. The extortion quest.
2. The Sid/Rodriguez feud.
3. Recovering the amulet from the Don's house.
4. Recovering the armor plates.
5. Captain Romanov's treasure chest.
6. Recovering the goblets.
7. The weed quest.

I jumped from the wall next to the commandan't office onto a high rock and from there, further down again - but you have to sneak and run your way out once you reach the ground, because the guards are very likely to spot you even when sneaking.