Guns on the first island?
 

Is it possible to loot a proper gun on the first island? I found a vendor who sells it but I am short on cash!

No, there are very few lootable guns in the game. Also, pistols are simply off hand weapons (press E to use one). Bigger guns like shotguns and muskets are actual weapons, but you have to join the Inquisition to get access to the good stuff. The whole fire arms tree is pointless unless you do.

Edit, figured I might add a little spoiler in case you really want a pistol:

Thanks for the answer Maylander. However I think some stuff have changed. I managed to get hold of shotgun from the first island (Tacarigua, the NPC Thompkins sells it) and I haven't even joined the Inquisition yet.

Nothing has changed. I said "access to good stuff". That shotgun is pretty much the best you can get outside the Inquisition, and it's not particularly good. Also, you will not have access to shotgun/musket trainers, so you'll be stuck with crap skills.