You have to pass through some caves to get to the Eastern side, there's a cave opening (near a lake with a Thunder Lizard nearby) that leads you under the mountain to the other side. Be careful of the Ghouls inside, you'll have to run past them if you're not strong enough to kill them yet.