Lol, don't worry, I didn't realize the existence of the Quest Map until much later then where you are right now.
I basically just went to all the areas that I didn't get to explore before and checked them all… Of course, doing that means a lot more headaches, but also a lot more treasures and XP .
But I wish there is a way to mark your map though. I am pretty sure I still have plenty of veins of ore that I missed because I couldn't mine it when I first found it.