Right, but you can never lose site of them and in most towns there are more than one exit.
I have been working on ways to control this, since we can't seem to turn it off atm and I have found that you can start assassinating them from the outside to the inside.
First a few quick pointers, during Sneaking you can't change Spells or Weapons (sadly) so you need to stop moving (motion), release Sneak, change Spells, then crouch again because you will be switching between Fireball and Sleep, right next to enemies, so you need to be able to do this without any kind of motion, so they don't hear you move.
Take an tougher example like Geldern, once you know you are going to free a town, wait till about midnight, start with the mines first cause you don't want to lose this xp.
As an example the one near the wolf mine where Samuel and his two hands go.
Remember all this needs to be done Sneaking even moving in the town and once this gets started if you get caught, its auto kill on site, so make a Hard Save before all this and Quick Save occasionally, maybe after you learn to clear a full room of say 5 Sleeping Orcs.
It's interesting because you actually need to put 4 of the Orcs to Sleep even though they are currently sleeping because they will wake from normal sleep but not the Spell.
I put the merchant guy and the two hands to Sleep, then Fireball Samuel and his two hands, it's not necessary to put the Slaves to Sleep as they don't report you even in town (unless you kill one of them).
Once you clear all the mines like this of potential xp it's time to move to the Gate Guards, I found starting with the Gate where you teleport in easiest, because remember you have a couple of Guards at the Treasure room.
So same plan Sneak up to one Gate guard from the side (behind if possible) put him to Sleep and while Sneaking Fireball the other, then the sleeping one.
The safest bet is to work your way around to the other gate from the inside then Sleeping the two Treasure Guards and on Gate Guard, then killing each in the order you put them to sleep as the first cast will wake first and so forth.
(I would avoid even walking close to or going near the Leaders Temple, at this time)
What you want to do is kill all patrols, even the ones on the walls so moving through the streets "taking out da trash"
Then you can take care of the indoor areas like the mine and bunk rooms, but still avoiding the actual Temple Headquarters.
This is a bit slow going so if your not comfortable moving the clock back to midnight due to a strict personal RPing style, then you can leave go do something else and come back the next night, since you have taken out the Gate Guards you won’t get auto attacked by teleporting back the next midnight.
The Temple is the Hardest, but you must come at it from the second level from the bunks which you have cleaned out, since the problem is about 6 constantly roaming guards.
As you ease out occasionally one will be to the left in the dark but you should be able to see any with your Sleep spell active.
The best way I found was looking at it as three sections, the front (nearest you) and the right and left side.
There are stationary Guard (s) on the right so attacking the front is very risky, so the best bet is to Sleep the front Guard (s) then trying to Sleep one on the left and then killing the other and then quickly back into the shadows near the starting point in the front before the others wake up.
Then starting over again Sleeping the front Guards and trying to take out the ones on the left till you have no roaming guards, finally Sleeping the front and killing that pesky Guard on the right and finishing off the front guard.
Now you take out the three Shaman, but stopping short of entering the Temple here and going all the way back to the front of the Temple you have been avoiding.
Depending on how many people you killed in the houses we are getting very close to Auto Retreat here maybe a Semi Hard Save, also iirc the alchemist and merchants turn on you as well so you might want to go head and take them out, oddly enough the guy whom wants to me Lares turns on you as well.
Ok here I Slept the Entrance Guard(s) at the entrance then killed Nerick, now basically you should have two entrance Guards one outside and one just inside sort of hidden to the left.
So I Slept the outside Guard then Fireball the inside Guard if you still haven't gotten a pop by now your pretty lucky as you should only have a Guard and the Head Shaman whom speaks like an Italian Godfather.
Basically you want to work in to be able to keep them bottled up and this way when they run they all run the same direction and you can follow them to their looping AI spot, weeee!