C&C is really good in this game. I dealt with the troll problem very quickly in my 2nd playthrough and found out that:
the mage in the lone house don't even get attacked by trolls - which means he doesn't ask my party to fetch his whip but he's not in a risk of dying (i.e. losing potential advisor)
It could also be related to:
How you dealt with the captive troll. I always imagined it was directly related to that, as it's Dimwit or whatever he's called attacking him alongside two other trolls.