Wearing armor of different factions question

So I have cp1.74 + qp4.2 + cm2 installed. As an example, should I be getting attacked by mercenaries if wearing rebel armor or vice versa? Cause it seems this is whats happening.

AFAIR this is correct.

A little warning, not really spoilerish:
Think carefully before you free a city. There's a counter on it. Free 3 cities and all orcs will consider you an enemy. Delay it. The world is open, do something else first.