Nordmarians hate the orcs as they have taken over their mines and smelters. Nordmarians even have orc hunters, and Nordmar is where you can forge a sword with added bounus 'orc slayer'!
Wearing orc armour wont put you into any trouble though.
Having an orc follower and dropping him might cause the locals to attack him (IIRC your follower gets the same reception as you, and revert to his "default" when dropped), but that's about it.