Ironically our GM stopped throwing this problem at us when we became Knights, Barons, a Count, and a Marquess (Marcher Lord).
The Marquess always wears Elven Chain Armor concealed under tailored cloths.
In our NEW campaign it is something to lookout for. Mind you my NEW character is a Minotaur. Armor is optional, but he wears bronze armored muscle Cuirass, Vambraces, Greaves, and a Phrygian-Chalcidian helmet over kilt of highland clan tartan.