You are not mean enough. Like when you say, "I had difficulty building encounters that would prove in any way challenging for them, and when I did, they immediately spent their experience points to eliminate their weaknesses." YOU RULE THE UNIVERSE.
If they defeat 10 thugs, let one escape and bring back 20 friends.
If they beat your big bad villain, tell the players that his big brother has shown up--take the same character sheet, add 20 to all the numbers on it, and have at it.
Assume the villains ALSO learn from their defeats. Make sure that whatever the heroes did last time won't work again.
Make one player's stupid skill the one they need to turn off the 50 PD/50 ED force fields the villains have acquired.
And you talk about how they spend their XP...why are you giving them XP to spend? If they aren't achieving the goals you give them--to role-play, work on background, and so on--they shouldn't be earning experience. For example, you could make a tally mark every time a player gets it right, and give out XP based on the number of tally marks each player racks up. Someone will be greedy enough to become creative in order to get ahead.
Ultimately, not everybody likes the same thing. If your idea of fun and theirs are just that different, maybe you need to find other people who won't fight you every step of the way. But in my experience, players eventually do find out that roleplaying is fun, even if they end up roleplaying "the guy who gets in trouble for shooting first." Try being mean before you give up.