Actually, my hackles go up whenever any of those three are uttered--as well as a fourth phrase, "Can you come here for a minute?" and a fifth, "Can I talk to you in private PLEASE?"--especially when there's a certain type of tone behind it.
Especially when women say it, "Can we talk?" too-often means, "I'm trying to be professional, sensitive, and nice, and I'm trying to pretend you have a full right to say no, but you're still about to get sweetly roped against your will into enduring a very embarrassing and/or annoying pseudo-conversation that is, at core level, a lecture, or a patronizing "teachable moment" that will make you feel 5 years old again. Enjoy."
Also, I hate it when someone says "Can we talk?" or "Can I talk to you for a minute?" or "Can I ask you a question?" particularly if it's done in THAT tone, and it turns out to be something insignificant, innocuous, and quick that didn't warrant a huge, nervous introduction. Thanks for getting me ready for a huge ordeal or long chat, just to ask me if the chair looked good there or if I was going to the store later, you twit. I always love a preamble when I'm being asked something that can be answered in two words!
I do realize that it's hard to have those crucial conversations and talk about touchy things, so in some sense there's no perfect way to tell someone they stink or that their socks are ugly.
Mostly, I get mad when people use those phrases, especially with a certain tone, needlessly, so that they cause people to worry, needlessly, about what the chat will be about.
And yes "to be honest" is usually totally benign filler.
"Gentlemen, the verdict is guilty, on all ten counts of first-degree stupidity. The penalty phase will now begin."--Divine, "Pink Flamingos."
"The strong rule the weak, and the cunning rule over all." HS!