Boys will be boys and boys.

"That ol boy aint got a lick o sense."
"Boy, git yer ass over here an git busy."
"Who's that boy lives up the head o the holler there."
"You boys best be gittin along."
"That boy kin throw bales better 'n anybody round here."
"You don't wanna be lettin that boy borrow yer tools."
"Boy oh boy, did you see the titties on that broad?"
"That boy's big as a mule."

These are all random exclamations one might hear on a daily basis anywhere around this part of the country. "Boy" is not a derogatory term. I might refer to my elderly neighbor as an "ol boy" or the twenty something young father down the road as "that young boy." If I want to address a collective group of us old hillside farmers, I might start the conversation by saying "Boys, let's go. . . "

So when does "boy" becoome a racial slur. I've used the term "boy" when talking to black friends. Around here, the black guys I know understand when it's just a natural part of conversation and when it's being used in a derogatory sense. I've never been accused of using it in a derogatory sense and never used it as such.

[There was an incident when I walked up to a black buddy of mine at work. He had just walked out of his lab and he had a digital monitor in his hand and was looking intensely at it and pushing a button or something and I said "Is that your GameBoy?" For about 10 seconds he looked at me with serious vengence in his eyes and I looked at him like "what the fuck is your problem" and then we both cracked up laughing.]

But I think it is clear when someone uses the term in a nonconfrontational colloquial manner and when it's being used as a way of being demeaning.

This past weekend a congressman by the name of Geoff Davis (spell it however you want, you're still named after the president of the confederacy) referred to Barack Obama as "that boy." Why would any politician on the national level even think of using the term "boy" when referring to any black man? Come on! Boy, you just gotta be plain old dumb.
If he'd been standing around the local fillin station talking to some of his buddies and made that comment. . . well, I dunno, maybe it could have just been a colloquial comment. And surely, in a packed room full of media people, he wasn't intending to belittle Mr. Obama by calling him a "boy." Was he? Maybe, when your name is Geoff Davis ya just can't help yourself.
How was the rest of the speech phrased? Was he speaking in colloquial English? No matter, this ol boy was an asshole. If it was just a colloquial comment, why didn't he correct himself right then and there? Nope, this guy really did call Mr. Obama a "boy" and I believe he meant it in a derogatory diminutive way. . . of the worse kind.
Rep. Geoff Davis (R) from Kentucky, you are Amurkkka's dumbass prick racist motherfucker of the week. Congratulations! Now sit down and shut the fuck up.