I remember how offended a black co-worker of mine became when I identified someone who called her as black, even though I had only heard the person over the phone. To this day I am still somewhat baffled by her anger. It was perfectly obvious to me that the caller was black because of the way she spoke--but I couldn't explain how I knew because I didn't have the linguistic vocabulary to explain--and even if I had, I think my co-worker would still have been offended. She didn't want to accept that yes, there are linguistic differences.
I used to love listening to Margaret Thatcher speak. That woman could have read the phone book to me, as far as I was concerned.
I live in Houston, Texas, and I know I do have an accent; it comes out sometimes, but I try to speak in more of a TV broadcast accent. I have had people tell me I sound as if I'm from Connecticut or even Ireland.