I can do a deep stern a-hole voice when I need to. I usually pause (long dramatic pause) and say "you called me...who are you". I generally assume that the caller is either a crook or someone with horrible manners. I think it's one of those social situations where it's ok to stand up at the level of "total dick". So weird.