Huh. I guess I never really thought about that. Technically, you're absolutely right. But if someone said to me, "I'll call you Thursday at midnight," I would expect a call Thursday night/Friday morning, not Wednesday night/Thursday morning. I guess this is one of those things people use incorrectly so often that it's become the norm.
It's funny. When people do that in casual conversation, it doesn't bug me. As long as you can get your point across, it's all good. It's when people put out official communications - advertisements, memos, publications, etc - where it really drives me nuts.