Well, it depends on the context.
Sure, I usually don't use the word "never" in any formal contexts.
But for casual quick slang talk, I usually don't think too much about it. Certainly don't proof-read much of what I write on forums.
But I did go back to replace the word "never" with "didn't". Happy now? Geez.