I don't actually doubt you stuck to the hard book definition I just notice that that reading doesn't seem widespread at all and that you'd be more efficiently communicating what you want to say with ONS.
Alltogether I prefer to stand with the dictionnary also, but sheer abundance of reactions tells me I'm not the only one to read it a different way, so all I can do at this point is to tell you what I hear by casual.
Casual is an ambigous word, there is the meaning of "not regularly", then without particular dedication, but also "unforced". By casual I hear doing something, sometimes, just for the heck of it; as opposed to: regularly, forced, dedicatedly, or professionally.
edit: You might even add "purposefully" if you stretch the antonyms a bit.
I would put "casual" as you interpreted it more like this:
"Not conforming to convention or tradition, or not undertaken with considerable planning, premeditation, or dedication."
EG, "Casual gamer" is not somebody who is fixated on 100% completion on hard mode, and unlocking every goddamn achievement-- they just want some care-free entertainment.
or, "Casual Dress"; Not conforming to expected office/corporate/formal dress code.
In that capacity, the evolution of the term makes sense: Blunt referral to it as "Extramarital sex", "One night stand", or worse yet, "Fornication" (Gawd, I can't beleive I wrote that word. blech.) was eschewed (for being offensive), and it was given a euphemism of "Casual sex"-- sex outside of "Normal convention" (ahem, like marriage.)
Personally, I don't give a flying helicopter dick about marriage. I care more about the mental health of the couple, not what makes baby jesus cry, which is why my concerns are more about introducing communicable diseases to your significant other, or with contributing to relationship problems because of mental repatterning from a long period of such sexual exploit that would need to be consciously overcome in order to be addressed. (and since it is not widely known about, does not routinely happen when people who WERE into one-night-stands suddenly find that they really really like a specific someone, but then have a hard time with the pair bonding.)