The "they mated and made fertile babies, so not different species!" argument is fail.
1) there is some pretty compelling data to show that only a single directional crossing produced fertile offspring, and yet more to show that such hybrids likely had a wealth of health problems.
Namely, there is no ancestral MtDNA from neanderthal females in modern human lineages. So, either neanderthal women were so butt ugly that no human males, ever, wanted them (a bold claim!), or there was some kind of circumstance to cause this from happening or for the lineages produced to survive. Even more interesting, is that neanderthal male sex chromosome is likewise not conseved in modern populations. So now you have the mystery of why all the offspring that survived to procreate were female, and born from early modern human women.
In addition, the presence of neanderthal genes has been linked with an increased predisposition to developing various autoimmune disorders.
Together, this suggests quite strongly that neanderthals and early modern humans where right on the edge of being infertile with each other.
2) There are numerous species that are able to produce viable offspring together. See for instance, the greater and lesser prairie chicken. They are able to be cross bred, and the chicks are not sterile. However, the plumage and birdsong of the hybrids does not attract mates from either pool. They are dead ends.
sorry, but meh. Bad argument is bad argument.
https://www.newscientist.com/article/mg22129542-600-neanderthal-human-sex-bred-light-skins-and-infertility/