I'd say that the proper field of science to counter religion with is not math or astronomy, but neurology. Many people who believe in some divine force, do so because they have had a personal experience that -cannot- be shared with others, cannot be reproduced, but that happens often enough that it can be documented. Neurology may find an explanation for this; it's already managed to document some parts. But I still think that the evidence for God (or for whatever) has been very reasonable for a long time.
That evidence is: I experienced something and describe it as X. You experience something, too, and describe it as X as well. So does that guy over there. After a while, these similar descriptions start to form a trend. And after a longer while--let's call it centuries--the dissimilar descriptions also form a trend. It's like a whole lot of blind men--millions--all touching just their own small part of an elephant, and each giving a name to their part of the shape, until the whole becomes describable. They may each be unreliable, but overall, the fuzzy cloud of data points coalesces into something recognizable. Regardless of whether they get accurate results, and regardless of whether there is error in here, this IS science. Religion IS the science of the unknown. And it's fair, in my mind, to call holy books their theses, their collections of data points and interpretations.
Now, modern neurology may be able to come in and say "Your description of what you experienced fits a divine power. It also fits this neurological event that we can produce whenever we want. Therefore, brain weirdness is a valid and more accurate theory". And modern psychology may be able to come in and say "You claim that a thousand people all felt the same thing, but double-blind trials in other fields demonstrate that this proves nothing and is just mob psychology"... and at that point, the onus is on religious people to congregate in smaller groups, try and document their experiences with scientific rigor, stuff like that, if they want to prove their point.
It is very important to note that, among those millions of blind men who are mapping out a shape to discover that it is an elephant, surely many people among them would walk right past that elephant and never find it. With no experiences, surely most of them would think that the rest are making it up. Surely any atheist who never has a religious experience should think that everyone else is crazy, or lying. And surely some of those blind men that missed the elephant are lying about it anyway. I just ask that they consider that some of us may have found what feels like an elephant, even if they haven't stumbled across anything yet. And not all of us even went looking for it. Hell, going looking doesn't mean you'll find it either.
TL;DR: Well of COURSE atheists think religious people are crazy; they haven't experienced the same things!