Because the Jews were a stateless nation before the concept of the state existed (i.e. the same reason why other such groups like the Roma were also hated and persecuted), and worse, they weren't Christian (the whole 'judeo-christian' concept is relatively very recent, historically speaking) in a time and place where the Christians were already happily butchering each other because of internal divisions.
their religion allowed them to do banking (unlike other ones).
This is somewhat inaccurate. The Old Testament also forbids usury. The notion of usury as sin in Christian doctrine comes from a little after 1000 CE with Saint Anselm (and, naturally, Aquinas had to put in his two cents). Notably, all that this attempted to forbid was the charging of interest; moneylending was still alive and well among Christians in the sense of venture capital. In short, it was permissible for Christians to invest their money in a business (thus sharing in the risk) but not to simply loan money (in effect, making money simply by virtue of being rich). On a similar note, it was considered permissible by the Church and its scholars for credit unions and bankers to charge for their services.
So, put simply, there was a very narrow segment of the moneylending sector in which Jews had a favorable advantage: pure monetary loans, charging interest on the same. Which they tended to fill not least because they were barred from many other business practices; the Church and local authorities tended to restrict them to positions as moneylenders and collectors of taxes, rent, &c.
In short, antisemitism forced Jews to adopt jobs seen in a negative light which later became the basis for antisemitic stereotypes about Jews working in 'evil' professions. It's a beautiful example of a vicious cycle.