I'd say that there's a pretty long way from prostitution to that.
Realistically they are actually prerequisites, rape must exist in order for there to be any demand for prostitution; without rape, prostitution becomes anachronistic. The supply side can be taken care of by creating an impoverished/oppressed/enslaved group of people; however there must also be a demand side in order to have prostitution. When you think about it carefully the prostitute-user and the rapist are psychologically kin; both of them desire to have sex with people that do not desire to have sex with them back.
This same logic could be applied to trade vs crime.
Theft must exist in order for their to be any demand for trade. Without theft, trade becomes anachronistic. ... When you think about it carefully the the trader and the thief are both psychologically kin; both of them desire to acquire goods from people that do not desire to just freely give the goods to them.
Just as you can play semantics and say someone who pays a prostitute is a rapist, because they wanted sex with someone who didn't want sex, you can flip the argument and say that
a prostitute is a thief, because she desired to take money that someone didn't want to give her. That's basically the exact same logic that you used.
An exchange and taking are two different things. In an exchange, you formally acknowledge that the other person is the owner of the thing that you are trading for. Theft is taking power, buying is acknowledging the seller's power. Sure, both end with you obtaining something, so they have that in common, but the power dynamic is completely different.
Sure, you can argue that slavery was a thing, and slavery could include sexual slavery. But
slavery also included manual laborers: so you could make the same argument that literally any economic exchange is anachronistic if you don't model outright slavery. Hell, the logic is even the same as your argument: any exchange of money for services is slavery since the employer wants the labor of someone who doesn't want to provide it. And conversely, all workers are thieves since they want to take the money of an employer who doesn't want to give it. So there's not much logic in singling out prostitution on
those grounds either. Historical slavery included slavery related to anything that could have exchange value, and that did include sex as much as it included literally everything.
The idea that all sex workers are impoverished oppressed and enslaved is also as a result of the predictable effects of
prohibition. Those sorts of things
always accompany prohibition, regardless of the nature of the prohibited thing. In Australia for example, where prostitution is legal and regulated, the average sex worker earns about twice what you'd get for a full time minimum wage job in the USA. If they're earning
more then of course, you can't deny that there could be an element of
choice in such an arrangement. They can earn more in less hours than they would in other unskilled jobs.
One criticism of the legal sex-industry in Australia is that it's run by a number of corporations (they provide the facilities, equipment etc), and these corporations take 50-60% of the money that is paid by the clients. So that's
proof of how terrible the sex industry is compared to other industries, right? Well,
not in any rational sense. There's a thing called
capitalism, where your boss gets a big cut of what you actually made. Again, only getting a percentage of the raw earnings is just a reality of capitalism, and not anything that stems specifically from sex work. Sex workers in Australia earn about double what someone doing 40 hours a week on minimum wage would in the USA. That 50% of earnings that the company gets goes towards rent, bills, paying staff and security guards as well as towards advertising. All that shit costs a packet of money, and it's shit that the girls accept the cut the company makes because they don't have to deal with any of that, rather than working solo, where they'd have to sort all that stuff out themselves. Women work for brothels for the exact same reason anyone works for a company rather than being a sole trader: economies of scale and division of labor.
In fact, there is evidence that the availability of porn, for example, is strongly
negatively correlated with rape, despite what anti-porn people will tell you. Prostitution and rape in general probably have a similar connection. Basically anything that can get your rocks off has a negative correlation to all the similar stuff.