The Emperor's rationale was that all sapient xenos had to be destroyed in order to remove their threat to humanity, real or theoretical. I don't think anybody argues that the Emperor was a good guy, he was the worst, most genocidal tyrant in history (see: The Last Church). Things were much better in the Dark Age of Technology, which from what records survive was pretty Star Trek and Noblebright, at least for humanity. To put it into perspective, one of the Mechanicus Arks has a dorsally mounted chrono cannon (it shoots you through time) from the DAoT. This chrono canon, being operated purely by the Ark's machine spirit, one-shotted a Necron cruiser which was handing the Imperial forces their teeth.
That is why finding a full STC is the human victory condition. That is why the Adeptus Mechanicus uses most of their fleets to fanatically track down five-thousand year old fragments of maps written on napkins that vaguely describe the last place they saw an STC. Nobody could stand against the Imperium if they got their hands on DAoT equipment. One of the tanks that the Imperial Guard uses in M41 is a fucking farming tractor with guns on it, because that managed to survive, and it is very serviceable as a weapon. The actual tank-equivalents would be easy to build Titans that tower into the lower edge of space and are used to siege moons and continents.
Also, it should be noted that the policy against xenos is far more de jure than de facto. Humans are by and large horrifically xenophobic, but the further you get from Terra the more likely you are to see interaction. First it's just a few natives who managed to avoid extermination trying to get along in the Underhives. Then it's the alien being horrible and deserving of extermination, but that would be so much work, and the hardcore Imperial factions never come here. Then you get out to the Eastern Fringe and you have people who were only brought back to the Imperium in the last few centuries, and they'll happily work with the xeno they know against the xeno that's trying to eat the entire galaxy.