People not voting doesn't make the politicians move towards the unknown vague mass of "what the non voters want" it just makes them concentrate more on the desires of the people that actually voted.
And so they'll have to lose more elections. They have to cater to people, even to people who do not vote. This is their litteral job.
Who is 'they'. Someone almost always wins any given election (and, even if it's ambiguous, often there's enough ambiguity for the necessarily 'glass half full, at least in public' types to spin that
they won, regardless of the identical but opposing voice(s) coming from their opponents) and then those people
having won generally have the power to help the non-voters, but may well consider it easier to draw in the proven power of the voters for the other guy/gal who are on the nearest fringe to them. (Or, in US politics more so than any other 'democracy', maybe just try to get them into a non-voring status.)
Give or take minor changes in order, the prioritising probably goes "Keep pur core voting > keep our fringe from drifting > try to drag some of their fringe over to us > stop the apathetic from siding with them > try not to actively antagonise the apathetic > I wonder if we can get any of them voting
for us?".Populist candidates/platforms do better at getting virgin/long-term-celibate voters to suddenly make their mark, but the worldwide populism battle is currently being won by "radically overturn the old ways in a raegquit until I get what *I* want¹" styles, where it has. Not "hey, how about we try something competent that wins over people of all stripes by careful and justified compromise at each step".
YMMV. There's quite a few different ways to interpret this all, because there's nothing much quite so absolute in politics.
¹ - "We promise! It's easy! The established politicians that aren't our type of populist will
tell you it won't work out better for you. Which is how you know that it will be easy, right? Because they leep saying it which proves they're lying. As we keep telling you. Ok, here's the deal: You don't like them; they don't like us; We don't like them; so therefore you must like us, just as we
say we like you!"