For (a vocal brand of) Republicans, the gap between too useless and too over-efficient, in 'gubmint'-related bodies, tends to be small. Possibly even negative, so they overlap.
Here in the UK, various places that
every bank has closed its high-street presence in have tended to use the Post Office as an account-agnostic money-handling counter.
(Never mind that the POs themselves have also had a recent history of moving out of their prominent GPO-era building to save costs (often leaving it empty and unused, not taken on by anyone else) and instead set up shop in the rear of some other shop ('minimart' types, I think, by US standards) which sounds to me like it'd be a rental agreement instead of what
historically was a publicly-owned building. But who knows how they (mis)managed this in most recent stretch of the intervening century or so.)
Not sure exactly how they handle no-bank-account people, a few decades ago you could "cash a
giro", and there must be a modern equivalent (apart from 'cash4cheques' establishments that also do payday loans/etc).
The big thing complained about over here is needing a photo-id to open most bank accounts (logical to be required, but if you're fallen between the cracks you might not have driver's licence or passport) or else an
obscene amount of other personal proofs-of-identity/residence, which having no postal address could indeed cause even problems.