Also fought and killed my first invader last night with a buddy. In the ~10 hours we've done coop, that is the first and only invader we've gotten. I imagine it's such a target rich environment for invasions that there's more people to invade than invaders doing it. (And they're all probably still trying to beat them game.) Still. With a 3 man coop squad we played for quite a while together and never saw an invader. It's just surprising for a soul's game. I wonder if people who say they're open to being invaded in SP or what have you get the lion's share of invasions, and people doing coop get what's left over.
As someone that's been doing a hefty bit of invading over the last day nearly everyone I invade has a phantom with them (or even two), and none of my invasion attempts have failed to find someone even if I'm off in some dungeon in the boonies.
I imagine the low number of invaders is at least partially due to how hard it is to find invasion items, and how you can be like 10 hours into the game and only find like 3 you can use which means that even if you like invading you need to save them. The fact that the reusable invasion item is hidden so well means that unless you are specifically seeking it out on the wiki you are going to be limited to just a few invasions.
And as you say, the fact that people are trying to go through the game themselves doesn't help.
Interestingly I've gotten reinforced with other invaders twice while invading, which seems to point to invading someone attracting other invaders.
---
Yesterday I was complaining about most of the spells I find being useless, but since then I've found a few that are indeed quite helpful, just takes a bit of luck finding the right place.