I just did some math and it looks like in many cases you're probably right about the orbital research, at least late game.
I used a vague approximation of my last game's numbers to figure up the research time for a first tier repeatable and came up with the fact that any system with less than +5 research to every field (i.e. basically no system except a forerunner home system with +10 to everything) was hurting my research in some tiny amount. A +5 system was about the breakeven point for one field, and a +4 system increased research time by about 0.1%, with a +2 system increasing it by about 0.15%. I've seen a system go as high as +12 in one field before, but even if I found one like that per field, which might be almost impossible for society research, I'd still be losing out by a tiny fraction in the end.
The numbers are highly dependent on the contribution from planets though. I'd try to model it to find some kind of equation that could tell you where to balance it, but it depends a lot on the output per planet and isn't really simple to do because of that.
In the end, I'll just take the small hits aesthetic's sake, even if they add up for having so many systems. It does tempt me to try to play a truly tall empire for a change though, or play as a pseudo fallen empire. Even in my life-seeded game I ended up with like 45 worlds + habitats and like 80 systems by the time the crisis hit...