Since visitors were added, I've observed the same problem. Visitors just.. decrease over time. It seems like initially, there can be consistently 1-3 (I've verified this with over a dozen embarks in the same visitor-attractive world) that arrive, then depart after a few months. After that, barring troupes and slayers if you breach the caverns, hitting the cap is... rare (for my forts). In some cases, after the initial 3, exactly zero visitors ever appear again.
Even with 100 civs in the same world, things don't work as you would expect, or hypothesizing would allow. I've even artificially increased the value of a tavern/fort to over 100M , and it makes exactly zero difference to the number of visitors. Having 0, 1, or 10 Tavern Keepers and/or Performers also doesn't seem to have any impact. Fortress citizen count doesn't seem to have any impact. Aboveground/belowground, nope. Trees/shrubs/grass, nope. Entirely carved/smoothed/engraved out of native gold, magnetite, candy, or chert, nope. The only thing that I've found that keeps them
away is a 100% Evil embark area. Other areas, such as Temples, Libraries, etc, nope.
Beyond that, Untamed Wilds is the same as Calm, although I must admit, I haven't tried a 'Good' embark. Lairs don't appear to matter. Corpses, magma, rivers, heck, even sieges don't seem to have any bearing.
In any case, it's not something immediately obvious that determines that 100 visitors would actually ever visit a fort, that I've seen, so I'm not sure why the default cap is set to that value.
I've checked this against the civs in the world, too, to see if there's something pre-embark that determines who visits. It may be related to 'human visitors' or 'elf visitors' or 'dwarf visitors' to/from/at/in other civs, but I've seen situation where numbers of many dozens/100+ of each and a Tavern/Inn of 100M value will get exactly zero visitors, ever. So, while it may have some bearing under some unknown ideal condition, it's certainly not the prime indicator.
Overall, as of 0.47.04, I give the visitor mechanic in general a hearty 1/5 gabbro stars.