Yeah, Doom is probably the best known of them but there were smaller ventures like some thing I think was part of the microsoft encyclopedia program.
Well no, the overworld doesn't "just" open up larger circles. Your new tool only actually lets you get at one large new area through a chokepoint of a path. Later tools open it up much more by giving shorter routes back to town (which the player has further incentive to take due to there usually being heart pieces about,) which really just requires me making the flow chart from dungeon to dungeon intersect on a town after 60% of the dungeons. This goes along with the pacing as well what with a first time player not really knowing where or what the next dungeon is going to be.
In LttP you didn't actually have barriers on the light overworld except that one bridge with the hammer pegs and I think one hookshot/glove spot. Technically flippers too but you just walked up there and bought them about as soon as you had cash (I think that's where the glove spot was actually.) It was really tools letting you into nooks and crannies the whole way through, aside from pesky npcs that wouldn't let you through a place and nagging voices in your head.
Link's Awakening had a much more Venn Diagram-y feel to it. Beach town woods and dungeon followed by another dungeon and then suddenly a huge field to run through with structures to ogle at and then one dungeon later you could start going into the mountains. After that it was nooks and crannies again (well I guess you could count animal town and the two dungeons near it as another large area.) Each of those though was gated through one particular path the tool opened up. Either you could walk through now that a stone wasn't impassable or you could jump over a particular pit for a portion of the new area.
So if you're asking how to fill them that's actually why I've bothered to describe this in maze terms. A maze without all these tools and NPCs is just a bunch of winding paths. What the keys and tools do is make dead ends into parts of the main path.
As for the problem of being able to "get through" that I've focussed on changing compact rooms into large areas where very few of the sides are constrained- well that's just fewer places to worry about being unsolvable. It takes more work to make it look nice than anything else.
Seeing as you've mentioned a flow chart there's a fairly simple way to procedurally fill the map with organic shapes. Lay down approximate positions for all the dungeons pretty much randomly, throw down some civilization near the first couple of dungeons (in these games the main castle in the world is more often a dungeon, albeit an unnumbered one, than a population center,) and then expand each region until you fill the map. Just "choose an available edge from the list of available ones and stick a tile there."
Now in both games I've brought up the areas were actually strikingly square without a lot of deviation. On the SNES where they could have two area palettes at once (or just larger ones all including generic grassland) they masked it somewhat but you could see a whole lot of high hills that lined up just so with rivers and somehow trees would line up too. The Oracle games were still working with very square building blocks so they couldn't put nice slanted edges on things but they did seem to go to an effort to have little portions jut into the space that would otherwise be a square area.
Working out the areas at a higher resolution still shouldn't pose any problems with the assets of a computer rather than a hand held. If I'm even less picky about angles the areas could be defined as Voronoi diagrams. Just need to bias the points toward the outer edges and choose the largest ones to be, say, the 1st, 3rd, and 5th numbered dungeons. And of course you combine about 25% of the zones touching each other so they've got the same theme, just so the dungeons don't seem to dominate the landscape too much.
This has made me realize I haven't put any real thought into height though.
No, not mountains height. I mean multiple floors on a dungeon or the overworld caves* that you have such a poor sense of direction in yet they spit you out at the exact distance you traveled. It might be a piece of cake to make the caves periodically act as connections for the overworld but sticking dungeon generation on a 3d grid might give me some headaches.
*Technically also alternate worlds like "the golden realm" in LttP or whatever dark mirror world from metroidvania type titles.
e: Random thought: I'd like a tool to be a camera based on that old (Japanese?) superstition that being photographed steals your soul. It could exorcise possessed objects in the most obvious sense and harm either undead class enemies or perhaps remove some invulnerability shield from some enemy. Puzzle design might take a lot of thought though. Probably be mostly mirrors.
Or you could do gimmicky stuff where the ghosts were some kind of currency or where you had to put one into a receptor to open doors instead of using small keys.