The game is literally a Microsoft Access database; it writes to the disk every time you advance an increment.
Financial Centers are fairly useless, especially in the early game. Money isn't typically a concern (though especially for me, since I prefer a conventional start. 20 years of surplus spending lets you amass quite the stockpile of cash), and usually there are better uses for your minerals and workers.
Infrastructure isn't something you'll need on Earth. If you're starting a colony on Mars or something, I usually drop off a few colonists and enough infrastructure to support them, and then let my civilian industry grow up shipping more dudes and infrastructure to the Red Planet.
You'll need only enough maintenance facilities as your largest ship; each facility you build allows the planet to support ships that are 1 hullsize larger (a unit the game uses often. 1 HS = 500 tons). For example if the largest military ship in your fleet is 10,000 tons, you'll want 20 maintenance facilities, but those 20 facilities can maintain an infinite number of 10k ton warships. Note that any ship classed as "commercial" is treated as a tenth of it's size for the purpose of this calculation (and most others involving ship costs; commercial shipyards are 10x cheaper and also 10x faster to expand than military ones, just as an example.)
I usually don't build fuel refineries either; the ones you start with will be more than adequate to supply your fuel needs for the foreseeable future.
To build up your "base" industry, construction factories and mines are the way to go, yes. I usually have half my industry dedicated to building up both of these until Earth's minerals start to run out, and some of my industry dedicated to continuously produce Labs as well.
Typically I'll start off building additional ship yards (it's good to lay down a few early on, since it'll take time to expand them to a decent size), a sector command (extra bonuses from a second governor are nice to have), and any expensive ship components I'll need to jumpstart survey ship construction (i.e. engines, sensors).
EDIT: Some good initial medium-term goals are: build a bunch of survey/exploration vessels and find suitable planets to colonize; develop enough offworld mining capacity (either in the form of automated mines + mass drivers or large ships with asteroid mining modules) to keep the economy running after you use up the Duranium reserves on Earth; assemble a fledgling fleet with several varieties of warship.