I think the concept of ship mode is quite sound, but it needs more fleshing out. Here's how I think it could work:
When you start, you have a certain amount of money, with this money, you have to buy a ship and hire a crew. You would only be able to choose between a few set designs that have been procedurally predesigned in worldgen, the ships you can choose depend on what civilisation you start as.
After choosing you ship, you can choose your crew, this could either be from a list, where they all have pre-determined skills, or it could be just like the current embark screen.
Afterwards, you would choose armaments for your ship, such as catapults and ballistae, as well as small arms (swords, bows etc).
After you start, you could then buy cargo from the town, receive missions and join fleets. This way, you could be a merchant, part of a navy, a pirate. Perhaps you could also start up colonies.
Over time, new ship designs would be made, procedurally, of course. You would also be able to commission a naval architect to design new ships to your specifications, for a price. Maybe you could also make your own designs. You would also be able to upgrade all the parts on the ship, everything on board would have a quality level.
There is one fundamental flaw in this though, and that is that if we are going to be able to zoom down to fortress mode scale, how would ships be able to turn? As we are restricted to our rather blocky grid.
Another, not so fundamental flaw, is that naval combat wasn't very common back then, and when it did happen, there was little ranged combat, it was mostly 2 ships sailing next to each other and boarding each other, which is slightly boring.