The MUD Medievia offers the closest thing to a whaling sim I've yet played, and for me it was about 80% there.
Ships are ridiculously expensive, and you should expect to more or less master making money in the game (or at least befriend someone who has) before you should expect to be able to afford one. More likely, befriend someone with a boat there, and get them to let you sail with them. Most likely though, the option of "ship train", where you are given a ship for free and can explore a small part of the sea, and face serpents but no other monsters, but without the chance to gain any profit.
Hunting serpents is the main thing you do on a ship (piracy, learning how to control a ship, and hunting a score or so of other not-at-all-whalelike creatures being the other main things to do on a ship).
You have computer controlled deckhands, which you shout orders too (players can do anything a deckhand can do, and usually better if the player is smart and fast - but many things take 5-30 sets of hands to do, so deckhands are most useful even so). The sails must be set, in one of three positions that greatly affect how the ship moves and turns. The anchor takes time to raise and drop. The ship is turned at the helm, which is controlled by the one player with their hands on it and turned by fractions of a degree based on your current heading, and can be lashed to hold mostly steady or mostly at a curve of any angle up to 90 degrees. The ship can catch fire, be bombarded by other ships, be rammed by sea serpents or other creatures, run into land (repeatedly until you get the heading changed), be boarded by player pirates or any of several creatures, including the giant heads of the serpents, become the nesting place of sea termites (A very FUN thing to exterminate, especially if the first try is unsuccessful). Each part of it (about 80 separate parts on a large ship, each of which can be "entered" as a separate room) has a specific function, can be individually damaged, and can be individually repaired. There are commands to fight fires, load the ship's guns, fire those guns, change the sails, repair damaged rooms, pump water from the ship, raise and lower the anchor, control the ship's direction, and many other things related to life aboard a ship from fishing to spying on other ships nearby to snagging parts of the carcasses of dead serpents as one sails past the body (worth a great deal to sell).
You have guns to the four sides, and can shout commands to your crew, and specify what percentage of your crew is doing what at any time. Getting charged by two serpents at once, one on either side of your ship, while there are firemoths in your sails and a baby kraken terrorizing your crew at the bow of your ship is quite, quite, quite exciting. Ships can easily die, and they can be fought for and often saved by experienced captains, and there's quite a bit of epic adventure to be had.
The serpents are the only part of this that is whale-hunting like, however. The rest is just an interesting sea adventure, fairly realistic to tend to and keep afloat, and fairly fantastic to adventure with and fight beasties off of. But those serpents.... usually you chum for them, dropping the meat of fish you have caught, or better yet meat ripped from other dead serpents into the sea to summon one up. It rises beside your ship, getting a good look at your while you look at it... there's four colors of them, each with different traits, and these can all crossbreed, from the slow and weak gold serpents to the ever so strong and tough reds, to the incredibly fast violets - a red/violet hybrid is often deadly to fight, requiring near-perfect timing to survive. Then each serpent has one of several head types, which affects if it can breath fire, move especially fast, or perhaps is weaker than normal... and several tail types which give the beast anything from extra movement speed to the ability to spin the heading of your ship at random intervals (Just fired a shot at that fantail? In the second or so it takes the deckhands to fire the volley, if that serpent's tail spins you while it charges to ram you... you may be left with deckhands firing at the empty sea and the serpent striking the previously unthreatened and unprepared side of your ship. Watch for it, be fast, or spend a lot of time on repairs - if you can get that time - until that serpent dies it is still trying to kill you). And count the number of heads, as well, if that serpent is not a hybrid - single colored serpents with more than one may not simply die when they die... they may instead split into one or MANY small but dangerous babies, all furious at your ship and all trying their best to sink you.
Think that firebreathing three headed violet epic-sized fantail serpent is to much for you to take? You're almost certainly right, as epics can have 25 babies when they die, all as fast as the mother, all fireheaded, and all fantails. So dont fire at that one.... chances are if you dont damage the serpent it will eventually sink below the waves to search for other ships to consider. Or maybe it will turn on you, swimming out a fair distance before charging back at you, again and again as you adjust your ship's direction and speed relative to it (the wind affects your speed, and your heading into the wind does too... the slowest serpents are slower than you by far, but the fastest faster than your ship can ever go). A distance of 15 can barely hit the serpent, but in order to break that charge you need to hit it while it is charging and inside a range of 5. And your deckhands need time to fire after you shout (and to reload before they can fire again), and they gain skill and speed over time... so when do you call that shot? And with which guns? You've only one gun on the bow, if you choose (or cannot avoid) to face the serpent head on, but a full 16 to either side, and can choose to fire both decks or just one... more damage if you fire both, and a large chance for each gun to hit with more guns firing at once. But if you miss the headshot which can break the charge, there's no chance to reload before that serpent is on your ship, damaging it, possibly breaking some of the guns on that side so they cannot be fired until they are repaired (and whoever is repairing it is not doing anything else, like firing more shots to kill that serpent, fighting the fires that it set by breathing on you when it rammed, or pumping water from the hold to keep the ship afloat, or even just snagging the valuable serpent meat from the water as you steer your ship past the vast carcass).
When you are done for the day, you return your ship to one of the docks scattered around the world, docking again being an exercise in timing and often teamwork, if only between yourself and your computer controlled crew. Replacing ammo, fully repairing the ship, selling the collected meat and the sea magic stripped from the beasts you killed - a skilled captain can net a huge profit, an unlucky one see the wealth-laden ship sink and be lost before it can be brought safely to dock.