Don't forget ramming. Hitting a boat with another boat is a good way to sink at least 50% of the boats involved.
It's safe to assume gunpowder is off the table unless Toady throws medieval stasis out a window so we can forget about cannons. Ballista and catapults shouldn't be relied on to win a fight, because hitting a moving target from a moving platform is hard no matter your weapon of choice, and just keeps getting harder the slower your projectile is. Boarding actions are probably going to be the bread and butter for naval combat.
If the ship relies entirely on sails for propulsion, you have that to worry about too. I still have flashbacks to Sid Meier's Pirates due what a pain in the ass working with(or more often against) the wind could be. Oar are more reliable, and better lend themselves to ramming, but are labor intensive and cut into the amount of manpower you can dedicate to fighting. Also, rowers are rarely so by choice, typically slaves, but generally can't be relied on to do their job reliably without supervision, because at the least you need a drummer to keep them rowing in rhythm. At best they're an extra set of mouths to feed, at worst they're a second enemy boarding party you have to deal with.
Then we have the fantasy wild card to consider. Rock-dropping flying beasts operating from carriers. Amphibious animal people conducting literal frogman operations. A dragon with a saddle and floaties acting as a living fireship. Wizards, just wizards. The sky is the limit.
Of course, you could always pass on all of that and go man mode by putting yourself on a raft and taking on an entire fleet by yourself. Results may vary but considering the average level of experience the playerbase has the outlook is positive. If we can depopulate entire cities on a regular basis, a few floaty boys shouldn't be that much harder.
So really, ship combat could be as complex as we want to make it.