I thought oceans always filled up to the original sea level from the edge of the map, so they do have infinite water. Of course, my earlier thought experiment method for an ice farm on the freezing ocean wouldn't work/would be just as fiddly as other ways, since any water exposed to the surface would freeze instantly and prevent the filling of any larger reservoir.
For a aquifer ice farm in a freezing biome (with just winter freezes, you can just dig an open pit, and harvest ice from it once per winter), here's an sideview I threw together in Excel:
* The magma reservoir on this level prevents the water above it from freezing. It's held there by a retracting bridge, which should be made of magma-safe materials, same as the mechanism linking it to a lever somewhere else.
** When the bridge is retracted, the magma falls down here. Both the pumps need to be made of magma-safe materials. Because this level won't be safe for a pump operator, you should build a windmill on the surface and link it up (just make the footprint of the open pit a bit bigger on the side with the pumps and dig a route through the aquifer down to the pumps for power transfer, and put a gear somewhere in there so you can toggle the pumps on/off - I left it out here to keep this slightly clearer plus didn't feel like doing yet another edit. I already uploaded 4 versions before I got this whole design right). The extra tile of magma to the right is there to help prevent magma evaporation losses.
*** As mentioned, the 2nd layer of magma is also here to prevent evaporation losses. This can be added later too, but without it you will lose about a 1/7 unit of magma per cycle off your whole (presumably square) footprint, because the last 1/7th of magma won't flow to the pump intake. With a 2nd level of magma, the last 1/7ths will flow around above the surface of the lower layer of magma.
To operate: Retract the bridge to drop the magma, after which all the water tiles will freeze, and the middle 3 (3x3 on the map, probably) can safely be mined out. The ice at the edges should not be mined out, you won't be able to get ice boulders from there anyway since new water will flow in from the aquifer and freeze, possibly encasing your miner as well. Then you can dig out the ice by channeling down from aquifer lvl 1, haul out the ice boulders, pull the lever to return the bridge, then pump the magma back up. The magma will melt the ice at the edges and keep all the water frozen while it refills the bottom level of the pit.
As a possible modification you could add an extra level of empty space so the pumps are safe to operate manually, but then you'd also need an additional pump and I couldn't be bothered to figure out how I'd make the pump layout neat in layout and still keep it 2D. Note that you can just dig out a slice like this and it'll work, but of course I'm assuming you'll spread out the main pit (soil and aquifer levels) and the reservoir into a square shape when viewed from above. One pump and the 1 magma tile are enough on the ** level though.
Other than not showing where the power comes from, the biggest inaccuracy in the picture is that the edge water tiles should be blinking between ≈ and ▲, but I didn't want to spend the time learning to make a gif out of this, and I don't know if animated gifs are allowed/work on the forums anyway.