Yeah. To start fishing, first thing you need is the merchant NPC, who's usually the second NPC you get, the first that actually spawns when you make room for him. His spawning condition is pretty simple, have 50 silver coins.
He'll sell you a bug net, you can use that to get various critters that are around. You use that to get bait, butterflies are the most noticeable but generally lower quality. Worms come from breaking the little dirt/stone/grass piles on the surface scattered around with your pick. You'll get a ton of them when it rains. They're pretty respectable bait as it stands, but you can upgrade them (and should) with a fallen star into a fancy enchanted nightcrawler. The thing about better bait is that the better it is, the more bonus to your chance of getting decent stuff and the less chance it has of getting consumed. You don't need too much bait to get started, as you tend to get more through fishing.
Now that you have some bait, you need a fishing rod. The wooden fishing rod is basically unusable. The earliest decent fishing rod is made from 8 iron/lead at an anvil. Now you're set to go fish. The easiest place to start the loot ball rolling are either of the oceans (at the far left/right side of the world). Just make a little platform out with a roof so zombies and such don't harass you while you gather stuff and start casting your rod.
Most of what you'll pull are logically, fish. Bass/red snapper/salmon/trout/shrimp/tuna/whatever with a white name all can be cooked into meals at either a crafting table or a cooking pot, depending on the fish, and not much else. At the ocean, the only other kind of fish you can get is a pink jellyfish which is just more bait as far as you're concerned. If your total fishing power is low enough (below a total of 50% power between rod, bait, and environmental factors), you can sometimes pull up trash, which has a grey name. They just take up space, go ahead and put them in the trash can. Past the fish and trash, everything you pull up can be useful. You can find a handful of accessories that all do nifty things which their description will tell you, some throwable bombs, a spear, a chainsaw, and the best pickaxe you can get before a certain ingame benchmark that changes a ton about the world, among other things.
You can also pull up crates. This is your primary objective most of the time. These things can have a ton of stuff inside of them, based on the rarity. The common crates, wooden, can have some useful accessories, ores and bars, potions, bait and money. The second tier of crate, iron crate, has a better chance at the useful accessories, a nice sword, a nifty pet, better chance at metal bars, potions, and money, and better bait. The rarest crates, gold crates, can have good metals, more potions, really nice bait, and even more money, plus a nifty mount that lets you swim fast. Once you start fishing in other areas, you can get biome specific crates that can have some of the loot that's exclusive to the area. Jungle crates, for example, can have a better fishing rod that otherwise only spawns in special buildings in the jungle sometimes.
One other thing of special note if you intend to fish, somewhere in the world, usually on the surface of the oceans, there is a sleeping NPC. Talk to him to wake him up, and he'll move in once you give him a house. He's the angler, and he's your BFF. He gives you a daily quest for a certain kind of fish, and if you bring it back to him he gives you rewards, which you really want because some of them are incredibly useful.
The
wiki is a bit more thourough on all of the details I didn't cover to keep this (relatively) brief, and once you are starting to get a hang for the game I strongly recommend you educate yourself on the finer points of breaking the game through fish.