I suggest not digging too hard into theory on probability distributions unless its especially interesting - just a few basics are sufficient to experiment with what can be involved in procedural generation of worlds/landscapes.
If its ok to be casual here about the technicalities within, here's an overview:
Most basic and useful distribution is most familar one, the *uniform* distribution which is characterised by being boringly flat and bounded (bounded by default convention between 0 and 1)
Not much needs said about it except its great and the most useful.
The gaussian distribution is the other really fundamental one - it is the curve we get from adding zillions of uniform rands, zillions of times. Its pretty much same as the 'bell curve' and even the poisson curve maps to it, just distorted to degrees by extra considerations.
The shape of gaussian is as fundamental to reality as the shape of the square roots.
Its the only random distribution which will produce a radially smooth blotch when plotted in two dimensions - or a sphericaly smooth blob in 3D - or hypersphericaly smooth cluster in 4D etc.
It is precisely perfectly balanced in that geometric respect and probably others too - a mystical distribution in my reckoning.
Anyway, there's some others such as "cauchy" distribution which iirc is two gaussians multiplied together and important to some theory/calculations, and 'beta' distribution in quantum theory or something, but in game programming the other fundamental distributions can get generated *accidentally* and *unknowingly* while we are tallying and juggling with uniform and gaussian rands.
A proper gaussian random variate (number) can be an odd thing to use in a procedure even when we want the mystical natural curve, because occasionally it can return a HUGE value - the gaussian distribution has no bounds, we cant request a random number between 0 and 10 from a gaussian distribution because the range of all gaussians is infinite.
Quite a smooth approximation of gaussian is doable by adding together, say 5 uniform rands. The max result will then be 5 and min will be 0, most likely will be 2.5 and the likelyhood curve of the values in between will be somewhat close to the perfect gaussian shape.
Likewise the simplest approximation of gaussian which can be perfectly fine in some gaming contexts is the *triangular* distribution resulting from adding just two (uniform) rands together. This is can be known as a *uniform sum* of order 2. Add 3 for *uniform sum* order 3. The uniform sum of order infinity is... gaussian.
I did spend time reading through stuff about other distributions in the course of doing some work with them but the uniform, uniform sums and gaussian are what sticks out as practical and accessible.