This is the one where things get super wonky and super drawn-out.
What works well in one gun may not work well in another gun. Typically, FMJ (full metal jacket) will be the most reliable in terms of feeding, with the caveat being that in 99% of cases, you shouldn't be loading FMJ ammo into a firearm when it's carried/stowed for personal defense. What you want is almost always JHP (jacketed hollow-point) ammo. Federal, AAC, Hornady, Speer, Winchester, they all make good JHP ammo. I rock Federal HST in 9mm for my carry gun and always have, but there are all sorts of good choices out there.
In a shotgun used for personal defense, what you want is buckshot ammo. Anyone who tells you that birdshot is good for personal defense is full of shit. (Take a look at the name of the ammo and tell me which one you think is better at taking down a human that's attacking you.

)
For training purposes, it's ideal that your practice ammo be the same weight, caliber and brand as your personal defense ammo. I carry 147gr Federal HSTs in 9mm, so 147gr Federal FMJ is what I practice with.
Overpressure ammo is another factor. Some guns can't handle overpressure ammo, but most modern guns do. Overpressure is labelled by the "+P" designator. "+P+" is an industry label for ammo that's loaded to greater operating pressures than a given brand's +P variety. Overpressure ammo has greater recoil than standard-pressure ammo, but has greater kinetic energy, thus making it (debatably) more effective than standard-pressure ammo. For 99% of all situations involving personal defense, there is zero need to use overpressure ammo. I use standard-pressure ammo.
The weight of a bullet is measured in grains (or gr, for short), like 147 grains for certain 9mm loads, for example. Just in 9mm, there are 115gr loads, 124gr loads, 135gr loads, 147gr loads, there are even 158gr loads for use with suppressors. Heavier bullets generate greater recoil.
Some calibers are measured by the metric system, while others are measured by the imperial system. 9mm, 10mm, 7.62×39mm, these are examples of caliber measured by the metric system. .40 S&W, .38 Special, .308, these are examples of caliber measured by the imperial system. Some calibers are measured by both and can be used interchangeably, like .223 and 5.56×45mm.
Some calibers are shorter or longer versions of other calibers. .40 S&W is a shorter version of 10mm with less recoil. You can use a .40 S&W in a gun chambered for 10mm, but not the other way around. This is also true of .357 Magnum and .38 Special, where the latter can be used in the former, but not vice versa. Some calibers are necked down, like .357 Sig where a 10mm was necked down to accept a 9mm bullet.
For shotguns, shell length is another measurement to take into consideration. The length of an open shell is used to create different loads before it is crimped (closed). For example, there are 12 gauge loads of 2-3/4 inch length with eight or nine pellets of 00 buckshot and a standard-pressure charge, and there are loads of 3 inch length with fifteen pellets of 00 buckshot and an overpressure charge (called a magnum load). Some shotguns have a chamber length that can accept longer shells, and others don't.
Shell/cartridge length can affect the operational reliability of a semi-automatic firearm. If you own a Glock 20 and try to use .40 S&W instead of 10mm in it, it may have issues with cycling properly.