We are taught that gravity is the force that keeps the atmosphere and everything else binded to Earth. However, observable reality shows us that the atmosphere expands in all directions as it fills any and all available volume. This is an observable natural phenomenon that can be observed from any place in the atmosphere at any time, from ground level up to the upper atmosphere. I find a contradiction between this observation and my understanding of gravity especially because the atmosphere rises and expands in all directions right at the surface of Earth where the force of gravity is claimed to be it's strongest being that it's closest to the center of Earth's mass and just as important, it's still expanding in all directions at the highest altitudes. So if gravity can't hold it down at the surface of Earth how could it ever hold it down at any higher altitude.
A couple thought experiments up front:
- If you breathe on a cold day, the air doesn't rush up, despite the vacuum of space above it. Why? It's not just because of its own gravity... but because of the gravity (weight) of all the gas above it that tends to keep it suppressed overall. Yes it does seem to spread in all directions diffusing, as gases do when there is room. But there is more to come on that...
- But other gases show the situation more; if you put out dry ice ($CO_2$), the gas will sink. This is because it's heavier. Despite being a gas and attempting to fill the free space. And so gravity is pulling it (and other gases). And that points out some of the way the atmosphere works. The lower atmosphere actually works out to be a relatively free place where gases are mixed around more because of more energy. But as you go up higher, the gases start to stratify out. This is where gases with more mass per volume (= density) start to be held lower consistently. So each tends to start to form a level. (We're talking way up there, well above the height of commercial jets). At highest levels, you find the lightest things like helium and hydrogen.
- The $CO_2$ motion is evidence that gravity is impacting gases. But when you breathe out air, it is similar to the mixed gas around it and will be more free within it.
- There is another way to make gas "lighter" rather than changing what it's made out of... temperature. Hot air balloons are an example of something that very much does go up specifically rather than just spread. That's due to density (again, basically how heavy it is in a given volume, so very much a result of gravity). Molecules move faster in warm air, so it's more spread out, so less weight per space, and so is lighter than the air around it and rises. (Also why you may notice your breath indeed tending upwards on a cold day, and likewise from fires/industrial smokestacks, because they are warmer than the environment)
Yet air is less dense as you go up, so it still reaches levels where it stops rising. So what's left is colder and still less dense. So there's a real significant decrease in gas as you go up, and air becomes much less dense (basically meaning gases will tend to stay below it, because it weighs more than the gas above it)
The majority of air winds up being at low levels because it is weighed upon by the air above it. even in the "well-mixed" low atmosphere (go up in an airplane or up a mountain, your ears pop). Higher up, more spread out means less total gas.
(Also worth noting is that the rising air expands to fill the increased space as it rises, but as it expands it uses up energy, cooling... and so stops rising)
So then doesn't the lightest gas, the gas at the top of the atmosphere, just leave?
The answer still tends to be no... because it still has some weight. (There's little remaining pressure force pushing it towards space because those top of atmosphere gases have reached very sparse quantities (= very low pressure)). Everything at every level has an escape velocity... basically the amount of energy needed to overcome the (remaining) gravitational pull and reach space before it slows down too much from gravity. This powerpoint presentation shows that for our most common gases, molecules very rarely reach that speed. For hydrogen and helium, the value is reached a bit more often, but still a very low percentage of the molecules, meaning it takes a long time. But yes, the lightest gases do manage to escape, and in a way it is because they are meandering out to fill the space, more unrestrained by gravity than the heavier molecules that predominantly make up air.
Some other important notes for your question:
The force of gravity is for most intents about the same at all levels of the atmosphere, because the atmosphere is so small compared to the Earth. This Quora question shows basically a 4% difference in gravity between the surface and top of atmosphere, and 9% to the ISS. So there's really not much difference in gravity down low vs up high.
So light gases and energetic molecules do escape. Heavier gases sink, lighter ones rise. So why isn't there a ton of insane motion? Because over time the Earth reaches equilibrium in its distribution of the atmosphere... where the gases trended to move around until a point where the buoyancy (push up by gas molecules from below) and pressure gradient towards space were balanced by the weight at each level... the hydrostatic equilibrium the other answers mention. Less gas settles up high, more gas settles down low, until the forces balance.
When you release a gas, you may well tend to be releasing a gas that was previously there. So in a sense, it's expanding to fill the space that it once previously took up. Of course you're taking out and putting back very tiny fractions of gas. But you can imagine that if you did release a significant amount of "new" gas (from extraterrestrial sources, or from chemical reactions of substances previously tied up in other states of matter), you would add more gas, and it would overall tend to be growing the amount of gas taking up space, and would cause the atmosphere to change size. And that would cause more gas to start to escape to space, by having a taller atmosphere. (Likewise if you heated the Earth significantly)
But coming back to your questions:
I find a contradiction between this observation and my understanding of gravity especially because the atmosphere rises and expands in all directions right at the surface of Earth where the force of gravity is claimed to be it's strongest being that it's closest to the center of Earth's mass and just as important, it's still expanding in all directions at the highest altitudes.
It may appear it is expanding in all directions. But it is actually quite limited over time. Perhaps a bit like how when you stir up sediments in water, it seems to be moving in all directions... but over time it will tend to be affected by gravity and sink back down. And the majority of the gas will over time tend back towards the lower levels of the atmosphere because of gravity too. And if you released air atmosphere at the higher altitudes (at matching temperature) of the thermosphere where it is "settled apart" more, it would tend to sink more notably (like CO2 does in our low-level atmosphere), more vividly showing how gravity is counteracting gases leaving.
However, observable reality shows us that the atmosphere expands in all directions as it fills any and all available volume
A very close measurement will reveal that in the presence of gravity the gas does not fill the volume uniformly. The density will be slightly higher at the bottom than at the top. It is just that in everyday situations the difference is not readily observed.
However, over the entire depth of the atmosphere the density varies from what we observe at the earth's surface and approaches zero as we rise arbitrarily high. But it never actually reaches zero no matter how far you rise (though one will reach an altitude where other effects like the solar wind will overwhelm any measurable atmosphere).
So although the atmosphere is bound to the planet it occupies all of outer space; there is no true "top" to the atmosphere. For our own purposes we do define various boundaries between the atmosphere and out space, but these definitions are arbitrary.
Indeed the density never actually reaches zero, and the effect of this is that the atmosphere of any planet will very slowly leak away.
In the radial direction (upwards), the atmosphere is very close to hydrostatic balance, and allows only very small fluctuations in velocity (Mach number <0.1%).
In the horizontal direction the atmosphere has much larger freedom of movement, being mainly constrained by the Coriolis and centrifugal balance at large scales, which can lead to wind speeds of up to 5-10% Mach number. Still small against a Mach number of 1, at which the atmosphere would depart from any pressure balance.
This subsonic movement of gas is just shifting mass around, there is no net expansion of the atmosphere into space, which could only happen via a radial supersonic, or close-to-sonic expansion (a so-called planetary wind). Hence the common saying (in scientific circles) that all movement in the atmosphere is just a small perturbation on top of a larger balance.
The simple answer is that it does not rise and expand in all directions. In your house or outside, the air is generally not expanding or contracting, and, on balance, neither rising nor sinking. Consider what happens when you blow up a balloon. If the air inside it was constantly expanding, then the balloon would get bigger even after you sealed it until it popped.
Of course, there are updrafts and downdrafts. But they balance out.
Certainly, if you pump all the air out of a chamber, and then open a valve to let air back in, it will expand in the chamber until it fills all the available space. But that is precisely because gravity is pulling down on the air everywhere, and the empty chamber is a place for the air to go to "get out of the way" of the surrounding air that's being pressed down by the weight of the air above it.
You say, "observable reality shows us that the atmosphere expands in all directions as it fills any and all available volume." But most available volume has already been filled, and once that happens, it stops expanding. It doesn't all just expand out into space and disappear.
Air at the same temperature and density released into (still) air at the same pressure will result in diffusion; it is an exchange of molecules where the individual molecules rarely go far in one go before bouncing off other molecules. Diffusion is a consequence of the energetic motions of molecules and their statistical randomness - there is no preferred direction. The mixture will tend to homogenize and tracking the individual molecules will show an expansion until homogeneity is reached - or other factor intevene. Atmosphere is not still and wind and turbulence mix and disperse them over longer distances.
Air released into vacuum will expand in all directions, because of air pressure, which is a consequence of those energetic motions of molecules and collisions with other molecules - which, embodying energy that is conserved, do not subside over time - but each molecule is also affected by gravity, so air released into a vacuum chamber on Earth is not expanding equally in all directions. The pressure gradient - from having the mass of air above plus gravity - will exist, even if very small elevation differences of a chamber would mean extremely precise measurements would be needed to detect it.
To escape to space a gas molecule needs a free path (no other molecules in the way) and sufficient velocity but on average gas molecules travel more slowly than Earth's escape velocity and relatively few manage to escape to space.
Where there is a free path (above most of the atmosphere) it is also colder, with slower average velocity.
To settle on the ground the energetic motions of gas molecules that prevent their compression has to slow a lot, ie it has to shed energy and cool, to liquid and ultimately solid.
The molecules on average aren't fast enough (ie hot enough) to reach space and too fast (not cold enough) to settle to the ground. It is held between, in equilibrium.
The effect of gravity on gas molecules is too small to notice on small laboratory scales
What you observe in a small laboratory scale is not representative of what you observe on a large scale. But there are other observations that hint the small scale experiments in a lab where a gas released "fills the space" dont tell us everything. For example, the atmospheric pressure at the top of everest is less than 1/3 the value at sea level. So, on a large scale, gases clearly don't "fill the space".
But why the disparity between the two experiments? Consider a single gas molecule in air. To vastly simplify, it moves with an average speed of about 500 m/s. If it were an isolated ball moving directly upwards, it would, under gravitational acceleration, stop moving at something like 30km up.
But there are plenty of other factors influencing its motion in the laboratory atmosphere. The mean free path of a typical air molecule (nitrogen or oxygen) in lab conditions is <100 nm. So the possibility of measuring the impact of gravitation on its motion, even under purely diffusion controlled conditions, is pretty, impossibly low. And this ignores turbulence (ie weather) which thoroughly mixes the lower atmosphere making it even harder to see any gravitational effects (which would, for example, lead to denser molecules being more common lower in the atmosphere). In fact the separation by component is unobservable at the top of Everest for this reason. The equilibrium effect balancing the kinetic energy with the effect of gravity on particle motion manifests as lower pressure with higher altitude.
So for reasons of simple physics, gravity does have an effect but such a small one that it only becomes apparent over scales of kilometers. If you had a vacuum in a tube a kilometer or two tall and released gas into it, you would observe a lower pressure at the top compared to the bottom. The gas would not expand evenly to fill the tube. Most labs are not that tall so the effect is not observable on the lab scale. But can be seen easily if you travel up a tall building or a modest mountain while holding a barometer.
Let's look at a cube of air in the atmosphere—say a 1 m × 1 m × 1 m large cube.
While the force on that cube caused by the gravity of the Earth is directed downwards and pulls that cube towards Earth, there is another force that counters that force and pushed the cube away from Earth. This counteracting force is caused by the fact that the pressure in the air just below the cube if higher than the pressure of the air just above the cube, since the pressure steadily decreases the higher up in the atmosphere you get.
The pressure of the air just below the cube, when multiplied by the area of the side of the cube that is directed downwards (which is 1 m × 1 m or 1 m²), causes an upwards force on that cube that effectively pushes the cube away from Earth.
Now, this force is much greater than the force caused by the gravity, but there is also a corresponding downwards force caused by the pressure of the air just above the cube, which effectively pushes the cube downwards, which is almost as great as the upwards force from the pressure of the air just below the cube. And it so happens that the sum of the force cause by the pressure of the air just above the cube, and the force caused by the gravity, equals to the force caused by the pressure in the air just below the cube. So, the net force on the cube is zero, and that's why the air doesn't just fall down to the surface of the Earth.
I have ignored the forces on the cube caused by the pressure of the air on each of the four sides in the horizontal directions, since the pressure there is symmetric and thus those forces cancel each other out.
Or, at least that's the case in the ideal scenario. In reality, you have pressure fluctuations which makes the net force deviate slightly from zero. This in turn cases an acceleration and consequently movement of the air, which we call "wind."