Atmospheric escape is the loss of planetary atmospheric gases to outer space. You'd never be able to contain ALL of any gas forever by gravity. Ultimately you end up in the rarefied atmosphere where there is some probability that a molecule of gas will reach escape velocity. The probability distribution is given by the Maxwell-Boltzmann distribution and the thermal escape mechanism is known as the Jeans escape.
On earth the problem is compounded by the fact that helium is lighter that the other gases. So (1) helium migrates to the upper atmosphere because of its density and (2) helium atoms on average have the same kinetic energy as oxygen and nitrogen molecules which means that helium atoms are on average traveling much faster than oxygen or nitrogen molecules.
All of this is made more complicated by needing a temperature profile of the atmosphere as a function of the height.
It doesn't help to assume temperature of background radiation because even at that temperature you can calculate the probability of a helium atom having 80% of light speed. This sort of like being able to calculate the probability of throwing $n$ heads in a row regardless of how big $n$ is.