Yes, carbon dioxide truly is a greenhouse gas.
With regard to the immediate greenhouse effect, 99.96% of the dry atmosphere might as well be vacuum. Nitrogen (N2), oxygen (O2), and argon are extremely weak greenhouse gases at the temperatures present in the atmosphere. The only greenhouse gases in our atmosphere are trace gases. (The most powerful greenhouse gas is water vapor, and that, too is a trace gas in our atmosphere.)
The 99.96% of the atmosphere that doesn't directly participate in the greenhouse effect is however present. At sea level, a molecule of carbon dioxide will travel freely for about a tenth of nanosecond before colliding with another gas molecule. This mean time between collisions increases to a few tens of nanoseconds at the top of the troposphere. A greenhouse gas molecule such as CO2 that has just absorbed a thermal infrared photon will very quickly transfer that absorbed energy to the non-greenhouse gases that comprise the vast majority of the atmosphere.