Imagine the whole ball being separated into a handful of concentric shells, with the outermost shell being the crust with the surface, and the lower shells ever deeper, hotter, and ghastly regions of the internals of our planet.
Now, just for the thought experiment, forget that each individual shell has "stuff" going on in itself, and just imagine it as a volume of whatever material (dirt, lava, etc.) there is.
The outermost shell then is acted on by gravity and pulled inwards/down. But that's all that gravity does. It does not create pressure on its own, at all! The pressure only occurs because the next layer resists the previous layer - or Earth would implode (which, incidentally, is what actually does happen in a supernova).
The principle is the same as if you heat water on the stove, in one of those pots with a screw-on lid. To create pressure, you need hot water, but you also need the lid resisting the water vapors trying to "get out". Remove either factor, and there is an open pot without any pressure.
Back to Earth: this phenomenon continues. As you noticed, the effect of gravity on each individual shell gets less and less (the shells weigh less, the farther you go down, even if you size them such that they have the same mass as the previous ones). But the downward force from all the upper layers is still there just fine - at this point, gravity has already done its work and it does not matter why the upper shells press down. All the matter there is already compressed as much as possible under the circumstances (of course not compressed as much as physically possible - our planet is much too small/light for that), so the forces from above are transmitted further down.
Obviously, in reality there aren't any well-defined shells and the effect on gravity works on all individual atoms making up Earth, but the principle remains. All that is on top presses down, and thus there is pressure at the very center, albeit there is no or very little gravity down there.