As it was explained to me at university there are two factors; buoyancy and erosion.
Rock buoyancy is a major factor, fresh Basalt is hot and dry and has a much lower density than older oceanic crust. This is why the mid-Atlantic ridge rises above the surrounding seabed.
The effect is even more pronounced with seamounts because they're composed of even hotter rock that takes much longer to cool and wet. As seamounts age they cool, making them denser. As they cool, they absorb water, making them still denser. Where that water interacts with the still cooling rock, secondary mineralisation occurs, which fills gas pockets and other voids, making the rock denser again.
All of this results in the seamount sinking farther into the crust which reduces it's height above sea-level over, relatively short, geological time.
At the same time the mounts are sinking they're also being weathered from above. This shortens them, and also disguises the depression in the seabed from the sinking mountain with debris from above, since erosion generally slows as the mounts sink deeper. Also, as they move north out of tropical waters and coral growth stops, they stop growing at all.
The newer mounts are taller only because they're younger, hotter, and drier, they'll sink and shrink as they age.