Altitudes are referred to a geoid (an imaginary surface of equal gravity strength), that is chosen to fit the mean sea level. Therefore, if we were to constantly adjust the geoid to fit the rising sea level, all the altitudes would indeed be decreasing in average.
However, it is very important to highlight that "in average" part, because of the rates involved. Let's have a look at the sea level rise rates:
As you can see in the above image (taken from Hawaiʻi Climate Adaptation Portal), the rates of rise are in the order of 4 mm/yr, a value that is small in comparison with other processes that can also change elevations. When you go down to the millimeter-scale, the topography is much more dynamic than you would think, for example:
- Post-glacial isostatic rebound: One of the reason the sea level is rising is because the glacier are melting. Mostly from mountain areas. As the glaciers melt, they unload the crust that can be pictured as floating on the mantle (a.k.a. magma at the interior of the Earth), and as any floating object that loose weight, it gain flotation and rises. Typical rates are about 10 mm/yr, but rates up to 39 mm/yr have been measured in heavily glaciated areas. Therefore, most mountains in medium and high latitudes are rising faster than the sea level.
- Erosion: Erosion rates are also of the order milimeters per year up to meters per year in river beds and other areas. So, in some areas the erosion can play a much bigger role than sea level when it comes to elevation change. To make it more complex, erosion will also trigger isostatic rebounding, in many case both processes are in balance at large scales.
- Deposition: Landforms like stuaries, beaches and deltas, are product of the balance between sediment deposition and erosion. If the sea level rise, the delta will rise too given enough time. The problem currently in some areas, is that the sedimentation can't quite catch up with sea level rise and coastal erosion.
- Tectonic uplift: Tectonics also is constantly changing the elevation trough slow deformation and Earthquakes. Parts of the Himalaya are rising at rates of 10 mm/yr, and large earthquakes can produce sudden elevation changes in the order of a few meters.
- Tectonic relaxation: As volcanic islands cool down, they sink, Hawai'i is doing so at rates of about one millimetre per year. And geologically young volcanoes are sinking due to the extra weight its formation put on the underlying crust.
Therefore, to say that the elevation of something specific is changing because of sea level rise is not something easy to backup, as there are many other processes to take into account. Scientist make fairly complex models to figure out which areas will actually experience flooding due to sea level rise, and unfortunately, so far they have found that there are many highly populated areas that will face big challenges for that reason.