I read that the altitude of mountains is measured from the sea level. So, if the sea level rises significantly due to global warming, will the mountains and hills lose some of their altitudes and become less tall?
It depends on which vertical datum your mountain elevation references. If the elevation of the mountain is expressed in meters above mean sea level (or some other tidal datum) then yes. If the elevation is expressed in meters above the geoid (a gravitational height) then no. See this link for a tutorial on vertical datums.
If the Sea level rises, will the mountains lose their height?
It depends on the mountain. Some mountains are rising due to plate tectonics or isostatic rebound. This can easily outpace sea level rise due to global warming. Some mountains are decreasing in height, not because of sea level rise but because of erosion. For example, the Appalachian mountains in the US and Canada are estimated to have once been as high as the Himalaya.
But in general, the answer is yes.