What purpose is served by taking sea level as the reference point for finding the height of any point on Earth when we all know that measuring the sea level is a very complex task and it differs from place to place and also keeps on changing? Why can't we just take Earth's centre as the reference point for finding the height of any location?
There are many reasons.
First, measuring from sea level is traditional. It started before we had an accurate way of measuring distance from the center of the Earth, and probably before most people knew it was (approximately) a sphere. But anyone living by the shore could determine sea level to within a few feet.
Another reason is that measuring from sea level tells you useful things like pressure altitude, which are important if you're a pilot or mountain climber. The polar and equatorial radii differ by about 14 miles (21 km), so depending on your location, a distance of say 6,365 km from the center could be deep underground or halfway to the stratosphere.
Why can't we just take Earth's centre as the reference point for finding the height of any location?
While there are many uses for Earth-centered, Earth-fixed coordinates, they're not particularly useful or meaningful in most human-centric endeavors, which happen on or near the surface of the Earth. Consider two points on the surface of the Earth, one at a distance of 6377.12 km from the center of the Earth, the other at 6359.59 km. At which location will you have to worry about altitude sickness? The answer is the one that is 17.53 km closer to the center of the Earth. The first location is Lake Asal in Africa, which is 155 meters below mean sea level. The second location is the Amundsen-Scott South Pole Station, which is 2835 meters above mean sea level.
Because those ECEF coordinates are not particularly useful or meaningful, the first thing a GPS receiver does after computing position in ECEF coordinates is to convert that position to latitude, longitude, and elevation above a reference ellipsoid. There's an issue with this, which is the use of a reference ellipsoid. Some places at sea level have an elevation of 85 meters above the reference ellipsoid, others as much as 107 meters below the reference ellipsoid.
While there certainly are challenges in the concept of mean sea level, it remains extremely useful. It tells us, for example, which parts of New Orleans are subject to coastal flooding, which parts of Fargo are subject to springtime flooding, and which parts of Colorado are subject to altitude sickness.