In the northern hemisphere, July and August are the hottest months statistically1, but the days start shortening after June 21st.
Is this just because the Northern Hemisphere keeps heating up after June 21st (just like the hottest part of a day is not when the sun is highest, but some hours later), or are other effects at play?
1 To my limited experience, Europe specifically, but maybe other northern parts as well. And the questions goes for other months in the southern hemisphere, of course.
The amount of surface heating from the sun is a function of time of year and latitude. You are aware of the dependence on the time of year with the sun over the equator at the equinoxes and ~ $\pm$23$^\circ$ N at the solstices. The varying position of the sun overhead means the projection of a solid angle from the sun onto the surface of the earth will vary as well. This just means that when the sun is low, the energy in a "beam" of sunlight is spread over a very large area and warming will be reduced. This is why a bright sunny day in the winter cannot provide the same heating as the bright sunny day in the summer.
Surface energy loss is a function of the temperature of the surface, and atmospheric energy loss is a function of the local atmospheric temperature. There are some complications here, namely clouds and greenhouse gases that prevent some radiative losses.
At the summer solstice when days are long and the sun is at its highest it is easy to understand that we are getting more energy than we are losing, and the days should be warm. What isn't so apparent is that after the days start shortening and the sun lowering in the sky that the days continue to warm. This is simply because we are still getting more energy than we are losing. It isn't until the energy losses (which happen 24 hours a day, regardless of day/night) finally overcome our energy gains (which only happen when the sun is out) that the days will start to cool.
From www.atmos.washington.edu
This image is from the northern hemisphere, but it is unclear what latitude it is for. The summer solstice is the maximum in solar insolation and the winter solstice the minimum. Radiative losses increase with temperature, and it is the point at which temperature has risen enough such that the loss is equal to the (diminishing) solar input. This is where the maximum temperature will occur. The lag between the solstice and the max temperature will vary with latitude.
As mentioned in a comment in the other answer, large bodies of water exaggerate this seasonal lag in temperature. This is for the same reasons as for land, with the added effect of the large heat capacity of water (about 3x that of land).
You see the same effect in daytime maximum and minimum temperatures where (barring anything but solar influence) the daily max is often in the late afternoon and the daily min is often just before sunrise. This is the same reasoning as above, just on a daily scale rather than seasonal. The sun is overhead at solar noon (this varies with local noon based on longitude, time zone (daylight saving time)) but continues to provide solar heating after solar noon. It isn't until the radiative losses overcome the radiative gains (afternoon to late afternoon) that the temperature will drop. Likewise when the sun goes down temperatures will cool down until the sum comes back up.
From www.vsc.edu
The reason for this is, as you pointed, the same than for day hours being the hottest after noon:
The atmosphere (and the sea) in July is hotter than in June, so even when you are getting less and less solar radiation, the effect is accumulative, and the atmosphere gets hotter and hotter. Only when the sum of the heat input (sunlight) plus the heat loss (radiation) gets negative, at the end of August, atmosphere starts to cold.
