Observe the vertical green lines in this diagram: Those events shows when temperature starts to sink when radiance is at top or even before that. If there was a simple connection between radiance and temperature the temperature should start to sink after radiance maximum.
What climatic mechanisms could start a global cooling when the radiance is maximal or heading to maximal, due to the Milankovitch cycles? Could it be a drastic increase of convection?
Constructive answers will be rewarded with bounties