I will answer mostly from archaeological perspective; I don't know much about the paleoclimatology before the first people started to produce stone tools. I know just some basics of other disciplines where they affect human (pre)history, but it might help.
First, even though longer and older cold periods are also refered as "ice ages", most people don't understand "ice age" as this, but as the time when our ancestors hunted mammoths. This is "archaeological" perspective. If you are interrested in the older geological history, Joe Kington's answer explains the "geological" perspective better than I could. From this perspective, no ice age ended 10000 BC - the climate during current "big Ice Age" just went little milder.
I've been told several times during paleolithic and paleoclimatology courses that according to current state of knowledge, there were 52 cold and the same number of warm periods during the quaternary. I didn't really try to find some more sources now, and I'm not sure whether the cold periods can all be classified as "ice ages". I'm not sure, but I think that Bølling-Allerød interstadial is considered to be one of the warm period and Younger Dryas might be an example of a cold period. If we divide the duration of quaternary by 52, we get around 50ky per cold + warm period; so either the 52 + 52 periods were longer and roughly regular, or Bølling-Allerød/ Younger Dryas drag the average length down, while ice ages lasting tens or hundreds thousands years drag the average up.
Anyway, even if there were just four big Ice Ages during quaternary, as scientists expected few decades ago, they wouldn't last millions of years, and the rounding error would be much smaller than 10ky.
Bølling-Allerød and Younger Dryas are especially relevant to your question, since they show that the last Ice Age (with maximum of glaciation around 20ky BP) didn't end suddenly, but that it ended for some 2000 years, then returned for some 1300 years and than ended for good (we usually mark this moment, around 11500 BP, as the beginning of Holocene). The prevailing theory is that the reason of the sudden cold shift is melting of a huge amount of ice, which decreased salinity of ocean water, slowing the Gulf stream down. So there is some "rounding error", but we are out of its scope - I don't think there is enough ice to slow down the Gulf stream enough to trigger something that could be called "ice age" in the "archaeological" sense.