So where did it go?
Carbon was never there (or here) to begin with. You need to look at this from the opposite way. By asking "where did it go" you assuming the Earth was here first, with carbon, and then the carbon was removed somehow. But, Earth formed from a gaseous solar nebula that had all the elements.
As the Earth-forming materials started condensing, they consisted primarily of the refractory elements (i.e., elements that form compounds with high melting and boiling points). The volatile elements (i.e., elements that form compounds that you would mostly recognise as gas or liquid).
These volatile elements never accreted to Earth in any significant amounts. They "boiled" away. They were removed by the solar wind farther away. Carbon was mostly in CO, CH4, CO2, etc. These are volatile compounds that never accreted to the Earth, and by the time Earth was large enough to have a gravity to sustain a proper atmosphere, the carbon was all gone.
It's not only carbon. Neon is one of the most abundant elements in the solar system, but there's almost none on Earth. Nitrogen - we only have it in the atmosphere, which is a negligible component of Earth's total mass. Same with hydrogen. Just a small crust of ocean on the surface.
That's why Earth is made of Si, Al, Mg, Fe and Ca oxides. Oxygen bonded to those elements makes very refractory solid minerals.
Further reading
https://en.wikipedia.org/wiki/Frost_line_(astrophysics)
https://en.wikipedia.org/wiki/Abundance_of_the_chemical_elements#Earth