As you said, the Andean belt is divided into four segments, usually called the northern, central, southern, and austral volcanic zones (NVZ, CVZ, SVZ and AVZ, respectively; your map lacks the AVZ). This has been interpreted as a difference in the angle of subduction. Under the active zones, the subducting plate (called "slab") sinks into the mantle, releasing water, and the hydrated mantle partially melts, a process called "metasomatism". But under the gaps, the slab has a low angle ("flat slab subduction"), so it doesn't sink deep enough to trigger the melting process. There is no partial melting of the mantle under these zones.
Now, why has the slab failed to sink under these zones? This is due to the presence of submarine ridges. Between the NVZ and CVZ, there is the Nazca Ridge. Similarly, there is the Juan Fernandez ridge between the CVZ and SVZ. Those ridges are young, making the oceanic crust less dense, thus more buoyant (usually it is old, dense oceanic crust that subducts). The ridges also make the oceanic crust thicker, thus harder to subduct.

Image source; by Chiton magnificus in 2011 and placed in the public domain.
Two papers if you want to go further: