No, lava does not push the plates apart.
The mid-ocean ridge basalt is passively filling the space left by the plates moving apart. Plate motion is driven by gravity (image by Vic DiVenere at Columbia; see note below):
There are thought to be two main effects at play: ridge push, and slab pull. We don't know much about their relative importance, but it probably varies spatially (e.g. Ribeiro, 2012, Soft Plate and Impact Tectonics, Springer).
Ridge push happens not because the new lava is pushing, but because the ridge is high up. It's so high that in some places it sticks out of the water (e.g. Iceland). It's high because it's hot — but not from deep convection. A geophysical technique called seismic tomography suggests that the hot zone is relatively shallow. As the new plate cools, it thickens and sinks 'downhill', pushing the older plate ahead.
Slab pull is easier to imagine: a subducting plate, sinking into the asthenosphere (uppermost mantle), pulls the younger plate behind it. Not all margins are destructive, however, so slab pull does not operate everywhere. Indeed, the relative important of push and pull is debated.
You will sometimes see figures like this one, strongly implying that convection and plate motion are strongly coupled. The earth is a complex system so the processes are certainly related, but we don't understand the frictional forces (drag) at the lithosphere–asthenosphere boundary well (e.g. read this primer by DiVenere). These images, while prevalent, are an unhelpful over-simplification.
Note: I'm using DiVenere's copyrighted image under 'fair use' terms. Please replace it with an open version if you find one! (But not one with convection cells :)