Hi,
You would have to remove as much of the material as possible that was used to bed your slabs on and any hardcore as well. The ground will be very compacted and will have to be dug over thoroughly adding compost as you go.
Perhaps you could build a raised bed on the area after, with some good top soil and compost, to give the plants better growing conditions.