It depends on how big you want the finished topiary to be but if they've been in those pots 4 years already I'd recommend potting them up a size or two so their roots can expand to carry more growth up top. Do this every two years till they get to the size you need and feed them in between times. You'll need patience. I started some plants off as cuttings 2 summers ago and they're still tiny and have done hardly anything this year in the cold and wet although the ones that went to fill gaps in my box hedge have grown much better.
A decent sized box ball or column or pyramid needs a decent sized base to grow from but you can start clipping to shape quite early on to thicken it but always leave at least 2" of the new growth so it gets bigger each year. Mid summer is the best time as box does most of its growing between spring and mid summer.
If you want to grow fancy forms you'll need to let the plants get much bigger before you start to shape them. There's some useful info here - http://www.boxtrees.com/topiary.html