If it's a sunny site and well drained, think about lavenders such as Hidcote or Munstead Dwarf - grey foliage, blue flowers, perfume and magnets for bees. They can be kept compact by pruning back once flowering finishes cutting about an inch into the new foliage growth.
For more colour, look at lonicera nitida which has a golden form or euonymus Emerald and Gold which has green and gold variegated foliage. There's a cream and green version too. Consider also dwarf conifers which can come in a range of colours including glaucous blue, bronze and gold. They respond well to regular trimming for a low hedge. I have one with blueish foliage with a fine white bar in the needle. Don't know the variety though.