A few that come to mind: Lithodora, a lovely deep blue-flowered creeping plant. Coreopsis, glorious yello flowers, although they tend to die after a few years. Saxifrages, such as "London Pride". Dianthus. Hellebores. Aquilegia. Thift (good for thin, stony soil). Pulmonaria (good for damper, shadier places). Most of these will spread or self-seed so you only need a small bit to start you off - I agree with Nutcutlet above, look out for summer fetes etc with plant stall where you can pick these things up cheaply, or beg some bits from other people's gardens.