The plants you mention are mainly shrubs rather than herbaceous plants (which die back in winter and regrow in spring/summer). There's a lot to be said for a shrubby border, as it can give year-round interest and colour and needn't take much maintenance. I like the ones you've mentioned already (although I have misgivings about roses, which can be a nuisance to look after). You might consider adding Cistus, Euoynmus, Berberis, Pyracanthus, Fatsia, and Hydrangeas. A black-leaved Elder could look good, and there's always good old flowering currant and Buddleia if you want some to grow fast.
I would avoid (if it wasn't already too late in the case of our garden) the more rampant cotoneasters and also Pernettya, which straggles around and puts out messy runners all over the place.