It sounds as if your soil has some clay in/under it with it staying wet & then drying out hard. If Rhodos cope then it's on the acidic side.
If that side is completely in shade all day, then colour will be in short supply. However most 'shade' areas do get some sort of light sometimes, unless North facing & surrounded by high walls/fences. A lot of shade tolerant plants are spring flowering, but not all. Think about what grows under trees in a dense wood.
You could try painting the fence a pale colour to lighten up the background behind your plants.
Further away from the fence there will be more light, so by widening the border you could have more plants there that will cope better.
Shrubs- Azaleas, rhododendrons, ribes, pieris, sarcococcca, skimmias, some viburnums- deciduous/evergreen, chanomeles, cornus- they will love the moisture, hydrangeas, including the climbing one which could eventually cover the fence, ivies too- lots of small leaved varigated types, euonymous- the varigated emerald & gold will climb, no flowers though, fuschias- lots of hardy ones, usually the less showy flower types.
Plants- heucheras, bergenias, small spring flowering bulbs- before any overhead tree canopy closes in, vincas- minor less invasive, solomons' seals, hardy geraniums, some campanulas, foxgloves, hellebores, japanese anemones, winter flowering jasmine- yellow flowers, some honeysuckles,campanulas, lysimachia, aquilegias, forget-me-nots, cyclamen, dicentras,
I could carry on walking around my garden, but think you've now got the idea. Shade gardens are challenging, but can be just as good, but in a different way to those in full sun. BTW there are lots of shades of green too! J.