The mosty likely culprit is the weather, They can be fickle about conditions when the buds are getting fat, If they have a cold spell, they will just throw up their little hearts in despair and refuse to open. Turning brown and falling off.
As your garden is full of colour, have you added more plants to the surrounding area in the last couple of years, that is taking the nutrients from the soil away from the peonys. this too can cause refusal of flowers opening. It would be a survival tactic.
have the light levels changed, ie, taller plants established around it, giving it shade?