Not with buddleia, but I've had it with crocosmia. I cut a load back and chopped up the leaves to go in the compost. When I got back inside I had difficulty in breathing for about an hour. Just as I was thinking I ought to call an ambulance, it stopped as soon as it started. So if you're susceptible to plant allergies you may want to be cautious around crocosmias too.