I grew up in the countryside around all sorts of poisonous plants - so did my brother who was (and is) close in age to me. We didn't even have a garden fence to keep us out of the woods and fields; we played in the churchyards, fields, ditches, woods and commons from a very young age.
Human brain development goes through various stages - one of the stages when children are young is one where recognition of dangerous foodstuffs is imprinted (as it would have been with small children when we were evolving living in caves with no fences around gardens). That's the reason why so many children with severe autism are excessively finnicky eaters (to put it mildly) - because that bit of brain development has become rigid.
If very young children are taught by their parents what is safe to eat and what is not, they will remember. I knew from a very young age that 'shiny red berries' were nasty and would make me ill - I was never tempted by them and neither were any of my friends. If you leave it too late to teach this information it's much harder for the brain to absorb the information and retain it's importance.
Of course, if someone has a moderate to severe learning disability the ability of their brain to take in this information and of course these children should be protected from harmful plants.