Your potatoes might have blight, or they might just have reached their natural end, or did they need watering? Anyway, if they've got to the stage of having fruited they're probably ready to harvest now. They won't continue to grow if they have no leaves.
I'd dig them up on a dry day, leave them in the open for a few hours just to set their skins and then store them in the cool and dark and use them.
By the way, yes the seeds are poisonous but that's ok, they're a normal part of the plant - just don't eat them!