Leave them alone. They are one of the primary movers in the compost world. They eat decayed or decaying matter and help break it down into the nice crumbly stuff that we like to spread on our soil. Their teeth are not strong enough to eat anything but very soft green material or as said, half decayed stuff. They love dark damp places. Only time they might be a nuisance is if you have lots of them where there is not enough food and they then attack seedlings.
If you have them where you do not want them then the answer is to clear out any rubbish plant material and let in the light. Talcum powder will stop them as they find it difficult to cross it.
Finally if you really have huge numbers then, fried in butter they are supposed to taste like shrimps.