I mostly use loose tea in tea-balls, which works out much much cheaper than tea bags and the tea leaves go straight in the compost bin. We also grind our own coffee beans for the Bodum coffeee jug, which is much more economical than any of those dosette thingys, tastes batter, and gives is grains we can put on the garden to deter slugs (not that we have found that 100% effective, I have to say!).
I'm not that bothered about the odd teabag ending up as "little papery skeletons". I recycle ALL my brown cardboard boxes as garden mulch (dig out weeds, apply 2 or 3 x layer of cardboard boxes, then apply 10cm minimum layer of stuff from garden shredder) and when I have to weed that section again there is frankly not a lot left of the cardboard, so a few teabags isn't likely to worry me much.