Is it "growing" out through the nestbox hole, or the roof, or what? Is it in a place where it is a nuisance to you or your neighbours? Bearing in mind that wasps are carnivorous and eat insects which eat our plants, if it isn't, personally I'd leave it alone. They'll be gone when the frosts arrive anyway.
If you're bent on destruction, as sotongeoff says, you should wait until dusk, when they have all returned to the nest. For safety's sake, I'd wear thick trousers, tucked into your socks, and a long-sleeved sweatshirt or similar. Wear gardening or thick rubber gloves. When I had one of these to deal with in my mailbox I found an old chiffon scarf and a straw hat and made myself a beekeeper-style head covering. That might have been slight overkill, but it made me feel better.
You should be able to get big aerosols of wasp & hornet killer frorm garden centres. Don't try to use a domestic fly & wasp spray.
If the nest is semi in the open it might be a good idea to try and surround the nesting box with a large bin bag or garden refuse bag to keep the wasps in and to concentrate & contain the poison. We don't want any blue-on-blue collateral damage to your bees etc! Give it a good burst of spray and retire. Come back in the morning (cautiously) and you should find it safe to clean out the nesting box.
