If you can't find an organised trip there's always bus and train and then a local service to the gardens:-
By bus
515 Kingston to Guildford via Surbiton and Wisley. There is no service on Sundays or Bank Holidays. For details call National Traveline on 0871 200 22 33. Visitors who produce their train or bus ticket on the day of their visit will gain entry for £8.69.
By rail
Closest stations are West Byfleet (3 miles) or Woking (4 miles) There are no taxis on standby at West Byfleet. Visitors who produce their train or bus ticket on the day of their visit will gain entry for £8.69.