They are a lot hardier than people think in the UK-I have had one in a pot outside for 5/6 years-although I live on the South Coast we still have had hard winters with snow/ice/frosts and it has survived that completely unprotected.
At that size there is not much you can do if you are worried-other that fleecing it to give it a bit of protection
Of course the big question is -what has happened in previous years and how long have you had it?