Some helpful replies. Most poster know about my neighbour, as regards the fence which sounds about the same hieght as the one built by my neighbour.I built a trellis a ft into my property to grow stuff on, which has been fine but the fence is falling down into the garden now. The jury's still out as to what to do about that little problem, I put a lock on the gate so he has to knock on the front door to talk about it. The point is he can't enter my garden without me knowing about it, he certainly can't lift the fence and just stroll in.
There have always been children in the house on the other side, it's rented but the owners did once live there and are actually nice people. I got sick of the family with lots of children who would play cricket in the garden every summer. They would knock on the front door though for their balls but spending entire afternoons answering it was no fun. In the end we took out a narrow panel near the house so they could just come and get their balls so long as they hadn't landed in the veg plot and we agreed if anyone hit the ball over the fence they lost a run instead of gaining one.
Then there is the current family with small children who complained the cat was pooing in their garden, I bought them water pistols and they spent several fun filled days squirting the cat if she went into their garden, the rule was they could only squirt if she was on the fence or in their garden but not if she was sat at the gap in the fence and on my side. She soon learnt to sit at the gap and the children would come over and stroke her. They were most upset when she died.
The point I'm making is you can sometimes work it out with some neighbours but not with others. I find it easier just to ignore neighbour from hell but keeping a diary of not nice stuff helps if you want to make a complaint about them, taking pics also helps so long as you tell them why your doing it.