I agree about the watering. If you don't water enough the year before a camellia won't flower the next. I water mine (also in a pot), very well, twice a week during the previous summer, maybe even more if it is very hot and mine has flowered wonderfully every year for 4 years.
(One problem I did have this year was due to the mad weather - the buds were all coming out nicely then we had that ridiculously hot couple of weeks in March and a lot of them dropped off or didn't come out. Now the weather is back to normal the buds are opening up again they way they should. I know that won't explain your problem for the past three years but if yours is in direct sun maybe it doesn't like that?)