You do say that your brickwork is modern. So I wouldn't have thought that issues about flaky brickwork were relevant.
A few years ago I had some ivy covering an old brick wall, which I removed. As everyone says, it leaves a heck of a mess.
What I did was to paint the wall with masonry paint. You can get paint mixed to any concievable shade at many larger DIY stores, and I chose a shade to match the brickwork as best as I could.
Today, several years on, you wouldn't notice that anything has been there.
I wouldn't recommend doing that, but it's an option.