The atheist could reasonably argue that his sensibilities are offended by the religious symbols he sees all around him, and that there ought to be a law to protect the sensitivities of atheists.
I don’t think the religious should have laws to protect their fragile sensibilities, and I’d like to think that most atheist’s sensibilities are not so fragile as to require the law to protect them. The religious have long demonised atheism and secularism whilst demanding protection for their bigoted and misogynistic sermons, and they should be free to continue to do so. Just as I should be free to call their sermons fanciful, bigoted and misogynistic. Things are much simpler that way.