I realize that is not what he said to his son, or he might have, he didn't really say how he started the discussion. Just that his son using the phrase 'Oh God' might upset his grandparents. Odd that if he is atheist, why is 'Oh God' used in his household?
I use "Good Lord!", "Oh Jesus!" "Oh God", etc. all the time. It's part of the English cultural vernacular. I don't limit myself from any language regardless of source. It certainly doesn't make me any less atheistic.