“I'm not one of those who wants to purge our society of our Christian history.”