Harvey Goldsmith’s comments come after Sheeran said he had changed his stance on the charity single ‘Do They Know It’s Christmas?’