'Fifty eight percent of my constituents voted to remain and you want me to tell them to clear off?'