It's only racist because the media, liberals, never-trumpers and Democrats want to set an agenda and define an agenda for Trump. Trouble is Trump doesn't bluff or fold. It's only racist because they want it to be so they declare it to be.
Trump was simply saying if you don't like the country then why are you here? If you don't like it then go back home rather than come here and attempt to change it.