This, by Dani Rodrik, is an important point:
the difference of views has nothing to do with the economics of immigration, on which I think we all agree. Expanded immigration is likely to exert downward pressure on workers' wages in the U.S. Where we disagree is on whether the gains to the rest of the world make this still a worthwhile effort (in the context, of course, of efforts to cushion the adverse effects on U.S.). As Alex Tabarrok points out in a recent post, the differences have to do with what we think is the relevant moral community for making public policy decisions. George thinks the purely national perspective is the right one, and he figures the aggregate gains for the U.S. are small relative to the distributional costs, which makes this bad policy. For my part, I believe cosmopolitan considerations should enter our calculus when the gains abroad (or to foreign nationals) are sufficiently large, which they would be with temporary labor flows.
The problem with cosmopolitan calculations is that they're relatively arbitrary. Whatever my intellectual thoughts on the subject, I don't, in policy analysis, give a dollar of gains to Bolivians the same weight I give a dollar of gains to Americans. So I wouldn't support a policy that raised aggregate happiness and economic well-being across the Americas by significantly eroding those metrics in the United States. But between those two extremes, I'm not precisely sure where the cut-off lies.
Thankfully, none of the immigration policies currently on the table will have a massive negative effect on Americans, and in fact may do quite the opposite. The immigration status quo is a worst-of-all-worlds for workers, as illegal immigrants exist outside the wage standards and labor laws that help level the playing field and avoid a high-speed race to the bottom. When these workers' rights can be enforced, they will no longer be so cheap relative to native-born workers, and thus native born workers may see their other advantages -- linguistic, cultural, etc -- win the day. If you really want to screw the low-income workforce though, keep the situation static, and force them to continually compete against a workforce that's paid less than minimum wage and isn't subject to the safety standards and sundry other regulations of legal workers.