Marvin Ammori has written a great post on why the U.S. shouldn't prosecute Julian Assange, not because he's a hero but precisely because jettisoning freedoms in the pursuit of unsympathetic characters is what leads to those freedoms being abandoned.
Many of our nation's landmark free speech decisions are not about heroes–several are about flag-burners, racists, Klansmen, and those with political views outside the mainstream. And yet we measure our commitment to freedom of speech, in part, by our willingness to protect even their rights despite disagreement with what they say, and why they say it.
If rights only apply to people who "deserve" them, then they aren't "rights." There's a lot more to the post, but I just think the above sentiment can't be repeated enough. You start throwing rights out the window because "bad guys" are involved, and ultimately "good guys" will find themselves without them.