There's a great conversation going on in the comments of my earlier post on felony disenfranchisement, and I meant to publish a response to Gulliver's comment, but for some reason it didn't publish, so I figured I'd just do another post.
My girlfriend and I have a running argument on this issue (fun life, eh?), and her position is much the same as yours: "I just think that if yo do your time, then you do your time." But what about gun ownership? What about residency restrictions/registration requirements for sex offenders? What about the myriad other ways we've decided it's fair for the state to renegotiate the social contract once a citizen has breached it?
I'm not a fan of sex-offender registries, and I don't think having a criminal record of any kind should prevent someone from owning a firearm. We shouldn't be treating someone convicted of theft or writing bad checks like an enforcer for organized crime. But both of those measures are at least rhetorically defensible as curtailing rights in addressing an actual public-safety problem. Someone with history of gun violence can't shoot you in the face with their vote.
I'm less sympathetic to your census issues; how exactly do politicians "draw power from a constituency that is voiceless and literally captive"? Another way to view it: those politicians' districts are treated with a weight commensurate with the population they are expected to serve, not the population that participates in their election.
It's true, politicians serve those who reside in their district, not simply those who vote. But the difference is that prisoners and formerly incarcerated people who are disenfranchised are prohibited from voting; they aren't simply choosing not to vote. If you're going to count for the purposes of apportionment, you should have the ability to decide whether or not to cast a ballot. In the case of prisoners being counted as constituents, the people in question are literally captive (behind bars) and voiceless in the sense that they have no say in who represents them.
