Steve Smith sends this graph:
In the political science literature, DW-NOMINATE scores a the most prominent measure of the ideology of members of Congress. This graph plots the standard deviation in those scores from the 84th through 111th Congresses (basically 1955-2010). The larger the standard deviation, the more ideological heterogeneity there is in the party.
In the earlier part of this period, the Democrats were clearly more heterogeneous, as one might expect in a party with defined liberal Northern and conservative Southern wings. But over this period, ideological heterogeneity in the Democratic Party decreases substantially. By about the 104th Congress (after the "Republican Revolution" of 1994), the parties are equally heterogeneous. And that has continued to be true. Via email, Smith says:
There is hardly any difference since start of the Gingrich era (he became Republican whip in 1989). "Gingrichism" encouraged a disciplined party to sharpen differences (he thought the public was on his side) and force Democrats to cast more difficult votes (he thought this would put Democrats from conservative districts in danger). Republicans gave this strategy credit for the 1994 victory. For detail, see Barbara Sinclair's Party Wars.
Democrats' reputation for less discipline, or at least less cohesiveness, was certainly deserved in the period between the late 1930s and 1980s.
Of course, DW-NOMINATE scores are based on roll call votes, as are party unity scores. But political scientists still interpret these trends as reflecting genuine changes in the ideological composition of the party, not simply as evidence that the parties are smarter about which bills they bring to the floor. Indeed, it's hard to imagine that the conversion of conservative Southern Democrats to Republicans, or their replacement by Republicans, wouldn't have genuine ideological implications. For these same reasons, party unity scores are relevant to the question of party discipline as well.
In sum, both parties, and especially the Democratic Party, have become more ideologically homogeneous and also more polarized. As a consequence, congressional representatives of both parties are more willing to empower leaders, precisely because members can be confident that leaders will follow what the party rank-and-file want. (This is conditional party government theory.) As a consequence, there is also growing unity in party behavior on the floor.
Most importantly, these data provide another piece of evidence counter to Westen's notion that Democrats are fundamentally less cohesive or disciplined than Republicans. See also my two previous posts.
