Sunday, November 28, 2010

Using probabilities to predict election results

The recent mid-term elections, especially the Senate results for Colorado and Nevada, made me wonder about the efficacy of likely-voter screens that pollsters normally use. Rather than discarding all "not-likely" voters, could one assign probabilities to different sub-groups formed based on likelihood-of-voting?
Thanks to Mark Blumenthal, I now know that the CBS/New York Times poll uses probabilities instead of a "likely voter" screen. Here is Mark Blumenthal's write-up on the method and its results for Presidential elections till 2004. How did the poll do in 2008?
First, all the polls (from the lovely charts at the erstwhile Pollster.com, now part of HuffPo):


Second, the CBS/Times polls:


Much more consistent than some other polls, including Rasmussen and USA Today/Gallup, for sure. You can click on any of the points to see the raw trend instead of the Pollster.com-generated trend line. The final result, of course, was Obama 52.9%, McCain 45.7% (Wiki). While the "all polls" trendlines finally converge to the actual results, the CBS/Times poll suggests the race was relatively consistent throughout the fall - especially the CBS/Times-specific trendline.

Here is a description of Registration-Based Sampling, which uses voter registration history to contact only those voters with a history of voting, rather than a self-reported history provided by randomly-contacted voters (Random Digit Dialing).

(Much thanks to Mark Blumenthal for the links: Follow him on Twitter, and on the new Huffington Post/Pollster website. Any errors in data interpretation are purely my fault.)

Wednesday, November 3, 2010

Cautiously optimistic about Colorado Senate & Michael Bennet

Currently (0146 Wednesday Nov 3), per MSNBC, Senator Michael Bennet and challenger Ken Buck are separated by a few thousand votes. Nate Silver and Dave Weigel appear to think Denver, Boulder and Arapahoe counties have not reported a large fraction of their results, so Senator Bennet's a shoo-in. While I am obviously a Bennet partisan (having volunteered long hours, including 12+ hours today), I believe the race is much, much tighter, though I would bet that Senator Bennet wins.
Here are the gory details:
1. Boulder County estimated total turnout at 119k, out of 164k "active" voters; as of writing, 86459 votes have been counted, including 228 polling-day-votes. Senator Bennet leads 66.5-29.2. If the County estimate is correct, that means an additional 12k vote gain for Senator Bennet.
2. Denver County has 273,922 "active" voters. In the last General/Governor mid-term, 2006 (PDF), "active turnout" was 58.56%. If that remains constant (got a text from Hickenlooper that turnout was low!), that means a total vote of 160,408; so far, 149k votes have been counted, and Bennet leads 71-24.7. So there's an additional 5k vote gain for Bennet.
At the other extreme, in 2008 the active turnout was 89% - that means another 95k votes are still out there, and Bennet easily wins. I somehow doubt this high turnout was the case (see Hick's text from earlier.)
UPDATE (Wed 0800): As of 0415 Mountain, Denver County's FINAL unofficial tally is 174k votes in Denver, with Bennet winning 71.2-24.4. So that's a lead of 81,450 votes in Denver County, compared to 69,060 when I wrote this post last night. So an additional 12k vote gain rather than an additional 5k gain, thanks to 5% higher turnout than in 2006 - 63.6% in the Senate race.

3. Arapahoe County appears to have ALL its vote counted, with a turnout of 164k out of 273k "active" voters, despite what the NYT says ("11% reporting").
UPDATE (Wed 2000): Denver Post says that as of this morning, 30k votes were still out; as of 5 PM, the County reports 180k votes tallied, with Bennet up 49-45.7. So even if another 15k votes are out there, Bennet will still win!
4. Weld and El Paso counties appear to be almost all done, as do most of the rural Colorado counties - ASSUMING the afore-mentioned NYT report is correct. Still, with the current totals including 170k votes from El Paso county (Colorado Springs!), I doubt there are many more votes there.
UPDATE: Weld County, Ken Buck's home, do appear to have all their ballots counted. El Paso, OTOH, has a sucky website.
UPDATE (0830 Wed): Per the NYT, El Paso county is 100% in, and Ken Buck gained (I think) another 3k votes. SENATOR MICHAEL BENNET IS AHEAD BY ALMOST 7500 VOTES, BOULDER STILL COUNTING!

Add up 1 & 2, we get (at a minimum) about a 17k additional vote gain for Senator Bennet from Denver and Boulder - a potential victory margin of 10k votes for the incumbent Democrat. (UPDATED/CORRECTED: This is BIGGER THAN the 0.5% margin of the highest vote-getter, or 3,900 votes - NOT the vote-TOTAL - that triggers a recount.)

As I said, cautiously optimistic.

FINAL UPDATE (Sat AM): With 97% precincts reporting, Michael Bennet has won by 15k votes - thanks to an increased turnout in Denver county.