How the Netflix Prize Was Won
Wired News (09/22/09) Van Buskirk, Eliot

The secret to the success of BellKor’s Pragmatic Chaos and The Ensemble, first- and second-place winners of the Netflix Prize, was teamwork. The Netflix Prize offered $1 million to the team that could improve its movie recommendation algorithm by 10 percent. Both of the winning teams combined the strengths of several smaller teams that had worked independently before being absorbed by a larger group.

“In combination, these teams could get better and better and better,” says Netflix’s Neil Hunt. BellKor’s Pragmatic Chaos and The Ensemble independently combined their members’ algorithms to design more complex ones that represented everyone’s input.

Rather than appointing a few team leaders that did most of the designing, the teams say they worked communally and thus increased their strength–even if some of the ideas seemed unrelated to the initial problem.

One of the algorithms BellKor’s Pragmatic Chaos used tracked the number of movies a Netflix viewer rated in one day. People who rate a large number of movies at once have most likely seen them a while ago, which affects their judgment. Although this data would not have contributed to the team’s success alone, when combined with other algorithms it increased the group’s performance.

The Big Chaos team, which was incorporated into BellKor’s Pragmatic Chaos, found that users rate movies more negatively or positively depending on the day. By providing an algorithm that took this seemingly irrelevant data into account, Big Chaos helped the winning team succeed. “One of the big lessons was developing diverse models that captured distinct effects, even if they’re very small effects,” says The Ensemble’s Joe Sill.