The NetFlix Prize

Suppose I were to tell you how much I liked or disliked 10 movies that you have seen. I rate each with a number from 1 to 5. Now, suppose I name 10 other movies that we've both seen and ask you to guess what I would rate those 10. Could you do it? And, how accurately?

Here's an example of rated-movies:

• Turbo: A Power Rangers Movie (5)
• Lemony Snicket: Series of unfortunate events (5)
• Million Dollar Baby (5)
• Macbeth (4)
• The Simple Life of Noah Dearborn (4)
• Terminator (4)
• The Untouchables (4)
• Thomas Crowne Affair (4)
• Blade Runner (4)
• Cold Mountain (3)

And here's a list for which one must guess the ratings:

• Nell
• Inside Man
• The Constant Gardner
• Stuart Little
• Mr. and Mrs., Smith
• Harry Potter and the Chamber of Secrets
• Monty Python and the Holy Grail
• Die Hard
• The Way we Live now
• Kiki's Delivery Service
• Maverick

Retailers like Amazon have to make recommendations based on previous user-ratings. They try their best to minimze the error (assuming they have enough user-ratings from the particular user, that they can work with). The online movie-rental companies try to do the same.

NetFlix claims that when they make estimates, on "average" the guess they make is about +/- 0.95 (about 1 off) from the real rating that the user eventually give to the movie. Frankly, on a 5 point scale, with the bulk of actual ratings on 2,3, and 4, that isn't too good. They want to improve their ratings.

They've set the following target: they want to reduce their "average" error from around +/- 0.95 to around +/- 0.85

If that does not sound ambitious, listen to this: they will pay a million dollars ($1,000,000) for anyone who can figure out how to do it.

Yup, that's the latest web-based contest. NetFlix will provide people with sample data containing thousands of movie ratings from their customers. They will then give you a set of Customer-Movie pairs and ask you to guess what the rating was for each pair. They will compare that to the real rating. Also, the best team that gets an error of +/- 0.94 or better will get a $50,000 "progress prize", but if any team can get their error below +/- 0.85, the $1,000,000 is theirs.

The competition was announced on October 6th and already 20 teams have beaten the 0.95 hurdle; so one of them will get the $50,000. The best team is down to an "average" error of 0.91

I think it's pretty innovative of NetFlix to solicit ideas this way.

Integration of Values, Integration of values and Action

A character in Atlas Shrugged says, “Tell me what a man finds sexually attractive and I will tell you his entire philosophy of life”?

As I wrote on OO.net, I think the quote should be read primarily as a philosophical statement, about the inter-connectedness of values. In particular, it is an explicit philosophical rejection of the ideas that sexual values have nothing to do with other values.

OTOH, I'd caution reading this as a statement about applied psychology. In particular, I'd caution against thinking that it is really simple: one observes a woman and from that one will actually infer the philosophy of her boyfriend. Without knowing a lot of detail about their relationship and his motivation, one will likely be jumping to conclusions.

To take an analogy, consider a statement like this: tell me the nature of the billiard ball, the nature of the table and room, and the nature of your shot, and I will tell you the exact course the ball will take when you hit it.

The "why" of certain sexual values can be pretty hard to figure out. So, while it might be true that one can tell a lot if one questions the man about his sexual values and about why he finds a particular woman attractive, gleaning such facts from casual observation can lead to hasty judgement.

Today, I read something similar that Warren Buffett said: "Tell me your heroes, I’ll tell you how you’ll turn out". It's similar to the quote from Atlas Shrugged in that they are both saying: your philosophy and basic values determine what you admire, be they in heroes or people of the opposite sex. [Again, this should not be taken too literally as a simple way to predict how someone will turn out.]

An activist fund

Suppose you are a billionaire who wants to contribute about $500,000 a year to fight against environmentalism. Also suppose that you have many millions invested in safe index funds (apart from your other more active investments).

Here's an idea for activism (not mine):

1) Instead of giving $500,000 every year, donate $10 million to an anti-environmentalism organization. The entire amount will be returned to you after a certain number of years, or whenever you decide to stop your activism.

2) The organization invests the money in an extremely diversified portfolio of S&P500 companies. The idea here is not to use any special financial expertise. The idea is not to try to do better than your S&P500 index fund. The idea is simply to replicate the results of an S&P500 fund, mechanically. So, for instance, $10 million across 500 companies would be $20,000 in each. That's enough to buy about 400 shares in most S&P500 companies.

3) Assuming that the S&P500 (including dividends) goes up 5% a year, the organization can withdraw about $500,000 out of the corpus each year.

Now comes the part that's different from other endowments:

4) The couple of guys employed by this anti-environmentalist organization are not really fund managers. They are activist/campaigners against the environmentalism-friendly policies of S&P500 companies. They can do this more effectively now, because they are shareholders!!

So, they might:

• ask PG&E to "disclose their involvement in state efforts to establish Kyoto Protocol-like, mandatory greenhouse gas (GHG) reductions for electric power plants".
• question GE for siding with global-warming advocates
• ask Citigroup to diversify its "environment partners" program
• urge insurance companies to conduct their own independent analysis of the connection between warming and weather-related losses
• ask Goldman Sachs to review its environment policy
• ask Microsoft to reverse its policy of phasing out PVC plastic
• ask Pepsi to disclose the business rationale behind certain charitable contributions
• lobby other JPMorgan Chase shareholders to have the company's lobbying efforts directed away from global-warming and toward litigation reform

Of course, a few shares in each company aren't sufficient to force any type of change (to say the least). However, it buys one a platform, and a hearing. With some managements, it could raise a question about the universality of the environmentalist ethic. It can put a damper on managers who want to use company funds for their pet environmentalist charity. It could get other owners -- like major fund managers -- asking questions about some company policy that is costly and only makes sense if one accepts environmentalist propaganda.

An effort like this does not need to be funded by one, single $10 million corpus. Instead, it can be funded as a mutual fund. So, a person investing (say) $2,000 in the fund, is giving up (at, say, 5%) $100 per year in foregone income. If 5,000 people invest $2,000 each, one has enough.

The above is my description of how the "Free Enterprise Action Fund" works. It is run by Steve Milloy, who started JunkScience.com . It has a little over $5 million and has been in operation for a few months now. I am a small but happy "donor/investor".