I also tried to bring out the point that while news articles on the subject focus a lot on the programming side of things, there’s a strong foundation on, among others, psychology and statistics too and that there are already good algorithms and one major challenge is the data in it’s own right.

I did not focus that much on implementation or user experience, because those are things that I’ve no idea about. For example, how to solve stock vs. flux, I just threw those questions to air. The approach you described sounds like a practical one.

What do you see as the challenges in this field?

http://en.wikipedia.org/wiki/Collaborative_filtering

What you call the pandora way is called the content based approach.

And i doubt a lot about this phrase : “Most of the collaborative filtering things today are just toys so none of this really matters.” Because i think you miss the point of the user experience trying to focus too much on the theoritical approach.

Your problem on a new movie should have end up on a discussion about stock vs flux. Back to the content based approach that will slowly leave the user/user Cf approach as the ratings flows in. Like in real life, interest for james bond is based on james bond, actors, director … etc until we have some newspapers/friends reviews.

It’s well written but for me it is too much or not enough.

Will be happy to talk with you about it next time your write an article on the subject.

It’s all just a number and the guys crunching the numbers are supposed to find out if someone’s just overrating movies. You also point out a good point why assuming everyone’s ratings as objective is pretty flawed. But, large enough sample and this average out, right? =)

The problem is that we’re not looking at aggregates, but trying to predict on individual level where averages aren’t that useful. Of course Netflix could optimize its operations based on just aggregate data, and that’s probably worth something too.

When people realise they gave way too high scores just because they were still excited about the movie, do they go back and “correct” them? Or do they think that “well, now when I’ve seen this movie, I think I gave a wrong score to those some movies?” Is it worth the effort to go back and re-rate movies after they’ve got new information on what to base their “more correct” rating?

The human video-clerks making mistakes but the excitement is the uncertainty part I briefly discussed in the end. The inverse of this is why all movies seem to be “safe bets” (sequels, generic story, etc.) because I guess it’s easy to estimate how much they reel in. For “interesting” movies, there’s always the risk. It’s like portfolio management from movie studios’ side, with high enough volume of safe bets, you can afford a couple interesting movies.

That’s a really good point that people aren’t really after good movies, but good experiences. One might wonder if it’s possible to forecast good experience from previous movie ratings (is the experience visible in the number?) And true, if everything’s good, then everything’s medicore, which leads to a problem that should the recommendation systems actually recommend you crap just to recalibrate your ratings. In the long run it’d be good for the user, but in short run you’re just wasting user’s time and money… =)

Strictly speaking, this isn’t AI, but one approach that I didn’t cover here was AI-like methods like machine learning, neural networks, genetic algortihms, etc.

And finally, true, there are countless factors here. One of the challenges is to find out what the important ones are.

In other words, I think its a mistake to draw generalisations from a single instance and a single user. We all have a Bambi, 2001, or Napoleon D. (I only liked the last 5 mins) in our closet, that doesn’t mean that our other ratings are wrong.

About new movies. I don’t know if ever look at IMDB-ratings for blockbusters as they just come out? I’ve come to learn (= be trained) that these movies are usually overrated the first few weeks, after which their ratings normalise—they can’t all be in the IMDB-top 100… About new movies, I’ve also come to learn that if only 50 people vote for a movie, they also usually overrate it, probably because they are the film-maker’s friends. In other words, users like me arent’ stupid and will know that new films can’t be rated perfectly through some magic voodoo.

There was another factor I found interesting in the NYTimes-story, which is that human video-clerks make mistakes, but also bring a form of excitement to the table. People like to be surprised and even extremely bad films can lead to some pretty heated debates afterward—any experience is all about the memories, after all, and if everything is “good,” it eventually becomes mediocre.

As far as pay-back is concerned, I think that either the Wired or the NYTimes story mentioned what kind of numbers, Netflix is renting out, 1 million is peanuts to them + its money well-spent.

Anyway, the more I learn and think about collab. filtering, the more I love it, because there are so many factors that these AIs don’t yet take into account yet.