Comments on: Dissecting the Netflix Dataset

By: data warehousing and mining

data warehousing and mining — Wed, 21 Mar 2007 16:27:42 +0000

data warehousing and mining

Here

By: » Evolving Datasets and the Netflix Prize [ Data Sciences Analytics ]

» Evolving Datasets and the Netflix Prize [ Data Sciences Analytics ] — Mon, 19 Mar 2007 09:36:02 +0000

[...] For a good look at some of the characteristics of it, see Dissecting the Netflix Dataset [...]

By: Markus

Markus — Sat, 17 Feb 2007 05:33:22 +0000

It was quite useful reading, found some interesting details about this topic. Thanks.

By: Kris Brower » Blog Archive » Netflix Data Statistics

Kris Brower » Blog Archive » Netflix Data Statistics — Thu, 01 Feb 2007 11:31:21 +0000

[...] I found some interesting analysis of the netflix prize dataset, but it was not exactly what I was looking for. I downloaded the dataset, and decided to do some analysis myself. I went with what I though was the easy route by installing pyflix, which in hindsight might not have been the fastest approach(pyflix took about 90 minutes to install after I figured out how to install NumPy in Ubuntu). Here are the results of my analysis of movie ratings over time. [...]

By: Ilya Grigorik

Ilya Grigorik — Mon, 01 Jan 2007 15:27:56 +0000

Alexandre, very good points. I’ve never been a Netflix user myself, but route #1 certainly makes sense. Your second proposition made me chuckle. This is exactly what we’re pursuing in this competition, but somehow, it never occurred to me that Netflix could have applied the same strategy themselves! :)

By: Alexandre Rafalovitch

Alexandre Rafalovitch — Sun, 31 Dec 2006 23:18:21 +0000

> Are early adopters (techies) more discerning in their ratings/choices? Are the movies getting better? (Doubt it!)

I can see two options here:
1) Movies (carried by NetFlix) did get better. I think it was fairly recently that they started buying festival’s movies as well as adding other sources that had less ‘popular’ but more interesting movies.
2) Algorithms. If their own suggestions algorithms got better over time, people will get better movies to rate.

By: Statistical Modeling, Causal Inference, and Social Science

Statistical Modeling, Causal Inference, and Social Science — Sat, 23 Dec 2006 18:38:10 +0000

Distributions of rankings

A few postings ago, Andrew wondered about the shape of the long tail. The OneEyedMan’s comment reminded us that the extensive NetFlixPrize dataset contains information about almost half a million users’ ratings on almost 20000 movies. It’s an excell…

By: Ilya Grigorik

Ilya Grigorik — Wed, 01 Nov 2006 22:05:42 +0000

Chris, that’s an interesting point. I didn’t know Netflix allowed half-star ratings before. However, looking at my data, it seems like they must have converted all such ratings to whole numbers. I’m not able to find a single half rating.

Now the questions is, when they rounded the numbers, did they round up or down? :)

By: Chris

Chris — Wed, 01 Nov 2006 17:08:06 +0000

In response to your question: “in 1998 the average was 3.4, by 2005 it steadily moved to 3.8. I wonder what accounts for the drift?”

Netflix used to allow for half star ratings. I would imagine that this would have an affect on the scores. Are people more likely to “round up” their ratings (a 3.5 would be a 4 vs a 3) than down?

Can you isolate the ratings made during the time when half star ratings were possible vs when they were not?

Maybe Netflix needs to reconsider going back to a half star rating system to improve the accuracy of their predictions before worrying about designing a new algorithm?

By: Ilya Grigorik

Ilya Grigorik — Tue, 31 Oct 2006 00:41:48 +0000

Grant, I added a log-log plot right at the bottom of the post.

Technically, you will never find a true Power law (Zipf, Pareto) distribution in real life simply because you will never have a sample of infinite size - you will always have a drooping tail. However, just as you have pointed out, the drooping tail itself can give us a lot of information. In this case, it looks like Netflix could really push the demand further down the tail by adopting more niche titles.