Comments on: SVD Recommendation System in Ruby http://www.igvita.com/2007/01/15/svd-recommendation-system-in-ruby/ A goal is a dream with a deadline. Fri, 29 Aug 2008 04:41:46 +0000 http://wordpress.org/?v=2.5.1 By: Ilya Grigorik http://www.igvita.com/2007/01/15/svd-recommendation-system-in-ruby/#comment-106366 Ilya Grigorik Fri, 18 Jul 2008 12:38:11 +0000 http://www.igvita.com/blog/2007/01/15/svd-recommendation-system-in-ruby/#comment-106366 It's hard to generalize between SVD, k-NN, or any other algorithm for that matter - it all depends on the dataset. Depending on the properties, sometimes a simple decision tree can often outperform the rest. Looking at the Netflix challenge: some of the most successful early attempts were with k-NN algorithms, SVD folks have had moderate success as well. However, now the trend seems to be towards blending: combining these predictors to capture latent variables and hence improve the performance. Machine Learning is as much black art as it's science. ;) It’s hard to generalize between SVD, k-NN, or any other algorithm for that matter - it all depends on the dataset. Depending on the properties, sometimes a simple decision tree can often outperform the rest.

Looking at the Netflix challenge: some of the most successful early attempts were with k-NN algorithms, SVD folks have had moderate success as well. However, now the trend seems to be towards blending: combining these predictors to capture latent variables and hence improve the performance.

Machine Learning is as much black art as it’s science. ;)

]]> By: ljdk http://www.igvita.com/2007/01/15/svd-recommendation-system-in-ruby/#comment-106349 ljdk Fri, 18 Jul 2008 09:15:02 +0000 http://www.igvita.com/blog/2007/01/15/svd-recommendation-system-in-ruby/#comment-106349 Really great post Ilya! One thing that makes me wonder is how does SVD compare to k-NN in terms of performance and quality i.e. which one should give better results in general. Are there betters ways in collaborative filtering field or folks at amazon or netflix probably use just modifications of those as well? Could you shed some light on that? Thanks. Really great post Ilya!
One thing that makes me wonder is how does SVD compare to k-NN in terms of performance and quality i.e. which one should give better results in general. Are there betters ways in collaborative filtering field or folks at amazon or netflix probably use just modifications of those as well? Could you shed some light on that? Thanks.

]]> By: Ilya Grigorik http://www.igvita.com/2007/01/15/svd-recommendation-system-in-ruby/#comment-103824 Ilya Grigorik Fri, 06 Jun 2008 12:06:46 +0000 http://www.igvita.com/blog/2007/01/15/svd-recommendation-system-in-ruby/#comment-103824 SAK, absolutely.. The model certainly applies, the implementation might be a different story - depending on the size of the dataset, you might have to look around for some C libraries. :) SAK, absolutely.. The model certainly applies, the implementation might be a different story - depending on the size of the dataset, you might have to look around for some C libraries. :)

]]> By: SAK http://www.igvita.com/2007/01/15/svd-recommendation-system-in-ruby/#comment-103646 SAK Tue, 03 Jun 2008 11:37:42 +0000 http://www.igvita.com/blog/2007/01/15/svd-recommendation-system-in-ruby/#comment-103646 hey nice post Ilya. I was just wondering if this would work on netflix dataset. Matrix computations are tedious you see.. SAK hey nice post Ilya.

I was just wondering if this would work on netflix dataset. Matrix computations are tedious you see..

SAK

]]> By: Ilya Grigorik http://www.igvita.com/2007/01/15/svd-recommendation-system-in-ruby/#comment-103561 Ilya Grigorik Thu, 29 May 2008 13:45:41 +0000 http://www.igvita.com/blog/2007/01/15/svd-recommendation-system-in-ruby/#comment-103561 Sanny, the idea behind SVD is to 'compress' the matrix - when you reduce it's dimensionality, you're using less bits to extract an equivalent (or an approximate) matrix, which means the amount of noise is reduced. Hence, you can capture latent variables, and produce a 'generalization', instead of working with the original / noisy data. Sanny, the idea behind SVD is to ‘compress’ the matrix - when you reduce it’s dimensionality, you’re using less bits to extract an equivalent (or an approximate) matrix, which means the amount of noise is reduced. Hence, you can capture latent variables, and produce a ‘generalization’, instead of working with the original / noisy data.

]]> By: Sanny Cheng http://www.igvita.com/2007/01/15/svd-recommendation-system-in-ruby/#comment-103490 Sanny Cheng Mon, 26 May 2008 07:47:27 +0000 http://www.igvita.com/blog/2007/01/15/svd-recommendation-system-in-ruby/#comment-103490 Hey, thanks for the great post! Could you explain the motive behind decomposing the matrix? What if you just took Bob’s preference vector and compared it (using the same cosine/Pearson dot-product you do anyway) to each other user? Hey, thanks for the great post!

Could you explain the motive behind decomposing the matrix? What if you just took Bob’s preference vector and compared it (using the same cosine/Pearson dot-product you do anyway) to each other user?

]]> By: fcamel’s blog » IR and DM algorithms codes http://www.igvita.com/2007/01/15/svd-recommendation-system-in-ruby/#comment-102410 fcamel’s blog » IR and DM algorithms codes Sun, 13 Apr 2008 04:43:33 +0000 http://www.igvita.com/blog/2007/01/15/svd-recommendation-system-in-ruby/#comment-102410 [...] SVD Recommendation System in Ruby [...] [...] SVD Recommendation System in Ruby [...]

]]> By: Ilya Grigorik http://www.igvita.com/2007/01/15/svd-recommendation-system-in-ruby/#comment-102283 Ilya Grigorik Mon, 31 Mar 2008 12:28:11 +0000 http://www.igvita.com/blog/2007/01/15/svd-recommendation-system-in-ruby/#comment-102283 Tomasz, that is awesome! Quiet the exercise, but a very interesting one to follow. Pjm, thanks for the link. I've also had a few emails with that exact question, I'll point them your way. Tomasz, that is awesome! Quiet the exercise, but a very interesting one to follow.

Pjm, thanks for the link. I’ve also had a few emails with that exact question, I’ll point them your way.

]]> By: pjm http://www.igvita.com/2007/01/15/svd-recommendation-system-in-ruby/#comment-102275 pjm Sun, 30 Mar 2008 17:13:04 +0000 http://www.igvita.com/blog/2007/01/15/svd-recommendation-system-in-ruby/#comment-102275 This is drifting from the original post, but I see other readers commenting here also had problems installing <code>linalg</code> on Mac OS X systems. I plowed through the installation, starting with building LAPACK from source, and <a href="http://www.commonmediainc.com/2008/03/24/building-lapack-and-rubys-linalg-on-mac-os-x/ " title="Building LAPACK and Ruby's linalg on Mac OS X" rel="nofollow"> wrote up the process which worked for me</a>. I would be happy to hear if this works for others, or if I've missed any steps. This is drifting from the original post, but I see other readers commenting here also had problems installing linalg on Mac OS X systems. I plowed through the installation, starting with building LAPACK from source, and wrote up the process which worked for me. I would be happy to hear if this works for others, or if I’ve missed any steps.

]]> By: Tomasz Korzeniowski http://www.igvita.com/2007/01/15/svd-recommendation-system-in-ruby/#comment-102274 Tomasz Korzeniowski Sun, 30 Mar 2008 16:19:39 +0000 http://www.igvita.com/blog/2007/01/15/svd-recommendation-system-in-ruby/#comment-102274 Great post!!! Thank you. Ilya, I was inspired by your post and have written an entry how to compute SVD with pencil plus a notebook ;-) I hope it will be usefull. http://tomek.codequest.eu/2008/03/17/how-to-compute-svd-and-conquer-the-world-before-breakfast/ Once again... thanks for a very nice article. Great post!!! Thank you.

Ilya, I was inspired by your post and have written an entry how to compute SVD with pencil plus a notebook ;-) I hope it will be usefull.

http://tomek.codequest.eu/2008/03/17/how-to-compute-svd-and-conquer-the-world-before-breakfast/

Once again… thanks for a very nice article.

]]>