Home > Approximation Theory, Data mining, Movies, Scientific Computing, Statistics > Sometimes Math is not the answer

Sometimes Math is not the answer

I would love to stand corrected in this case, though. Let me explain first the reason behind this claim—It will take a minute, so bear with me:

Say there is a new movie released, and you would like to know how good it is, or whether you and your partner will enjoy watching it together. There are plenty of online resources out there that will give you enough information to make an educated opinion but, let’s face it, you will not have the complete picture unless you actually go see the movie (sorry for the pun).

For example, I fell for “The Blair Witch Project:” their amazing advertising campaign promised me thrill and originality. On top of that, the averaged evaluation of many movie critics that had access to previews claimed that this was a flick not to be missed… Heck, I even bought the DVD for my sister before even watching it!—She and I have a similar taste with respect to movies. The disappointment was, obviously, epic. Before that, and many a time afterwards, I have tripped over the same stone. If nothing else, I learned not to trust commercials and sneak previews any more (“Release the Kraken!,” anyone?)

The only remaining resource should then be the advice of the knowledgeable movie critics—provided you trust on their integrity, that is. Then it hit me: My taste in movies, so similar to my sister’s, could be completely different to that of the “average critic”. Being that the case, why would I trust what a bunch of experts have to say? The mathematician in me took over, and started planning a potential algorithm:

  • step 1: go ahead and score a bunch of movies.
  • step 2: A first function would compare my artistic taste with that of another given critic, and decide how much should I trust this person. I assign to each of these individuals a numerical value between 0 and 1: the closer to zero, the least likely I would trust their opinion—in Approximation theory we call this a weight.
  • step 3: Once I have a reliable set of weights, and given a precise movie, I could gather the scores of a bunch of critics and, instead of performing a standard average of their evaluations, do a weighted one.

So far, Math seems to be the answer.

Let’s say that I would like to make this application available to the public (this is where the problem gets interesting). Once new users acquire this product, the first step is, obviously, to let them train the program by allowing to give their opinion about a large number of movies (ala Netflix). The more films they watch, and the more they evaluate them, the better, right? Not necessarily!

Say I am a compulsive and avid SciFi geek, that watches nothing but Star(Wars|Trek|Man), “Galactica Serenity of Thrones,” “Captain Terminator of the Rings,” “Conan, the Batman Alien Avatar,” or whatever. How good will the response of my basic code to, say, “Citizen Kane,” or “West Side Story?”

A decent app should encourage users to watch and provide an opinion on a set of carefully chosen movies (famous or not, good or terrible, fuzzy-feeling or Ôdishon-uncomfortable) If this selection is good enough, we are on business.

I would like to pose this question to all (no matter whether mathematician, statistician or none of the above): What should be the minimum number of movies so that this application works reliably? The question is tricky and subtle: the answer probably depends heavily on the user. And related to this question there is another, more crucial: What would be a good set of movies to choose from?

I am sure that you would include in this list…

This is where I am not sure there is still a good mathematical model to make that choice of movies. Maybe data mining could do the trick. Why don’t you give me a hand, and start posting a selection of about ten-to-twenty films that should be included (and why!)? Remember: they don’t necessarily have to be all masterpieces: as a matter of fact, I believe that you need to have some terrible movies in the list, especially if you don’t believe they are all that bad.

Data Mining: Practical Machine Learning Tools and Techniques, Third Edition (The Morgan Kaufmann Series in Data Management Systems)

  1. Justin James
    April 29, 2012 at 11:11 pm

    I don’t know if it’s still going on, but Netflix has a substantial ($1 million, if memory serves) bounty out for any algorithm that works substantially better than their own (I think the threshold is 10% better in A/B testing). You may want to check that out and a lot of the discussions around it for this project. 😀


  2. Eric Choate
    April 30, 2012 at 2:44 am

    I remember an interview with the guys who maintain the Netflix algorithm and ran the contest. One thing they observed: No matter how many films someone rated, it was impossible to accurately predict whether or not a person liked Napoleon Dynamite.

    I have thought about trying to creating a list of twenty films (it seemed large enough) to give to someone in a getting-to-know-you context to get an idea of how compatible their film taste is to mine, although I was thinking more of a free response than a star rating. I don’t know if I ever really settled on a full list, but here are some that I considered.

    1. Network (do you respond to literate, talkie movies with a cynical sense of humor?)
    2. Eternal Sunshine of the Spotless Mind
    3. Requiem for a Dream (Technical brilliance but it’s tough to watch)
    4. Hedwig and the Angry Inch (It’s relatively obscure, and so if they knew of the film, it would tell you something about the breadth of their film knowledge. Also, if they did know of it, whether or not they wanted to see it, would tell you something about their taste. Even if they didn’t like it, they could be open to something like it. Plus, it’s a musical.)
    5. Something by the Coen Brothers, probably Fargo
    6. 2001: A Space Odyssey (It’s both scifi and classic. Also, I would suspect it has a strong correlation with Solaris, but it would be more likely for someone to have seen it.)
    7. Something by Errol Morris? I think there should be one documentary.
    8. Mulholland Dr. (Can you like something you don’t understand?)
    9. Something by Pixar, possibly Up.
    10/11. Either The Artist and a musical from the 50s or 60s or Singin’ in the Rain and a silent film from the 20s, like The General or Modern Times or maybe The Passion of Joan of Arc. I like The Artist or Singin’ in the Rain because there should be a movie about making movies. Tim Burton’s Ed Wood would also work well on for this, too.
    12/13. Either a Kurosawa film (Rashomon, or maybe Ikiru?) and a recent European film (Amelie?), or a modern Asian film (Oldboy?) and a European classic.
    14. Batman. I mean the one with Adam West.
    15. Field of Dream? A sports movie could be good, but maybe this is too atypical.
    16. Terminator 2, Total Recall, Predator, or Conan the Barbarian
    17. Fight Club? The problem with including it is that I’m not sure how strongly liking it/not liking it correlates with thoughts on other films.

    I agree that there should be a terrible film or two, but I haven’t figured what the best ones should be.

    The key seems to be finding films that strongly correlate with other films, but that it is not a film that everyone agrees about, which seem to be contradictory things.

    I put Titanic on my list only because I haven’t gotten around to seeing it yet. How would you use a response of “I haven’t seen that film”? Should you break it down for reason? “I have never heard of that film,” “I know exactly what film you are talking about and want to stay as far away from it as possible,” or “It’s #350 on my Netflix queue.” Maybe Hostel would be a good film for the list. Everyone knows about it, but it seems like you would have a good mix of people who saw it and liked it, saw it and hated it, and refused to see it on principle.

    • May 1, 2012 at 9:05 am

      I remember having this conversation with you before! (probably in Preston). And I think that your approach is probably a good enhancement to this potential algorithm: rather than a single universal choice of “training movies”, the app should allow the users to pick. What I believe should not change is the set of genres: the code needs to know your “opinion” about sport-movies as well, and in that subset, how you feel about boxing vs. hockey, etc. It is getting more and more complicated but, if you want to be accurate…

  3. Perico de los Palotes (Peter of the Sticks)
    April 30, 2012 at 9:37 am

    Comentario de un ignorante atrevido (y aburrido, jeje…)

    Igual lo he entendido mal, pero no veo que haya problema con tu idea. Siguiendo tu ejemplo, si yo soy un “friki” de la ciencia ficción mis propios gustos me delatarán: en vez de proponer una lista corta sobre la que votar, dime simplemente “vota para empezar las 50 pelis que te dé la gana”. Inicialmente no votaré por Ciudadano Kane porque ni me acordaré de ella, así que mi “peso” cuando alguien pregunte por esa peli será para empezar cero, o casi: no soy un crítico “cualificado” para ese género (porque no lo veo, o si lo veo ocasionalmente ni me acuerdo al pensar en 50 pelis). Probablemente votaré sólo por pelis que me gustan, y las pondré por las nubes, pero servirá para encuadrarme en mi “género”. Más adelante, si se me ocurre ver “Acabator VIII” y resulta que es una castaña incluso para un friki, pues la pondré a caldo y mi voto sí “pesará” entre los demás frikis que ven las mismas pelis que yo. Y si zapeando pongo “La 2” y me trago “Uranus” en francés, cuando me despierte le pondré un cero, pero mi voto no valdrá gran cosa cuando un cinéfilo a la antigua usanza busque la peli.

    Puede que el problema sea que hay muchas posibles pelis que el friki puede elegir y necesitarás una muestra muy amplia para que haya varios votos en todas. Tal vez ayude a acotar la cosa si introduces un “factor experto” tuyo, que asocie las pelis por afinidad. Es decir, si en la votación inicial alguien elige ponerle un 10 a “2001, una odisea en el espacio” y a “Alien: el octavo pasajero”, automáticamente se le casca un 10 en la trastienda a “Blade Runner” aunque no la haya votado inicialmente. Y ese “factor experto” me imagino que puede mejorar aprendiendo a medida que la gente vote más pelis y acote cuáles son “de la misma familia” por obtener votaciones similares repetidas veces de gente que está “cualificada” para ese género.

    Igual todo esto ya lo has tenido en cuenta, o no tiene sentido práctico para lo que quieres (hoy estoy de puente y lo de pensar antes de escribir es para los días laborables) Y por supuesto perdona el rollo.

    Un abrazo, chato. A por ese milloncejo.

    • May 1, 2012 at 9:12 am

      ¡Cuánta razón tienes! El problema es que usando sólo este sistema tan básico de pesos sin haber expuesto al usuario a todo tipo de películas, estás asumiendo que individuos que nunca han visto un determinado género, no les va a gustar. La primera vez que vi una película de Kurosawa, por ejemplo, no tenía ni idea de que me iba a encontrar. Me encantó, y en los siguientes meses (años, en realidad) me preocupé por coleccionar tantas como pude. Ahora las tengo todas, pero no todas me gustan por igual. El algoritmo debería ser lo suficientemente inteligente como para deducir mi reacción *antes* de ver la primera, y quizás recomendarme cuáles de este director me van a interesar más.

  4. Justin James
    May 1, 2012 at 9:08 am

    Eric – Oddly, I’ve only seen 4 or so of the films on that list (though I’ve seen all of the ones you group under “Arnold films”). And I’ve never seen Napoleon Dynamite… though I *have* seen Mullholland Drive, and most of Lynch’s other stuff… and I love them. It’s not hard to understand Mulholland Drive, but only if you’ve seen Lost Highway and Twin Peaks, Lynch keeps making and re-making the same film, just with a different, more obtuse context every time.


  5. Alex
    May 2, 2012 at 11:32 am

    ¡Hola ahijado! I wonder if another metric might narrow down the choices a little more closely. I offer “Thor” as my movie example. Popular concensus was that the movie was absolutely awful (my oldest son saw it and told me not to bother with it) yet I was intrigued by the previews I saw and remained curious about it. On a weekend when I had the kids, we watched the movie and found it to be very enjoyable. Please forgive my boldness but I’m thinking “Likelyhood to like what others consider to be a bad movie” is something that might be addressed in your algorithim somehow. Perhaps if you provide some weighted value to preferred genre(s) (I am more likely to enjoy sifi movies regardless of what other people say) you might be able to zero in on a good match for me. Another question within the genre might be “Do a majority of people in this genre who have seen a given movie like it even though it is panned by the rest of the world (looking towards cult classics)?
    I’d like to try your algorithim once you start alpha testing. My kids tell me my movie tastes are “unique”. I hope we can get together for coffee soon. Hasta pronto. Cuidate. Tu Padrino.

    • May 5, 2012 at 9:40 am

      Thanks for the input, padrino. You are right about the dangers of choosing extremely simple metrics, versus putting emphasis on taste, genre, state of mind, etc. The more I think about it (and the more comments I receive) the harder this problem gets.

  1. May 16, 2012 at 10:46 am

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: