Ah, fall is in the air. School is back in session, leaves are changing colors, and football players are sustaining life-threatening concussions for our enjoyment. That must mean it’s time for Overthinking It’s annual round of analysis of the IMDb Top 250 Movies list, third edition! This year’s installment comes a bit shy of a full year since the previous edition, but with the venerable movie website’s 20th anniversary fast approaching (October 17–mark your calendars), I wanted to get this out in time to be part of the conversation.
IMDb Top 250 List, Distribution of Movies by Year
Missed last year’s edition? Here’s a quick summary: as you know, the IMDb Top 250 list is an organic list, constantly changing based on the whims of user ratings, and as such, I thought it’d be interesting to see how this list changed over time. The short answer: when comparing the list from October 2008 to that of October 2009, the median year of the dataset increased by 2 years, which suggests that the lists’ known bias towards newer movies intensified over that year. As further evidence of this bias, I also found that twelve of the nineteen additions to the 2009 list came from 2008 and 2009, and that only five of the nineteen drop-offs from the list came from the last decade.
So how did this year’s comparison turn out? Have Internet fanboys intensified their grip on the IMDb Top 250 list? Are all of the Twilight movies now in the Top 250? Find out, after the jump!
First, I present the Big Impressive Graph Which Actually Tells Us Very Little By Itself:
(Click for Larger Version)
As was the case last year, the solid line is a 4th order polynomial trend-line. Also as was the case last year, this graph by itself doesn’t tell you a whole lot, other than that the list is very friendly to movies in the past decade or so. And that I like graphs a lot. There are more graphs of movie lists in the 2008 and 2009 editions if you share in my fascination, but let’s not get distracted. On to the main findings!
1. Median Year
When I fired up Excel to do this analysis, I was pretty sure that I would find that the overall shift of the dataset in terms of median year would be greater than the concurrent shift in time. And I was right:
- Median Year of IDMb Top 250 List as of 9/30/2008: 1975
- Median Year of IMDb Top 250 List as of 10/18/2009: 1977 (jumps 2 years after 1 year)
- Median Year of IMDb Top 250 List as of 9/26/2010: 1981.5 (jumps 4.5 years after 1 year)
Yup, you read that right. Over the course of the last year, the median year of the IMDb Top 250 movies list increased by 4.5 years, which suggests that not only is the lists’ bias towards newer movies still present, it’s intensifying over time.
So what’s driving this tilt towards newer movies?
2. Additions to the List
Compared to the 2009-2008 additions, the 2010-2009 additions are less concentrated over the past two years, but they are overwhelmingly from the last decade:
| Title | Rank | Rating | Year |
| Inception | 4 | 9 | 2010 |
| Toy Story 3 | 16 | 8.7 | 2010 |
| How to Train Your Dragon | 184 | 8 | 2010 |
| Kick-Ass | 195 | 8 | 2010 |
| Avatar | 123 | 8.2 | 2009 |
| The Secret in Their Eyes | 169 | 8.1 | 2009 |
| Hauru no ugoku shiro (Howl’s Moving Castle) | 248 | 7.9 | 2004 |
| Pirates of the Caribbean: The Curse of the Black Pearl | 219 | 8 | 2003 |
| Mou gaan dou(Infernal Affairs) | 243 | 7.9 | 2002 |
| Monsters, Inc. | 239 | 7.9 | 2001 |
| Mulholland Dr. | 247 | 7.9 | 2001 |
| Toy Story 2 | 229 | 8 | 1999 |
| The Truman Show | 227 | 8 | 1998 |
| The Nightmare Before Christmas | 241 | 7.9 | 1993 |
| Edward Scissorhands | 250 | 7.9 | 1990 |
| Tonari no Totoro (My Neighbor Totoro) | 228 | 8 | 1988 |
| Rain Man | 245 | 7.9 | 1988 |
| Barry Lyndon | 234 | 8 | 1975 |
| The 400 Blows | 220 | 8 | 1959 |
| Ikiru | 222 | 8 | 1952 |
I think it’s pretty safe to say that Kick-Ass is the primary beneficiary of the IMDb fanboy effect that pushes newer genre movies onto the list when conventional wisdom says they shouldn’t belong there. Last year, I singled out Zombieland as an example of this phenomenon, and sure enough, it made this year’s list of…
3. Subtractions from the List
| Title | Rank (2009) |
Rating (2009) |
Year |
| Zombieland | 164 | 8.1 | 2009 |
| (500) Days of Summer | 197 | 8 | 2009 |
| The Curious Case of Benjamin Button | 215 | 8 | 2008 |
| Changeling | 233 | 8 | 2008 |
| Glory | 248 | 8 | 1989 |
| Harold and Maude | 242 | 8 | 1971 |
| Sweet Smell of Success | 238 | 8 | 1957 |
| Umberto D. | 194 | 8.1 | 1952 |
| The Day the Earth Stood Still | 243 | 8 | 1951 |
| Great Expectations | 241 | 8 | 1946 |
| The Lost Weekend | 219 | 8 | 1945 |
| Laura | 236 | 8 | 1944 |
| The Ox-Bow Incident | 222 | 8 | 1943 |
| His Girl Friday | 212 | 8 | 1940 |
| The Philadelphia Story | 225 | 8 | 1940 |
| Bringing Up Baby | 245 | 8 | 1938 |
| The Lady Vanishes | 202 | 8 | 1938 |
| Frankenstein | 229 | 8 | 1931 |
| Sherlock Jr. | 184 | 8.1 | 1924 |
| Safety Last! | 228 | 8 | 1923 |
Some newcomers to the 2009 list did predictably drop off as their fanboy/Oscar effects wore off, but just as it was last year, the list of drop-offs is dominated by older movies. Frankenstein, the iconic horror classic, is no longer in the Top 250 List? Well, he joins Nosferatu, who fell off the list last year.
Guess which one's on Team Edward and which one's on Team Jacob
4. Conclusion
So now that we know this tilt towards newer movies is intensifying, does that change our perception of the IMDb Top 250 list? We’ve always known it to be a more populist take on movie rankings than critic-driven exercises such as the AFI Top 100 list, Rotten Tomatoes, etc., but will its populism eventually go too far, thereby undermining the validity of the list?
It’s tough to say. At what point does this system become broken? Probably when all of the Twilight movies are on the list, but that’s an extreme example. And even if that day comes to pass, we’d have only ourselves and our society’s changing value system to blame (assuming this is not the result of a dastardly 4chan prank or systemic failure of the ranking algorithm).
I’m always surprised to see people get so worked up over the IMDb Top 250 rankings. Distilling a complex movie down to a single data point an inherently counter-intuitive exercise, yet we so desperately want there to be an authoritative way to do so. And when one seems to emerge–in this case, the IMDb list–we can never agree on its composition or its validity.
My stand-offish attitude may at first seem at odds with this whole data analysis exercise, but I argue that whatever your stance is on this or any pop culture ranking list, it only helps to have more information like this at your disposal. If for no reason, it helps prepare you for conversations with people who approach this list far more casually or with less skepticism than you do. (It may also make you come off as an insufferable pedant to those people, but I bear no responsibility for any of that.) But more importantly, it reinforces an obvious, yet often forgotten, fact: tastes change over time, and they do so unpredictably. There’s no guarantee that the gulf between current year and median year on the IMDb list will continue to narrow, just as there’s no guarantee that Patton will remain on the list. Or that New Moon won’t ever get on the list.
Readers: download the datasets for the last 3 years and go to town! See what other hidden trends lie within these numbers and post your insights into the comments. Just no fighting over whether Shawshank Redemption (number one across all three years) is better than The Godfather (number two across all three years).
Is there any way of weighting a joint metacritic/IMDB comparison to come up with critical and popular acclaim together? Would be interesting to see which movies then only featured on one list but not the other. And how many on the purely critical list were popular on OTI.
i wonder how it would change the list if instead of the current ~14% female voters and ~50% 18-29 demographics the voting population was more similar to the real world movie watching population? (or is it?)
and we should note how the voters population changes over time anyway…
Oh, I love to see this site gather this sort of ‘weight of history’. I sincerely hope to be able to congratulate OTI on its 20th birthday in 2028.
Let’s see, what will be the median year of IMDB’s Top 250 movies, assuming the bias towards new movies continues to intensify with 2,5 years a year? The answer is an awesome 2490! So the most popular films according to the IMDB in 2028 will be late 25th century fare.
Already in 2014 the median year will be in the future (2024,5) meaning the most popular movies are future ones.
Have we just *proven* that time travel will be invented by 2013? I dare you deny it. ;-)
That, or we’ll be so spoiled and dissatisfied with our pop culture that the concept of nostalgia will have to be replaced with one that describes our adoration of things we anticipate will be good but that we will quickly forget about (or not regard as highly) once they are finally available for consumption. With the growing number of sequels it’s not too unrealistic of a concept. I’m cutting myself off before I launch into a rant about how if things keep going along this way we’re all going to hell in a handbag.
I love the logic of your point, though.
What about the Bottom 100 movies on IMDB?
How well do terrible movies stand the test of time?
Do they get better? Worse?
When you ask questions about shifts in public opinion, what society hates is just as important as what it loves.
And how does this relate to Sharktopus?
We must know!
I think some so-called “bad” movies grow in stature over time (by that, I don’t mean they’re considered worse, but better, and not because they’re “bad”.
Two examples:
a)The Orson Welles “Macbeth” (1948), which was torn to shreds by critics and considered one of the worst Shakespeare films ever made for a very long time. Now it’s considered one of the best cinematic adaptations of the play, and an underrated masterpiece.
b) The film version of “Man of La Mancha” (1972), which was likewise torn to shreds by critics at the time of its release (and still is by some), partly because most of the actors in it can’t sing. But the pretentious critics slaughtered it for not being a faithful adaptation of Cervantes’s novel “Don Quixote”, although screenwriter/librettist Dale Wasserman specifically stated that he didn’t intend it to be, while Broadway critics praised the stage version of the musical for capturing the spirit of “Don Quixote”. Could there have been an unspoken rivalry between the two camps of critics?
His Girl Friday, The Philadelphia Story, and Bringing Up Baby are off the list? Outrage!
As always, it’s a lovely graph. :)
One thing I find interesting to think about is your statement that “tastes change over time”, but we don’t necessarily know that to be true. I’d love to see a similar list from, say, 1950 that showed that the top 250 movies of all time from that year included a lot of fanboy dreck from 1950 that no one remembers any more. if any of you happen to be time traveling in the near future (uh.. I guess it doesn’t matter when you’re time traveling), please found IMDB in the early 1900s so that we can do this analysis.
I wonder what effect Netflix has by its choices of which movies are added to its list of Instant View titles. I think Monsters Inc, Mullholland Drive, and Ikiru are all new additions this year. I know I hadn’t seen them before and added them this past year, past six months even.
As long as IMDB’s voting continues to be dominated male film geeks under the age of 30 I wouldn’t too worried about Twilight flicks showing up in the Top 250. Though as you implied in the article there is no guarantee that IMDB’s demographics will not shift.
“Is there any way of weighting a joint metacritic/IMDB comparison to come up with critical and popular acclaim together?”
Timothy, in fact there is such an invention:
http://www.overthinkingit.com/2010/02/02/antichrist-and-uncle-jesse/
And that will teach me that one should never forget Callot.
dabang
Isn’t there a quite logical answer to this whole move? As newcomers to IMDB will most likely be younger people (it is a logical assumption that the part of newcomers which are young will be quite bigger then the part of old newcomers) their knowledge of films will be limited. Thus will their votes be limited to newer films, and within that group of films they know the ones now up high are the best.
As time goes by they will most likely view more movies, also older ones, and perhaps realize their initial rating on a movie was too high, but once voted it says.
In other words: the ratings on these movies are nothing else then the whim of the moment and give no view on how good these movies are in the total picture.
Interesting analysis. Currently I am on a quest to see the IMDb top 250. I started out about 9 months ago and found out I had already seen 82. Yesterday I hit the 200 mark (tho a couple of the older films I have seen have fallen out – His Girl Friday & The Philadelphia Story being the most notable). When looking at the newer additions, I tend to wait a little while to see what they do. A perfect example of this was the new Indy flick. When it first came out it was in the top 250 for a couple of months, but surely enough it did fall out eventually. Now imho it is a film that is fun, but it should never have been in the top 250 – that was pure hype. Maybe the solution is to only allow films into the top 250 after a couple of years? That way the list would stay a little purer and be less susceptible? Anyhow, I am off now to continue on my quest.
I agree with Keith, with some type of moratorium on titles before they can make the top 250. When you look at the years films were made and the amount of votes, you see many newer films have many more votes. as expected. I’m actually surprised the entire Twilight series isn’t in the top 250, to be honest, but then I’m guessing the IMDB doesn’t attract too many teenage girls. I’m glad to so many Kubrick films on the list, as I think he has the most, aside from Hitchcock. I’m saddened to see so few by so many other great directors, many who were in yesterday’s trivia question! To see so many great black and white films drop off the list really makes me sad. Kane at 37th? I bet it’s #1 in the 2010 critics poll again, though!
Halbe, Keith and Rick are all correct and make very good points. I don’t think a film should be able to make the list until 1 year after it’s initial release. For a few reasons: 1) Hype. All new movies receive way to much hype and every year bad movies make the list. Well, not necessarily bad, but films that don’t deserve top 250 status. 2) Timing. If you release a feel-good movie or a Holiday movie around said holiday or remembrance it will get a whole bunch of support and everyone drooling over it being the best film. Whilst forgotten classics fall by the wayside. 3) Money. Some of the best movies ever are independent films that no one has heard of because they don’t get the support or “tv-time” that other films do.
Exactly. What is a good movie anyways? IMDB’s top250 gives us just one way giving that question it’s answer. Nothing wrong with it, as long as we remember that it is by far an absolute one.
The simplest way to explain the shift toward newer films is this: the list includes films with a rating of 8.0/10 and above. Every year, there are about five or so new movies that end up with a weighted average above 8.0. Eventually, as the audience of each film expands to reach the casual fans, those average ratings fall. New releases are almost always going to receive significantly higher ratings from the enthusiastic fans who were so interested that they rushed out to watch it in theaters right away. A film’s rating in its first year reflects more the reaction of those fans than of the broader film-watching audience that eventually will judge the movie’s quality. The sampling problem is exacerbated by the fact that the site users are also more likely to be younger and to rate mostly new releases.
IMDB’s algorithm places it in a delicate position. In an effort to keep new releases from dominating the list, they continue to increase the minimum number of votes and add weight to films with a higher vote total. Many of those “new” entries to the list are not new at all, but in fact “old” entries returning to the list. Rain Man, Edward Scissorhands, The Truman Show, Toy Story 2, Mulholland Dr., Monsters Inc., and Pirates of the Caribbean are all films with more than 100,000 votes that have benefitted from the updated rules. Even Rocky (#206) and Planet of the Apes (#231) were off the list as recently as a few years ago. These films climbed the rankings not because of an increased average rating, but simply because of the new vote quotas and weighting formula. The end result of the increased quotas is to push out the older films with lower vote totals in favor of more populist newer films.
Also, I think your analysis would be stronger if you didn’t undermine your credibility by using the word “fanboy” and by mentioning Twilight several times. Doing so makes you sound no different from every other Internet user out there.
While the disproportionate ranking of recent releases is a valid discussion for how “important” the IMDB 250 is, it does not explain the big difference in the median shift from last year to this. Yes, 6 of the additions to the list were released within the last year, but 4 of the subtractions were from the previous one year period, (and the trend will likely continue next year when Kick-Ass and Avatar drop off) so that effect is overstated in this context.
Where the number is really ratcheting up is that the oldest addition to the list is newer than 2/3 of the subtractions. 20% (4) of the subtractions are from a narrow period (1938-40) 60+ years ago, which pushes the median forward significantly. My totally uneducated guess is that the older films are losing exposure. Are channels like TCM and AMC shifting the median of what they are showing? Certainly as AMC has success with “TV” shows they have less hours to show movies. Is the current economy killing the little movie houses like the one in my college town where I first saw “Casablanca,” “Citizen Kane,” and “From Here to Eternity” in my early 20’s?
A poor economy factor that may temporarily reverse the trend: If Blockbuster and its 98% shelf space to movies of the last 18 months does not survive bankruptcy reorganization, more people will be exposed to the vast collection that is Netflix.
Finally, as we move through a technology with finite start point and a theoretically infinite end point, all things being equal, the median should move forward slightly more than a year each year. If every year of movie history starts with an equal number of movies on the list, if the total list holds a constant number, to give the new year an equal share, all of the existing years have to give up a fraction of what they hold. In the year of the first analysis, nearly 50 years of popular film were prior to the median year of 1975, less than 35 years were after. If each year gave an equal share to give 2008/9 a bar on the chart, the pre-meridian years are giving up more total, in turn moving it more.
Not trying to be smarter than whomever cooked up the algorithm which determines which movies go on that list and how far up/down they are, but, erm… the algorithm is flawed.
There are many issues with it, but, mainly, I would say that it doesn’t account for exposure. Movies made in the 20’s, 30’s, 40’s and so on, even though have been around longer than say “Inception”, have a far lesser chance of being seen by as many people as “Inception” has. DVD rentals, cable TV, internet downloads… all these tend to promote new(er) movies over old(er) ones. When they do promote the old(er) ones, they do so based on what critics (in general) and popular opinion (in particular) have decided are “classics”. I would say that the same argument applies to foreign movies: less exposure, far less chances of being “voted into” the list.
Also, chances that someone who has seen, I don’t know, “Too Hot To Handle” (a 1938 Gable and Loy flick which was quite successful at the time) going online and voting for it are far less than the chances of people (fanboys or not) going to discuss how awesome the Nth “Batman” movie has been.
This list, at best, reflects the tastes of our generation, and perhaps not even that. Rather, what’s “popular” with our generation. It is by no means a list of “good movies”, even though quite a few on it ARE indeed good.
I think the IMDB Top 250 should take age and gender into consideration. For Inception, 82% of voters were male, (74% of which were between the ages of 18-29).
And that’s not just for new action movies. Look at the vote breakdown of any movie on the IMDB Top 250 (or at all on IMDB), and you will discover where most of the votes are coming from.
Imagine what the Top 250 if each demographic was weighted equally. There would be a lot more “classic” older movies, and the fanboys would have less of a say.
Oh, and this is coming from a 23-year-old male.
Internet users/voters are younger these days and hence watch films for younger audiences. They think all films they watch are great (or have nothing in them that is informative/intelligent/questionable/open ended). They mainly see films from the past decade below these are what is more available to then, they go to the cinema with their friends, they watch DVDs others are talking about, they are not going to take the time to watch Casablanca when they could be at a house party instead.
It is arguable that newer films are better, or at least easier to have more people initially vote it well and the marketing targets those that may frequent IMDB more than those who are critics/older viewers and are less likely to hold weight in voting for those older pictures….these critics may also like the new films and also vote them highly.
With less older films on the list, they are fewer people able to put in the time to watch them…and those younger voters may not enjoy it and also vote it down, wherever others may get around to watching the newer films and realise they are not that great and hence older films pop up on the list again.
In the short term I think younger films will continue to dominate the list if adequate enough films are made…the graphical indication may vary depending on the month though.
Boulderman – iCM, Report Specialist/Excel Guru (Microsoft Excel 2010 Advisor) My website is my short film, please vote via watching the “trailer” (actual film). Thanks