At Overthinking It, Law & Order is analyzed by two separate yet equally important groups: the people who watch the show and send in the data, and the people who build the spreadsheets. These are their findings…
In May 2010, I announced an effort to crowdsource a list of how all 456 episodes of Law & Order ended. I predicted that it was going to be a while before we could unveil the results:
Years from now, long after you’ve forgotten all the crazy revelations in the final episode of Lost, you’ll still be catching reruns of L&O after work, marveling at Dennis Farina’s mustache. And when you do, come back here and tell me if the good guys won that day.
Well, it’s “years from now,” and we did it. And by “we,” I mean mainly Josh Kyu Saiewitz, who has been watching the show in order and emailing me the results. Thanks to him, I reported on the first 10 seasons back in February 2011. Now we’ve got all 20 seasons plugged into Excel, and it’s time to bring the evidence to the grandest jury of all: you, the overthinkers.
First off, a note about how I categorized these. There are a lot of Law & Order episodes in which an “opening” prosecution leads to a more interesting “final” prosecution. (Think about all those episodes in which there’s a hitman, but also a person who hired him or incited him to violence.) In these cases, I only considered the “final” prosecution that the episode is leading up to. For instance, in the season 10 episode “Gunshow,” a man kills 15 people in Central Park. Police catch him relatively early and he pleads out. But then McCoy decides to go after the gun manufacturer, who he accuses of purposely making the semi-automatic weapon easy to convert to fully-automatic. He wins the jury verdict, but the judge throws the conviction out. So even though they successfully nail the actual killer, this episode goes into the database with an outcome of “Not Guilty.”
Okay, ready for some graphs? Chung-CHUNG.
For starters, let’s look at the frequency of each outcome. Since multi-part episodes are meant to be one story, I’m only counting the final part, giving us a total of 450 L&O stories to calculate percentages with.
Click to enlarge!
“Implied win” refers to episodes in which you don’t see a plea bargain or Guilty verdict, but it’s pretty clear that’s the way things are headed. For instance, if the killer’s wife tearfully agrees to testify against him and then the episode ends, it’s an “implied win.” We don’t know the outcome, but we are led to believe it’s going to be some flavor of Justice. (The rare cases where the result was completely unclear went into the Other category.)
Over the entire run of the show, more than a third of all the episodes ended in Guilty verdicts, while another third ended in plea bargains. 80% of episodes ended in solid wins: either Guilty verdicts, plea bargains, or implied victories. That’s not too shabby, considering that the actual NYPD has a homicide clearance rate of about 50%. (Although you have to figure Law & Order isn’t meant to represent every case these detectives investigated; in 20 seasons, I don’t think there was a single murder that didn’t result in an arrest.)
(UPDATE 12/10/12: One of the commenters on Reddit has pointed out that the “clearance rate” has nothing to do with convictions, only arrests. In that case, Law & Order‘s clearance rate would be nearly 100%, since even in the rare episodes without a trial somebody usually gets arrested. I guess I’d know this stuff if I had watched The Wire.)
Another thing that’s not realistic is that there are more Guilty verdicts than plea bargains. In real life, about 95% of all felony convictions are pleas. And going back to the data for just seasons 1-10, we see that the plea bargain used to be the most common outcome (by a smidge).
That means that pleas must have fallen off dramatically in seasons 11-20. Let’s look at the data in a different way.
We can see that after the first season, in which the writers barely knew the plea bargain existed, it quickly became the most popular outcome… until season 14, when it was eclipsed by the Guilty verdict and stayed down until season 18.
Speaking of season 14, check out how the rate of Not Guilty verdicts plunges and hits 0% in the show’s final season.
But in the interest of fairness, I should point out that since there are only a handful of Not Guilty verdicts per season, there’s not a lot of data going into this trend. Here’s another way to display the same chart.
Nevertheless, I have a theory to propose. Look at the Not Guilty rate plotted against the show’s Nielsen ratings.
Viewership peaks in season 12. Then it starts to drop season after season. By season 14, Dick Wolf feels like he has to respond. So what does he do? He cuts down on the Debby Downer Not Guilty episodes while simultaneously reducing the number of plea bargains in favor of clear-cut Guilty verdicts. In the final season, with the show hemorrhaging viewers and facing cancellation, he didn’t dare air a single episode where the bad guy gets away scot free.
It’s an interesting theory, but like I said, just looking at Not Guilty episodes doesn’t give us many data points. Instead, let’s turn to a different metric: the Success Rate. Success Rate is ALMOST the same as Guilty + Plea + Implied Win, but not quite. Sometimes in Law & Order there are wins that feel like losses and losses that feel like wins. Success Rate is basically a measure of whether the D.A. is satisfied with the outcome (irregardless of how the outcomes appears to the public). For instance, look at the finale of season 18, “Excalibur.” In order to convict the murderer, Jack McCoy needs the governor to testify, which would reveal that he has a thing for prostitutes. The governor engineers a plea bargain, thus avoiding the sex scandal. In this case, the outcome may be Plea Bargain, but it’s definitely not a success; McCoy’s case was completely derailed.
Or take the season 13 episode “Panic.” Towards the end, it starts to seem that the defendant’s daughter may be the real killer. The defendant then offers to take a plea in order to shield her from investigation. McCoy doesn’t want to do it, but in the absence of further evidence he has no choice. So once again it’s a plea bargain but not a success.
Success Rate is obviously subjective, but I think it’s a better measuring stick than simply adding up Guilty + Plea + Implied Win. It’s an attempt to look past the verdict and determine how the ending felt. (You might say it’s a CSI solution to a Law & Order problem.) Here’s the Success Rate plotted against the show’s ratings.
Wow, look at season 17! That was a year in which the show’s ratings hit an all-time low. It was also a year in which every episode but one was a clear victory for the D.A.’s office. You could argue that the high Success Rate might have contributed to the low ratings, but note that the success rate dropped between seasons 13 and 14, and so did the ratings.
What it looks like to me is that starting in season 14 and peaking in season 17, Dick Wolf was trying to play it increasingly safe: no unsatisfying outcomes that could turn even more of his shrinking audience away. In fact, when Law & Order was renewed after season 17, the New York Times described it as “a reprieve for the show, which had seemed increasingly likely to be canceled over the last few weeks.” I’d say the high Success Rate was defensive ball.
So what happened in season 18, when the Success Rate dropped and ratings actually went up? Jeremy Sisto, Anthony Anderson, and Linus Roache happened. The show got some fresh blood, which won back some of the old viewers (or at least stopped the bleeding). That bought the writers enough breathing room to do a few more episodes with dark endings.
What I’d say the numbers show (or at least hint at) is that when the ratings are dropping, a showrunner feels like he has to do something. With a show like Law & Order, that means giving the people feel-good wins. But of course, that wasn’t what made the show a success in the first place, and it certainly wasn’t what was going to save it.
In other words, the 95% Success Rate was the Law & Order equivalent of Fonzie jumping the shark.
Want to take a crack at your own analysis? Download my spreadsheet and have at it! For now, I rest my case.
(UPDATE: Make sure to check out Part 2 of my analysis, where I see if the Law & Order success rate correlates with any data sets from the real world.)
L&O was a very Occupy Wall Street friendly show before there was an Occupy movement. I would be very interested in seeing a breakdown of the people convicted in this completely fantastical universe. How many CEOs, Trust Fund Kiddies, Bankers, Yuppies and evil Republicans did they send to prison in L&O land compared to the actual demographics you see in New York prisons? How did the relative wealth of those convicted affect the ratings?
Thats asinine. There is no story in arresting a random person that murders another for some random reason or whatever. Said murderer gets a public defender and is convicted or plea deals. There is no story there.
Even the stories that involve drug dealers and murder have a more intricate story.
The reason for using upper class people is because their money is used for a plot device to advance the story and create suspense. No suspense or show if the perp obviously committed the crime and is convicted easily.
I, too, would love to see a breakdown of who the perps were on the shows; specifically on L&O:SVU.
Invariably, it was:
A rich person
the child of a rich person
a Christian
a “Right-Winger”
a member of the military
a Republican
I found the early seasons fairly mixed and it didn’t strike me as politically biased. Or possibly biased toward conservative/Right. Remember the black guys, various, who used some racial or mob-psychology defense the show clearly deemed bogus? Or the Sixties Radical who is caught at some point? Or Angie Harmon’s character?
After Moriarty left, played a liberal but I believe the man is likely super Right-wing, it maybe switched some.
How is it that in so many thread on many different blogs, regardless of subject matter, the first post drags politics into the discussion, even when it’s a non-political topic like this one? Do the people who see everything through a political prism, especially entertainment, incessantly refresh Web pages, looking for the opportunity to be first and to derail the discussion? Don’t they have anything better to do, or anything more interesting to add than tired political talking points? Not everything is a battle in an unending culture war.
Couldn’t have put it better myself. These neurotic people need to stop their insane crusade, or at least keep their rants in the appropriate place.
I’d need to see this cross referenced with LAOSVU endings to draw solid conclusions.
So, can you confirm David Simon’s claim that there are more murders in the L&O franchise in a year than in real New York?
I don’t see how this can be true. There are more than 400 murders a year in New York.
http://cityroom.blogs.nytimes.com/2011/01/03/murders-rise-in-new-york-city-but-remain-near-historic-lows/
There was one crazy year (2005) when there were FOUR Law & Order shows airing simultaneously. But even then, that’s a max of 90 episodes a year. Even figuring that some episodes feature multiple homicides, I think we’re well short of 400.
So, I did a (very) rough calculation of this and while it’s definitely impossible for Simons’s statement to be true for the first half of L&O’s run, it might be plausible for the last few years of the overall franchise. Part of the reason for this is that the franchise really took off when crime rates in NY were at their historic lows.
From 2001 (when Criminal intent started) until 2010 (when the original stopped there were approximately 58 episodes L&O on the air each year. [I’m not counting trial by jury because it only lasted 1 season] According to the article you linked to, in those years there were never more than 600 murders. This means that in all those years each L&O episode from across the franchise needed to have an average of about 4 murders per episode to top the real murder rate. My memory of L&O is hazy and I didn’t see a body count in your data, but I don’t think there’s many episodes with zero murders (maybe more in SVU?) and I think there’s a few with at least 2. Personally, I think an average of 4 murders/episode is pretty plausible for the franchise.
Four as an average sounds pretty high to me.
I’d say the average per episode has got to be closer to 1 than to 4.
Fair enough. Like I said, I was going from my memory of the shows, which is, frankly, pretty hazy. The article cited at least one episode w/ 15 victims and I remember at least a couple of other multiple victim episodes, but I may have overestimated their frequency.
The year was 2005. There were 539 murders in NYC. Trial By Jury was only on in the 2005-2006 season, so we’ll use that for episodes. TbJ had 13 episodes. 22 each for SVU, original, and CI, so 79 episodes.
Something like 14 of the SVU episodes didn’t have anyone die in them (after a quick episode scan), and at least one more didn’t have anyone murdered. 539/65 is 8.3.
Not even close to plausible
Awesome, Matt! I’m so glad that the data ended up yielding some fascinating results.
Personally, I feel like Law and Order’s key innovation out of all of them was its ability to time and again go for a loss, or a Pyrrhic loss, or a win that concluded the story but didn’t negate the loss that had happened. The characters were fallible human beings doing the best they could in a flawed system to bring a little bit of law and a little bit of order to a troubled world. I’m not surprised that the showrunners may have tried to up their success rate (like any other prosecutor) to protect the show, but I feel like that was a big compromise of the show’s key values–and perhaps part of what makes those last few seasons before the last few seasons some of the least interesting of the show’s entire run.
Anyway, I’m super proud to have been a part of the project. Call me when you need somebody to watch all 789 episodes of Doctor Who, or something crazy like that.
Thank YOU Kyu! I agree with you that the episodes where the characters fought the good fight and lost are the ones that stick with me. And even though I don’t want EVERY episode to end that way, the fact that it SOMETIMES happened made the victories that much better.
You know what I love about the show? How thankless and exhausting it made the job seem. This is probably the least glamorous show about cops and lawyers that is not The Wire. I’m reminded of the last lines of a massive three-parter in season 7:
Adam Schiff: Started with a murder, ends with an execution. You got what you wanted. Take the rest of the week off.
Jack McCoy: It’s Friday, Adam.
Adam Schiff: So it is. See you on Monday.
The original “Dragnet” was pretty mundane in its depiction of the police.
An immortal quote, and amazing work on this entire project.
Given that I can only work from home (writing, data analysis…building excel spreadseets!) with L&O, SVU or CI in the background, this article is the penultimate of my two nerd worlds colliding.
Matthew, we were born to watch all L&O together.
When does the SVU and CI version come out? Trial by Jury doesn’t really count in my book, even with Lennie Briscoe, and the LA one is dead to me since it was allowed to survive but not the OG L&O….
The author has it all wrong. Yeah, there might be a correlation between “success rate” and ratings, but it’s a spurious one.
The far more significant factor explaining L&O’s rise and fall? (In fact, the only one?)
Two words: Jerry Orbach.
Absolutely true…Orbach’s departure (death) is the reason L&O TBJ only had one season, as well.
Great article! Now do the same thing with SVU!
*irregardless is not a word
“Irregardless” has an entry in the dictionary which pretty much just tells people that it’s not a word and they should never use it:
http://www.merriam-webster.com/dictionary/irregardless
A friend of mine likes to use disirregardless.
Wait…the dictionary is now including non-words? Isn’t that like a birding manual including flying squirrels and large moths?
Holy crap — this piece got its own item on the A.V. Club:
http://www.avclub.com/articles/here-is-every-law-order-verdict-graphed-and-analyz,88703/
Is this the first OTI article to be mentioned at that august locale?
Made it to TV Guide, too. http://www.tvguide.com/News/Every-Law-Order-Verdict-Analysis-1056310.aspx
Did the rate of clear violations of the suspect’s constitutional rights go up after 24 was launched?
I can’t tell you how many times I get requests for information like this (and for L&O SVU, too). Thank you for all the work you all put into this.
LOVE this! Thanks for all the efforts to put this together! Always wanted to do something like this. Anyway cheers and long live this amazing franchise!
Greatly interested in the race of the perpetrators and victims. My adhoc assessment has been that it has a larger percent of Whites as criminals. Perhaps something to do with nature of crime that is shown. Or something else.
Fantastic post. I was fairly obsessed with L&O for the first 15 years or so, including a span during law school where I watched 2-3 episodes every night. My “sense” as a viewer is borne out by your stats.
Also, going to assume the use of “irregardless” is a clever jab at Lawyers Using Big Dumb Words and not a part of your daily vocabulary.
[looks around sheepishly] Yup, that’s what I was going for!
First of all… bravo sir!!!! Secondly, I wanted to inquire if you have any data regarding the cast members for each episode/season readily collected and formatted? This data and only this data, will prove once and for all that Jack McCoy is the best human in existence.
I didn’t do that… but Slate accepted my challenge, downloaded the spreadsheet, and produced some goddamn gorgeous charts that show the success rate for each character:
http://www.slate.com/blogs/browbeat/2012/11/14/law_order_chart_guilty_verdicts_and_pleas_reached_by_mccoy_stone_kincaid.html
I want to know what they use to make such pretty charts!
They probably created those charts in Adobe Illustrator.
It’s nice to see that other people have too much time on their hands :)
Very interesting, thanks so much from a big L&O fan.
I really don’t think, from a purely subjective viewpoint, that there is any correspondence between verdicts and Nielsen ratings. I gained and lost interest almost entirely because of the cast. I didn’t watch too much of the last season or two, but the writing and directing seemed to hold up well. I just disliked Linus Roache and didn’t love other cast.
God bless you and the work you’ve done. L & O is easily my favorite show, and I’m glad to see that someone else obssessed about it the way I have.
All right, I confess that I am an L$O addict. Never miss a rerun. TNT should shelve The Mentalist and put L&O reruns back on.
You are the most important person in the history of forever. Thank you for your service.
I haven’t read all of the comments… but the one common thread… that I haven’t googled for is… how many defendants a) defended themselves b) testified at their own trial?
First off this happens almost never yet… I would say a fair share of defendants testified for themselves….
any one have that statistic?
Did you count the characters who were mistakenly charged and then the charges were dropped before the end of the episode? Early on that was kind of common.
I love it! Can we also see data on the defendant (age, gender, economic background) and the conviction (first degree murder, manslaughter, etc.)? If you want help collecting this data, let me know! I have a background in criminal defense and am always looking for an excuse to watch the show.
I wish I’d know about this, I would have gladly volunteered time/data to the project. Hell, as soon as NetFlix posted the first 8 seasons; for weeks, I sat and watched them one by one…just b’cuz!
Great work!
Be interesting to see an overview of LGBT storylines.
How many LGBT defendants and how many victims? How diverse?
I like the show, it’s been ok, but it hasn’t been very groundbreaking. Relies a lot on cliches, like the secreted closeted homosexual twist, that is pretty stale trick. It shows up a lot.
this is really amazing work!
Interesting and a fun read in a geeky kind of way, but maybe the show just lived out its lifespan and died a natural death.
I see a big downward spike in successful outcomes between season 2 and 4 going as a bit below 60% and then a big spike upward to over 80%, with the number staying in the 75%+ range for most of the rest of the run.
Is it possible that entering syndication changed the show’s attitude toward success? When a series is in first run, most fans catch most episodes in sequential order. Successes may not be as important as longer range story and character arcs. But in syndication, people are catching episodes in random order and each one must work as a stand-alone unit. Perhaps the feeling was that more shows had to end successfully for the good of the syndicated value of the show.
The way you only counted one case or trial per episode makes a comparison to real NYPD cases scientifically invalid. Your experimental group (Law & Order) is not measured in the same way the control group (real cases) was measured. The way you tallied the data probably undercounts plea deals and dismissed/dropped charges cases. I think you need to reanalyze the data, counting all cases. Reducing the data to one case per show was a mistake.
+1
Is there a correlation between the defendant’s attorney being black and the outcome of the trial? I remember some episodes where the defendant’s attorney is a black woman who tries very hard to justify the crime committed but finally the defendant is found guilty.
For the love of God, I thought I was the only one who needed to know information like this. There was a time (around 1993, I believe) when the same L&O episode would play 4x a day between A&E and NBC, I think. Oh, I lost so much sleep. I can *still* identify the plot within the first 10 seconds of any episode. Holycrapthisisthemostawesomesetofgraphs!
Holy crap, now this article has been referenced by Andrew Sullivan! Good on ya, guys!
“The defendant then offers to take a plea in order to shield her from investigation. McCoy doesn’t want to do it, but in the absence of further evidence he has no choice. So once again it’s a plea bargain but not a success.”
It’s not a plea bargain when the defendant pleads to the top count.
The judge gave McCoy a choice. He could have dropped the charges. Jeopardy would attach, and the father couldn’t be tried again if McCoy changed his mind. McCoy felt that to be the less palatable option.