Which Picks Did NFL Mock Drafts Get Most Wrong?

With the first round of the NFL draft complete, it appears that the wisdom of the crowds wasn’t particularly wise. The first three picks went relatively as expected, but the draft went off script with the Oakland Raiders’ pick at No. 4 overall: defensive end Clelin Ferrell of Clemson — a player who mock drafters believed would go somewhere in the middle of the first round. The Raiders’ pick was the first of many that defied expectations and left amateur GMs scratching their heads.

In the case of the New York Giants, some fans were banging their heads against the wall and collapsing in tears. New York, which passed on many quarterbacks a year ago to take running back Saquon Barkley, took Duke QB Daniel Jones at No. 6. Jones averaged a 20.4 pick in mock drafts taken in the last 30 days before the draft but came off the board an eyebrow-raising 14.4 picks earlier. The Giants seemed to be trying to get ahead of a quarterback run that didn’t exist: Ohio State’s Dwayne Haskins lasted until Washington took him at No. 15 (6.2 picks later than expected), and no subsequent QBs were taken on Thursday night.

But the New York football Giants, armed with three picks in the first round alone, weren’t finished reaching. Using the 17th overall pick they acquired when they dealt Odell Beckham Jr. to the Browns, the Giants selected DT Dexter Lawrence of Clemson, 10.5 picks earlier than expected. The Giants were able to capture some surplus value with their third and final pick of the first round, however: Georgia CB Deandre Baker lasted 3.2 picks longer than expected and should help fill the void in the Giants secondary that was left when Eli Apple was traded to New Orleans last October for picks in the fourth and seventh rounds.

The NFL draft has been full of surprises

The first round of the 2019 NFL draft by each player’s pick and his average draft position (ADP) in mock drafts since March 26, 2019

team player Position pick ADP diff
Arizona Kyler Murray QB 1 1.8 -0.8
San Francisco Nick Bosa DE 2 2.1 -0.1
N.Y. Jets Quinnen Williams DT 3 3.7 -0.7
Oakland Clelin Ferrell DE 4 19.0 -15.0
Tampa Bay Devin White LB 5 7.0 -2.0
N.Y. Giants Daniel Jones QB 6 20.4 -14.4
Jacksonville Josh Allen LB 7 3.7 +3.3
Detroit TJ Hockenson TE 8 13.0 -5.0
Buffalo Ed Oliver DT 9 9.3 -0.3
Pittsburgh Devin Bush LB 10 15.5 -5.5
Cincinnati Jonah Williams OT 11 13.3 -2.3
Green Bay Rashan Gary DE 12 11.2 +0.8
Miami Christian Wilkins DT 13 19.0 -6.0
Atlanta Chris Lindstrom G 14 29.3 -15.3
Washington Dwayne Haskins QB 15 8.8 +6.2
Carolina Brian Burns LB 16 16.0 +0.0
N.Y. Giants Dexter Lawrence DT 17 27.5 -10.5
Minnesota Garrett Bradbury C 18 25.7 -7.7
Tennessee Jeffery Simmons DT 19 29.5 -10.5
Denver Noah Fant TE 20 22.9 -2.9
Green Bay Darnell Savage S 21 54.7 -33.7
Philadelphia Andre Dillard OT 22 17.6 +4.4
Houston Tytus Howard OT 23 60.7 -37.7
Oakland Josh Jacobs RB 24 27.2 -3.2
Baltimore Marquise Brown WR 25 25.4 -0.4
Washington Montez Sweat DE 26 10.6 +15.4
Oakland Johnathan Abram S 27 33.6 -6.6
L.A. Chargers Jerry Tillery DT 28 31.6 -3.6
Seattle L.J. Collier DE 29 62.9 -33.9
N.Y. Giants Deandre Baker CB 30 26.8 +3.2
Atlanta Kaleb McGary OT 31 43.3 -12.3
New England N’Keal Harry WR 32 29.3 +2.7

Sources: NFL, Ben Robinson

The selections of Lawrence and Ferrell were part of a larger trend: NFL GMs appear to have been particularly enamored with Clemson players. Three Tiger defensive standouts from the national championship team were selected in the first round, and they went 10.5 slots earlier on average than mock drafts predicted.

A dominant theme of the night, as expected, was NFL teams trying to find the next star pass rusher. But it was a pass rusher who had the biggest slide down the board among the first-round selections. Washington appears to have gotten a substantial value when it selected Mississippi State DE Montez Sweat 26th overall. In a draft class stacked with edge rushing talent, Sweat came off the board 15.4 picks later than expected.3

When we look at all 32 first-round picks, the correlation between what mock drafters expected and what actually occurred was about the same in 2019 as it was in 2018. In 2019, the average draft position in mock drafts explained 48 percent of variance, down slightly from 49 percent of variance explained in 2018. This year’s first round skewed toward reaches, with six teams trading up on draft day to get their guys. Overall, players came off the board six picks earlier than expected; last year, that difference was five spots.

As a result, Day 2 of the draft should be one in which savvy teams can find more value than they may have initially anticipated. That could even drive more pick swapping, as teams look to swoop in and grab coveted players like mock draft darling D.K. Metcalf on the cheap.


From ABC News:
Biggest picks from the 1st round of NFL Draft

When We Say 70 Percent, It Really Means 70 Percent

One of FiveThirtyEight’s goals has always been to get people to think more carefully about probability. When we’re forecasting an upcoming election or sporting event, we’ll go to great lengths to analyze and explain the sources of real-world uncertainty and the extent to which events — say, a Senate race in Texas and another one in Florida — are correlated with one another. We’ll spend a lot of time working on how to build robust models that don’t suffer from p-hacking or overfitting and which will perform roughly as well when we’re making new predictions as when we’re backtesting them. There’s a lot of science in this, as well as a lot of art. We really care about the difference between a 60 percent chance and a 70 percent chance.

That’s not always how we’re judged, though. Both our fans and our critics sometimes look at our probabilistic forecasts as binary predictions. Not only might they not care about the difference between a 60 percent chance and a 70 percent chance, they sometimes treat a 55 percent chance the same way as a 95 percent one.

There are also frustrating moments related to the sheer number of forecasts that we put out — for instance, forecasts of hundreds of U.S. House races, or dozens of presidential primaries, or the thousands of NBA games in a typical season. If you want to make us look bad, you’ll have a lot of opportunities to do so because some — many, actually — of these forecasts will inevitably be “wrong.”

Sometimes, there are more sophisticated-seeming criticisms. “Sure, your forecasts are probabilistic,” people who think they’re very clever will say. “But all that means is that you can never be wrong. Even a 1 percent chance happens sometimes, after all. So what’s the point of it all?”

I don’t want to make it sound like we’ve had a rough go of things overall.1 But we do think it’s important that our forecasts are successful on their own terms — that is, in the way that we have always said they should be judged. That’s what our latest project — “How Good Are FiveThirtyEight Forecasts?” — is all about.

That way is principally via calibration. Calibration measures whether, over the long run, events occur about as often as you say they’re going to occur. For instance, of all the events that you forecast as having an 80 percent chance of happening, they should indeed occur about 80 out of 100 times; that’s good calibration. If these events happen only 60 out of 100 times, you have problems — your forecasts aren’t well-calibrated and are overconfident. But it’s just as bad if they occur 98 out of 100 times, in which case your forecasts are underconfident.

Calibration isn’t the only thing that matters when judging a forecast. Skilled forecasting also requires discrimination — that is, distinguishing relatively more likely events from relatively less likely ones. (If at the start of the 68-team NCAA men’s basketball tournament, you assigned each team a 1 in 68 chance of winning, your forecast would be well-calibrated, but it wouldn’t be a skillful forecast.) Personally, I also think it’s important how a forecast lines up relative to reasonable alternatives, e.g., how it compares with other models or the market price or the “conventional wisdom.” If you say there’s a 29 percent chance of event X occurring when everyone else says 10 percent or 2 percent or simply never really entertains X as a possibility, your forecast should probably get credit rather than blame if the event actually happens. But let’s leave that aside for now. (I’m not bitter or anything. OK, maybe I am.)

The catch about calibration is that it takes a fairly large sample size to measure it properly. If you have just 10 events that you say have an 80 percent chance of happening, you could pretty easily have them occur five out of 10 times or 10 out of 10 times as the result of chance alone. Once you get up to dozens or hundreds or thousands of events, these anomalies become much less likely.

But the thing is, FiveThirtyEight has made thousands of forecasts. We’ve been issuing forecasts of elections and sporting events for a long time — for more than 11 years, since the first version of the site was launched in March 2008. The interactive lists almost all of the probabilistic sports and election forecasts that we’ve designed and published since then. You can see how all our U.S. House forecasts have done, for example, or our men’s and women’s March Madness predictions. There are NFL games and of course presidential elections. There are a few important notes about the scope of what’s included in the footnotes,2 and for years before FiveThirtyEight was acquired by ESPN/Disney/ABC News (in 2013) — when our record-keeping wasn’t as good — we’ve sometimes had to rely on archived versions of the site if we couldn’t otherwise verify exactly what forecast was published at what time.

What you’ll find, though, is that our calibration has generally been very, very good. For instance, out of the 5,589 events (between sports and politics combined) that we said had a 70 chance of happening (rounded to the nearest 5 percent), they in fact occurred 71 percent of the time. Or of the 55,853 events3 that we said had about a 5 percent chance of occurring, they happened 4 percent of the time.

We did discover a handful of cases where we weren’t entirely satisfied with a model’s performance. For instance, our NBA game forecasts have historically been a bit overconfident in lopsided matchups — e.g., teams that were supposed to win 85 percent of the time in fact won only 79 percent of the time. These aren’t huge discrepancies, but given a large enough sample, some of them are on the threshold of being statistically significant. In the particular case of the NBA, we substantially redesigned our model before this season, so we’ll see how the new version does.4

Our forecasts of elections have actually been a little bit underconfident, historically. For instance, candidates who we said were supposed to win 75 percent of the time have won 83 percent of the time. These differences are generally not statistically significant, given that election outcomes are highly correlated and that we issue dozens of forecasts (one every day, and sometimes using several different versions of a model) for any given race. But we do think underconfidence can be a problem if replicated over a large enough sample, so it’s something we’ll keep an eye out for.

It’s just not true, though, that there have been an especially large number of upsets in politics relative to polls or forecasts (or at least not relative to FiveThirtyEight’s forecasts). In fact, there have been fewer upsets than our forecasts expected.

There’s a lot more to explore in the interactive, including Brier skill scores for each of our forecasts, which do account for discrimination as well as calibration. We’ll continue to update the interactive as elections or sporting events are completed.

None of this ought to mean that FiveThirtyEight or our forecasts — which are a relatively small part of what we do — are immune from criticism or that our models can’t be improved. We’re studying ways to improve all the time.

But we’ve been publishing forecasts for more than a decade now, and although we’ve sometimes tried to do an after-action report following a big election or sporting event, this is the first time we’ve studied all of our forecast models in a comprehensive way. So we were relieved to discover that our forecasts really do what they’re supposed to do. When we say something has a 70 percent chance of occurring, it doesn’t mean that it will always happen, and it isn’t supposed to. But empirically, 70 percent in a FiveThirtyEight forecast really does mean about 70 percent, 30 percent really does mean about 30 percent, 5 percent really does mean about 5 percent, and so forth. Our forecasts haven’t always been right, but they’ve been right just about as often as they’re supposed to be right.

The NFL Is Drafting Quarterbacks All Wrong

No position in professional sports is more important or more misunderstood than the quarterback. NFL scouts, coaches and general managers — the world’s foremost experts on football player evaluation — have been notoriously terrible at separating good QB prospects from the bad through the years. No franchise or GM has shown the ability to beat the draft over time, and economists Cade Massey and Richard Thaler have convincingly shown that the league’s lack of consistent draft success is likely due to overconfidence rather than an efficient market. Throw in the fact that young QBs are sometimes placed in schemes that fail to take advantage of their skills,1 that red flags regarding character go unidentified or ignored2 and that prospects often lack stable coaching environments, and there is no shortage of explanations for the recurring evaluation failures.

All of this uncertainty makes the NFL draft extremely exciting: You never know for certain who will be good and who will be an absolute bust. Last year, much of the pre-draft speculation surrounded where current Buffalo Bills starting QB Josh Allen — who is tall and can hit an upright from his knees from 50 yards away — would be selected. This year, when Oklahoma’s Kyler Murray decided to forgo a career in baseball for a chance to become a top pick in the 2019 NFL draft, his measurables captured attention in a different way. Murray, listed at 5-foot-10 and 194 pounds, is 7 inches shorter and more than 40 pounds lighter than Allen, and he’s the the smallest top QB prospect in recent memory. While some scouts and NFL decision makers think Murray’s odds for NFL success are long — or have him off their draft boards entirely because of his lack of size — there is strong evidence in the form of metrics and models that he is actually a good bet to succeed.

Like the rest of the league, practitioners of analytics have a pretty poor track record at predicting QB success. It wasn’t just Browns fans who were high on Johnny Manziel — many predictive performance metrics liked him as well. If some of the world’s best football talent evaluators are convinced that Murray’s height is at least a minor red flag, how can we be confident that a 5-foot-10 college QB will be productive in the NFL? When it comes to the draft, deep humility is warranted. Still, there are solid reasons to be excited about Murray.

Completion percentage is the performance measurable that best translates from college to the NFL. The metric’s shortcomings — players can pad their completion percentage with short, safe passes, for instance — are well-known. But even in its raw form, it’s a useful predictive tool.

Completion percentage translates from college to the NFL

Share of NFL quarterback performance predicted by college performance in seven measures, 2011-18

share predicted
Completion percentage 17.9%
Average depth of target 16.7
ESPN’s Total QBR 12.1
Yards per game 9.2
Touchdown rate 8.5
Yards per attempt 7.0
Adjusted yards per attempt 4.2

For players who attempted at least 100 passes in the NFL.

“Share predicted” here refers to the amount of variance in the dependent variable explained by the independent variable in a bivariate regression.

Source: ESPN Stats & information group

Its kissing cousin in the pantheon of stats that translate from college to the pros is average depth of target: Passers who throw short (or deep) in college tend to continue that pattern in the NFL. These two metrics can be combined3 to create an expected completion percentage, which helps correct the deficiencies in raw completion percentage. If you give more credit to players who routinely complete deeper passes — and dock passers who dump off and check down more frequently — you can get a clearer picture of a player’s true accuracy and decision-making.

Another important adjustment is to account for the level of competition a player faced. ESPN’s Total Quarterback Rating does this, and we’re doing it, too. For instance, passes in the Big Ten are completed at a lower rate than in the Big 12 and the Pac-12. We should boost players from conferences where it is tougher to complete a pass and ding players whose numbers are generated in conferences where passing is easier.

When we make these adjustments, and then subtract expected completion percentage from a QB’s actual completion percentage, we get a new metric: completion percentage over expected, or CPOE. An example: In 2011 at Wisconsin, Russell Wilson had a raw completion percentage of 73 percent. We would expect an average QB in the same conference who attempted the same number of passes at the same depths that Wilson attempted to have a completion percentage of just 57 percent. So Wilson posted an incredible CPOE of +16 percentage points in his last year of college. CPOE translates slightly less to the NFL than either raw completion percentage or average depth of target,4 but it does a substantially better job of predicting on-field production. In stat nerd parlance, we’ve traded a little stability for increased relevance.

CPOE best predicts yards per attempt in the NFL

Share of an NFL quarterback’s yards per attempt predicted by college performance measures, 2011-18

share predicted
Completion percentage over expected 15.5%
Completion percentage 13.5
ESPN’s Total QBR 13.2
Yards per attempt 7.0
Average depth of target 0.0

For players who attempted at least 100 passes in the NFL.

“Share predicted” here refers to the amount of variance in the dependent variable explained by the independent variable in a bivariate regression.

Source: ESPN Stats & Information group

The test of a good metric is that it is stable over time (for example from college to the NFL) and that it correlates with something important or valuable. Completion percentage over expected is slightly more stable than other advanced metrics like QBR. CPOE is also the best predictor of NFL yards per attempt. Since yards per attempt correlates well with NFL wins, and winning is both important and valuable, we’ve found a solid metric. It should help us identify NFL prospects likely to be good — so long as they are drafted and see enough playing time to accumulate 100 or more passing attempts.5

But before we stuff the metric into a model and start ranking this year’s quarterback prospects, it’s worth asking why CPOE in college might be a good measure of QB skill. One possible explanation is that it’s measuring not just accuracy but also the signal from other qualities that are crucial to pro success. The ability to consistently find the open receiver and complete a pass to him requires a quarterback first to read a defense and then to throw on time and on target. Throwing with anticipation and football IQ are both crucial to playing in the NFL at a high level, and they are likely both a part of the success signal in the metric.

CPOE is also probably capturing the ability to execute a system efficiently. A quarterback who understands how each piece of the offense complements the others and constrains the opposing defense is a huge asset for his team. The term “system QB” has a negative connotation in player evaluation circles that is probably unwarranted. If a quarterback is operating at a high level, he is inseparable from the system he’s being asked to run. It’s also likely the case that the mental and physical abilities to run any system efficiently are traits that translate — even if only imperfectly — to the pro game.

CPOE also measures accuracy, of course — which many believe is the most important trait a QB can posses. Some coaches believe accuracy is an innate skill and not something that can be taught once a player has reached college. Others believe that mechanical flaws can be corrected if other traits like arm strength are present. The Bills clearly hold this view or they wouldn’t have drafted Allen, a player with an incredibly live arm but who had a college completion rate 9.2 percentage points below expected. But regardless of whether accuracy can be taught at the NFL level, all evaluators acknowledge its importance.

With all this in mind, I built a simple logistic regression model that attempts to identify players who will go on to establish a career mark of at least 7.1 yards per attempt in the NFL — the league average from 2009 to 2018. The model took into account CPOE and six other metrics, all calculated for the player’s college career.6 There are 49 quarterbacks who have entered the NFL since 2012 who have also attempted at least 100 passes — except for small-school QBs for whom advanced college data wasn’t available. I randomly split those players into two sets and used one set to build the model and the second set to test to see if the model is any better than random chance at identifying which prospects will go on to play productive NFL football. Though the model is relatively simple — and it would be wonderful if the sample size were larger — the results are promising. The model correctly identified many players who went on to have NFL success and many more who didn’t. The best estimate for its generalized accuracy is that it will correctly identify a QB prospect as a hit or a bust in around 74 percent of cases.7 The table below shows the results of the model, labeled Predict, and includes players’ college stats.

Results from the quarterback prospect model

A random sample of the 49 quarterbacks who were drafted since 2012* by model probability, along with college stats including completion percentage over expected (CPOE)

College stats
name CPOE YPA Avg. depth of target Total QBR Predicted prob.† Career NFL YPA
Russell Wilson +16 10.3 10.4 94 >99% 7.9
Johnny Manziel +9 9.1 8.8 89 99 6.5
Jameis Winston +8 9.4 9.6 83 98 7.6
Kellen Moore +10 8.7 7.8 86 97 7.5
Deshaun Watson +5 8.4 8.8 86 93 8.3
Sam Darnold +5 8.5 9.5 80 77 6.9
Matt Barkley +4 8.2 8.1 77 73 7.4
Jared Goff +1 7.8 9.0 74 61 7.7
Kevin Hogan +4 8.5 9.3 80 37 6.1
Marcus Mariota +4 9.3 8.2 90 33 7.2
Kirk Cousins +4 7.9 8.5 58 29 7.6
Paxton Lynch +2 7.4 7.9 59 14 6.2
Geno Smith +3 8.2 7.3 74 5 6.8
Nathan Peterman +1 7.9 8.9 71 4 4.3
Zach Mettenberger +4 8.8 10.5 71 4 6.8
Trevor Siemian 0 6.4 8.2 53 3 6.8
Matt McGloin -2 7.2 8.5 60 2 6.7
Blake Bortles +4 8.5 7.5 78 1 6.7
Lamar Jackson 0 8.3 11.0 82 0 7.1

*And have attempted at least 100 passes in the NFL.

†Probability the player will meet or exceed a career yards per passing attempt average of 7.1.

Source: ESPN stats & information group

Humility is warranted at this moment, so let’s point and laugh at the failures first. After all, all models are universally wrong, but some can be useful. This one was wrong about Johnny Football, as it practically guaranteed Manziel to be an above-average NFL quarterback. What it didn’t know about was Johnny’s love of all-night parties and other off-field shenanigans. Kellen Moore, a lefty passer who had a decorated career at Boise State, is another hiccup for the model. Moore is an interesting case of a player who just barely reached the 100 passing attempt threshold and eclipsed 7.1 yards per attempt for his NFL career but still bounced around the league and never found success or even a starting job. So the model predicted his statistical success in yards per attempt but not his actual success on the field. The problem here is that our success metric — career yards per attempt over 7.1 — doesn’t perfectly discriminate between good and bad NFL QBs. Much like human evaluators, models can sometimes be right for the wrong reasons, and Moore is a prime example.8

The model was also suspiciously bad at predicting Lamar Jackson, ranking him as an almost sure bust as a passer. Jackson’s career yards per attempt — most of those attempts coming in just seven games — is right at the 7.1 threshold, and while he is no one’s idea of Drew Brees, a success probability of zero seems an overly harsh assessment for a player that has clear talent — especially running the ball — and has already helped his team to the playoffs.

Still, the good outweighs the bad. The only other false negatives in the bunch are Kirk Cousins and Marcus Mariota, both of whom have career yards per attempt figures above 7.1. Meanwhile the low probabilities assigned to passers like Nate Peterman, Zach Mettenberger, Paxton Lynch, Geno Smith and Blake Bortles all appear reasonably prescient.

Looking forward and applying the model to the current draft class, we find a few surprises. Kyler Murray sits comfortably at the top with a 97 percent probability of being an above-average pro quarterback. Murray’s physical and statistical production comps with Russell Wilson are especially striking. Wilson and Murray had roughly the same yards per attempt in college, identical average depth of target and similar Total QBR.9 Both are also under 6 feet tall and played baseball at a high level. As far as comps go for short QBs, you really can’t do any better.

Murray isn’t just a scrambler who excels working outside of the pocket and on broken plays, either. According to the ESPN Stats & Information Group, 91 percent of Murray’s 377 pass attempts in 2018 came inside the pocket, and 81.6 percent of those throws were on target and catchable. Murray faced five or more defensive backs on 82 percent of his passing attempts and threw a catchable pass 78.8 percent of the time. Against nickel and dime packages, he was even better when blitzed, with 79.1 percent of his passes charted as catchable when the defense brought pressure. And Murray didn’t just check down to the outlet receiver when the other team sent heat. Kyler pushed the ball downfield at depths of 20 yards or greater 21 percent of the time vs. a blitzing defender.

Meanwhile the other consensus first-round talent, Ohio State’s Dwayne Haskins, is viewed as much less of a sure thing by the model. While his CPOE is identical to Murray’s and his QBR is similar, the model rates his yards per attempt and low average depth of target as red flags that drag down his probability of success. Nickel is the base defense in the NFL, so a quarterback’s performance against it is important, and Haskins faced five or more defensive backs far less often than Murray, dropping back against nickel or dime on just 63 percent of his pass attempts. And when Haskins was blitzed out of those looks, he was not as adept at delivering on-target passes, with 76.4 percent charted as catchable despite only 6.6 percent traveling 20 yards or more in the air.

Kyler Murray’s accuracy and rushing put him atop his class

College quarterbacks invited to the 2019 NFL combine by their career statistics and predicted probability of success*

College stats
Player CPOE YPA Avg. depth of target Total QBR Predicted Prob.†
Kyler Murray +9% 10.4 10.4 92 97%
Will Grier +6 9.0 10.2 78 90
Ryan Finley +4 7.6 8.5 76 78
Jordan Ta’amu +6 9.5 10.1 72 72
Dwayne Haskins +9 9.1 7.8 87 63
Brett Rypien +5 8.4 9.9 67 39
Jake Browning +3 8.3 8.8 73 38
Clayton Thorson 0 6.3 7.9 61 29
Trace McSorley +3 8.1 9.7 73 22
Daniel Jones -2 6.4 7.7 62 17
Gardner Minshew +2 7.1 6.8 70 4
Jarrett Stidham +3 8.5 8.3 69 3
Kyle Shurmur -3 7.0 9.0 59 1
Drew Lock -1 7.9 10.3 66 <1
Tyree Jackson -2 7.3 10.4 59 <1
Nick Fitzgerald -4 6.6 10.2 72 <1

*Excluding Easton Stick because of lack of data

†Probability the player will meet or exceed a career average of 7.1 yards per passing attempt

Source: ESPN Stats & Information Group

Other surprises from the consensus top-four prospects are the rankings of Duke’s Daniel Jones and Missouri’s Drew Lock — both of whom completed fewer passes than we would expect, and both of whom were assigned a low probability of NFL success. Teams should probably be very wary of both players. Since 2011, college QB prospects with completion percentages under expected — a list that includes Brock Osweiler, Trevor Siemian, Mike Glennon, Matt McGloin and Jacoby Brissett — have all failed to post career yards per attempt above the league average of 7.1. Meanwhile West Virginia’s Will Grier — a player few experts have mocked to go in the first round — looks to be the second-best QB prospect of the class. With his excellent college production and nearly prototypical size at 6-foot-3 and 217 pounds, Grier is a player whose stock could rise with a good performance on and off the field at the combine.

There are many weeks of interviews, testing and evaluation left to come for each of these prospects, and analytics are just one piece of the process. Models are certainly not a player’s destiny. Murray might end up profiling as a selfish diva who can’t play well with others. Lock could somehow morph into Patrick Mahomes. But ultimately the model and the metrics agree with Arizona Cardinals coach Kliff Kingsbury’s assessment that Murray is worthy of the top overall pick in the draft. Ship him off to a team with an early pick and a creative play-caller, and enjoy the air raid fever dream that follows.

Legendary Quarterback John Elway Can’t Figure Out Quarterbacks

John Elway sees a Hall of Fame quarterback every time he looks in the mirror. So you can imagine the frustration of the Denver Broncos’ general manager that he hasn’t been able to spot a new franchise QB since Peyton Manning left town three years ago.

When the Broncos open the 2019 season, Joe Flacco is expected to be the fifth player to get a crack at the position since the 2015 campaign that culminated in a Super Bowl victory. Flacco’s signing came as a shock to Case Keenum, the team’s starting quarterback last year. But it should hardly have been a surprise given that, in Elway’s eight years in charge of the team’s roster, he has already cycled through seven different starting QBs.1 Flacco hardly seems a long-term answer entering his age-34 season — or an answer at all given that he’s 39th out of 42 qualifying starters in yards per pass attempt over the past three seasons, according to ESPN’s Stats & Information Group.

The Broncos are 29th out of the 32 teams in Total Quarterback Rating2 since the start of the 2016 season. And the teams behind them — the Jets, Cardinals and Browns — drafted quarterbacks in 2018 with top-10 picks.

Denver is throwing nowhere fast

NFL teams ranked by Total Quarterback Rating, 2016-2018

Rank Team Passer Rating Total QB Rating
32 Cleveland Browns 75.9 39.0
31 Arizona Cardinals 76.9 41.7
30 New York Jets 75.3 42.7
29 Denver Broncos 79.7 43.0
28 Miami Dolphins 88.2 43.9
27 Jacksonville Jaguars 80.9 47.3
26 Chicago Bears 85.5 47.5
25 New York Giants 86.1 48.0
24 St. Louis/L.A. Rams 89.6 49.0
23 San Francisco 49ers 83.1 49.3
22 Cincinnati Bengals 88.4 49.7
21 Carolina Panthers 82.7 50.0
20 Baltimore Ravens 82.7 50.2
19 Oakland Raiders 91.5 51.1
18 Buffalo Bills 76.9 51.2
17 Tennessee Titans 86.8 55.1
16 Washington Redskins 90.3 56.2
15 Houston Texans 85.6 57.7
14 Tampa Bay Buccaneers 90.6 60.2
13 Indianapolis Colts 91.7 60.5
12 Philadelphia Eagles 92.4 60.9
11 Seattle Seahawks 98.6 61.4
10 Detroit Lions 93.4 61.6
9 Minnesota Vikings 98.8 62.2
8 Green Bay Packers 93.0 62.3
7 San Diego/L.A. Chargers 95.5 62.5
6 Pittsburgh Steelers 94.6 65.4
5 Dallas Cowboys 95.3 67.6
4 Kansas City Chiefs 102.8 69.3
3 New Orleans Saints 105.9 69.7
2 Atlanta Falcons 105.9 71.7
1 New England Patriots 103.2 72.5

Source: Espn Stats & Information Group

It’s difficult to make the case that Flacco is any more likely to reverse the team’s fortunes at quarterback than Keenum was. So the Broncos’ search for a quarterback probably isn’t over. Elway admitted after the season that the team needed to find a long-term solution. But they’re not the only team in the NFL with that problem.

Washington, Miami and Jacksonville are reportedly looking for new quarterbacks, probably vying with Denver to add one via the draft. And the Giants are widely expected to draft a quarterback after general manager Dave Gettleman refused to commit to Eli Manning as the team’s 2019 starter.

Usually, endless quarterback searches correlate with losing. The best example since Elway joined the Broncos’ front office in 2011 is the Browns, who have won just 24.6 percent of their contests while seeing 10 quarterbacks start at least five games. But Denver has managed to win 62.6 percent of its games in that time — the highest winning percentage of any team with at least five different quarterbacks who started at least five games.

Most of Denver’s success in this time frame came with Manning under center. He delivered consistency and success (45-12 regular-season record) to a franchise that hasn’t seen the same quarterback start five seasons in a row since Elway did it himself. So a more impressive showing may belong to the Houston Texans, who somehow posted a 51.2 winning percentage despite starting eight different quarterbacks. The Texans finally seem to have found their long-term answer: Deshaun Watson, for whom they traded up from the 25th slot in the 2017 draft. The Broncos had a higher selection to trade that year (20th) but held it to select tackle Garett Bolles.

One of Elway’s problems is that even without Manning, the Broncos have not been bad enough to be in position to draft a top quarterback. This year, they’re slated to pick 10th — behind both the Giants and Jaguars. But with a 20-28 record over the past three seasons, they’re not overcoming bad QB play, either.

If Flacco does take over as signal-caller, this would be the second year in a row that the Broncos will have looked to another team for its starting QB. That’s an unconventional path to finding a signal caller given that last year, with Keenum, Denver was one of just four teams3 to have a passing leader by attempts who had ever played with another club.

Elway has tried drafting quarterbacks, too, selecting presumed Manning replacement Brock Osweiler in the second round of the 2012 draft, Trevor Siemian in the seventh round in 2015 and 26th-overall selection Paxton Lynch — Elway’s one first-round quarterback pick — a year later. Despite the opportunity to learn tips and tricks from Elway, who mastered the position at a Hall of Fame level, none of those players is currently on Denver’s roster.

Elway had a chance last year to spend a premium pick on a quarterback but passed on Josh Allen and Josh Rosen. That was after being beaten to the punch by the Jets for the quarterback he was rumored to prefer among all others in 2018, Sam Darnold.4

When he took the job, Elway expected that a consensus franchise quarterback would have to be acquired with a top-five-overall pick. He also believed that one could be developed with the right supporting cast, including the coaching staff. But he did not sound like someone who knew what to look for.

“You look for those traits that you see in each quarterback that you believe can translate into a franchise guy,” Elway told the Denver Post in 2011. “There’s the stuff you can see on film, but there’s so much more that you can’t see on film.”

Flacco has a reputation for being big-armed like Elway, who was famed for imprinting the “Elway Cross” into the chests of his receivers with the velocity of his perfect spirals. But that element to Flacco’s game has faded in recent years: Since 2016, he ranks 37th out of 41 quarterbacks in Raw QBR on passes 20-plus yards from the line of scrimmage.

Yet Elway seems content to bet on Flacco’s reversing the team’s fortunes at least in the short term. Does Elway see something in Flacco that few others can, given how widely the transaction has been panned? Or is it possible that one of the greatest quarterbacks in NFL history just can’t judge or develop quarterbacks?

Cognitive scientist Sian Beilock, the president of Barnard College, wrote in Psychology Today that there’s little chance that former athletes can remember what made them great. In fact, those athletes probably couldn’t have communicated it even when they were playing. “When your performance flows largely outside of your conscious awareness, your memories of what you’ve done are just not that good,” Beilock wrote. “This makes it hard to teach what you know to someone else. … As you get better and better at what you do, your ability to communicate your understanding or to help others learn that skill often gets worse and worse.”

The same presumably holds true for knowing what to look for in a player at the position you played. So it just may be that the worst person to pick the new Elway for the Broncos is Elway himself.

Neil Paine contributed research.

The Super Bowl’s Best Matchup Is Gladys Knight vs. The Clock

Super Bowl LIII is not only about two of the league’s best offenses squaring off against one another — New England and Los Angeles — it’s also about America’s other favorite pastime: gambling. The total amount bet on the Super Bowl1 has risen from $40 million in 1991 to more than $158 million in 2018, and much of that growth has come from “props” or proposition bets.

For readers who aren’t degenerate gamblers, prop bets are wagers you can place on events during a game that don’t directly involve the final outcome. This year there are the standard prop bets, like if the Patriots will score a touchdown in the first quarter (they never have in a Super Bowl), or if the Rams will rush for more than 127.5 yards (they averaged 143.3 yards per game in the regular season and the playoffs). But there are also more exotic prop bets on things like whether Donald Trump will tweet more than six times during the game. (The implied probability on one offshore book is 58 percent that he will hit the over.)

Another interesting wager is on the length of Gladys Knight’s rendition of “The Star-Spangled Banner.” Several offshore books have set the total for the anthem at 1 minute 47 seconds, and the implied odds for both the under and the over were set at one book at -115 — a 53.5 percent implied probability — on both sides.2 The implied probabilities being equal indicates that the book has no real opinion on the length of Gladys’s performance — they just want to take a percentage from each side of the wager and hope bettors will place their bets evenly on both.

But is Knight performing the anthem in over/under 107 seconds really close to a 50 percent proposition? Or is there evidence that might convince us that the oddsmakers got the probabilities wrong?

To find out, I went to Youtube and watched 40 Super Bowl national anthems from 1979 to 2018. I eliminated any anthems with trumpeters (there were two) and then started timing the anthem from the moment the singer first started to sing and ended the timer after the completion of the first utterance of “brave.”3 Using this methodology, the 40-year average of all national anthem singers4 is 106.1 seconds, roughly in line with the total set by the books. So the total is correct so far as the average goes, but it also seems lazy. Surely there are other factors that might help us better predict how long Gladys might sing.

For starters, the performance time of the anthem has changed as the Super Bowl has grown to become the unparalleled cultural phenomenon we now enjoy each year. As the pomp, circumstance and viewership have increased, the time anthem performers spend on the stage has also risen.

So while anthems have gotten longer over time, the 40-year average is not fully accounting for that trend. When you do account for it5 the best forecast for the 2019 anthem is actually 119 seconds, 13 seconds over the 40-year average.

Gender of the anthem singer is also significant. Men tend to sing the anthem more quickly than women — though not many men have sung the anthem in recent years, when the anthems have been getting longer overall. Still, the all-time shortest anthem performance was by a man — the incomparable Neil Diamond — who got in and out like a boss in a cool 61 seconds. And the longest anthem ever performed at a Super Bowl was by the unforgettable Natalie Cole in 1994, which clocked in at a diva-esque 148 seconds.

Finally, Knight herself appears to be a singer who knows how to stretch a note. Using whosampled, I identified 31 covers performed by Knight and timed the cover performance of each using similar criteria to the anthem timing. Knight’s covers were 7 percent longer than the originals on average, good for a bonus 12.7 seconds of soothing soul per track. In perhaps the best comp to the national anthem — “Ave Maria,” a soaring, vocal-heavy standard covered by hundreds of artists — Gladys’ performance was 37 percent longer than the standard version.

Gladys Knight takes her time with interpretations

Difference in song length between Knight’s covers and the original songs

Song Original Artist Difference
Feel Like Makin’ Love Roberta Flack +129 sec.
The Look of Love Dusty Springfield +85
Yesterday The Beatles +70
Help Me Make It Through the Night Kris Krisofferson +66
Ave Maria Anna Moffo +65
For Once in My Life Barbara McNair +50
Midnight Train to Georgia Cissy Houston +42
The Way We Were Barbara Streisand +36
Fire and Rain James Taylor +33
I’m Gonna Make You Love Me Dee Dee Warwick +30
Groovin’ The Young Rascals +27
The Need to Be Jim Weatherly +13
Average +13
Everybody Needs Love The Temptations +8
You’re the Best Thing That Ever Happened to Me Ray Price +8
Goin’ Out of My Head Little Anthony and the Imperials +7
All I Could Do Is Cry Etta James +2
Baby I Need Your Loving The Four Tops +1
Tracks of My Tears Smokey Robinson & The Miracles 0
Yes, I’m ready Barbara Mason -1
Baby Don’t Change Your Mind The Stylistics -1
I Wish It Would Rain The Temptations -4
Cloud Nine The Temptations -9
Keep an Eye Diana Ross & The Supremes -9
You’ve Lost That Lovin’ Feelin’ The Righteous Brothers -14
I Feel a Song (in My Heart) Sandra Richardson -19
Let It Be The Beatles -21
Is There a Place? The Supremes -34
Wind Beneath My Wings Roger Whittaker -34
Heard It Through the Grapevine Marvin Gaye -39
Thank You Sly & the Family Stone -45
Every Beat of My Heart The Royals -49

Sources: YouTube, Whosampled

Taking a larger view, only two anthems in the past 15 years have been performed faster than the 40-year average of 1 minute 47 seconds. And when I looked at the age of the anthem singers, I found no significant correlation between age and performance time.6 On the other hand, we can look at one of Knight’s previous performances of “The Star-Spangled Banner” itself, which is solid piece of evidence against the over, running for 92 seconds. It was, however, performed 28 years ago. All things considered, the bookmakers appear to have this line wrong on Gladys, and her upcoming anthem performance is probably going to go over 107 seconds.

Researching a single prop was a lot of work, and it’s understandable why books might not want to put this level of effort into each and every bet they publish. But it does imply that there are profitable edges for some Super Bowl props. Using the Twitter machine, I threw up a bat signal for a gambling expert to help me confirm my priors. Rufus Peabody, a professional sports bettor and former ESPN contributor who is well-known in gambling circles for the scale and volume of his Super Bowl prop wagers, agreed to help.

“The time and effort to accurately value props is pretty high,” Peabody said. “Some books put more effort into their props than others, and for some props there’s almost no data. Books will move the lines aggressively when sharp bets are made though, which helps them adjust.”

I’ve been keeping an eye on the Gladys anthem line, and it hasn’t moved all week. I was tempted to bet the over, but when I was confronted with the prospect of having to convert real money into Bitcoin in order to place a bet on an offshore site, I decided to abort. When I looked around for somewhere to place the bet in Las Vegas — where they accept actual money — I struck out. Peabody explained that prop bets like anthem length are illegal in Las Vegas because of restrictions on the types of sources casinos can use to “grade” or determine the outcome of a bet.

Even if it won’t net me any cash, I’ll be pulling for Knight to go over regardless. I want her to belt out that last note in “home of the brave” for an egregiously long time. After all, my Twitter credibility is on the line, and that’s serious business.

There’s Really Never Been An NFL Dynasty Like The Patriots

The New England Patriots are back in yet another Super Bowl — No. 9 since 2001, for those keeping track — and this time they’re the favorite to beat the Los Angeles Rams, according to both Las Vegas and our Elo model. Tom Brady, Bill Belichick and friends have been doing this kind of thing for so long that sometimes it’s easy to take their greatness for granted. But with another championship potentially looming, we thought we’d zoom out and take stock of just how incredible New England’s success has actually been. Because, love or hate the Patriots, we’ve never seen anything like what they’ve accomplished over the better part of the past two decades.

New England has enjoyed some of the most dominant seasons of all time.

Let’s start at the single-season level. To grade a team’s Elo dominance, we like to use a blend of its final end-of-season rating, its peak rating and its season-long average rating.1 According to that metric, the Patriots own a number of the greatest teams of the Super Bowl era (since 1966) — including both the greatest team to win a Super Bowl (in 2004) and the greatest team to not win a Super Bowl (in 2007).

The best single-season teams of the Super Bowl era

NFL teams ranked by a blend of their final, peak and season-long average Elo ratings, since 1966

Super Bowl winners Didn’t win Super Bowl
Team Year Elo Blend Team Year Elo Blend
1 New England 2004 1792 1 New England 2007 1824
2 Denver 1998 1771 2 Baltimore 1968 1766
3 San Francisco 1989 1770 3 Washington 1983 1762
4 Miami 1973 1767 4 Green Bay 1997 1758
5 Chicago 1985 1767 5 Seattle 2014 1749
6 Dallas 1993 1765 6 Green Bay 2011 1748
7 Pittsburgh 1975 1760 7 Indianapolis 2005 1742
8 San Francisco 1984 1759 8 San Francisco 1990 1742
9 Washington 1991 1756 9 Indianapolis 2007 1737
10 Miami 1972 1754 10 New England 2011 1734

Source: Pro-Football-Reference.com

Despite their loss to the New York Giants in one of the most thrilling Super Bowls ever, the 2007 Pats, who went 16-0 in the regular season, remain the highest-rated team in NFL history — in addition to being one of the most talented and influential teams ever assembled.2 And unlike that 2007 squad, the 2004 Patriots finished the job and capped off a 17-2 season with a Super Bowl crown, in a campaign that contained part of an NFL-record 21-game winning streak.

This year’s Pats are not in that conversation. But the 2016 version was the 16th-best team to win a Super Bowl, according to Elo, and the 2017 version that lost to the Eagles last February ranks as the 14th-best nonwinner of the Super Bowl era.

The Pats’ dynasty is the most impressive of the Super Bowl era (according to Elo).

Sometimes it’s difficult to pin down when a dynasty begins and ends, but one way to look at it is to find the stretch of seasons that would be the most difficult for a generic contender to replicate. (We also did this for the NBA last summer when looking at the Golden State Warriors’ place in history.)

To do that for any given franchise, we take the single-season blended ratings from above and calculate their harmonic mean over every possible span of seasons. (The harmonic mean is a special kind of average that rewards high marks across every value in a set — in this case, elevating teams that were consistently great.) Then we compare that number to what a team with an initial Elo rating of 16173 would be expected to have over the same number of seasons. Since it becomes progressively harder to maintain a high mean Elo as more seasons pass, this helps balance short bursts of greatness against longer, more sustained periods of dominance.

The most impressive dynasties are the ones that exceed expectations the most. And after filtering for teams that won at least two Super Bowls in a given span (plus tossing out duplicate overlapping stretches for the same franchise), the NFL’s best stretch of seasons belongs to the Patriots since 2003 — potentially including this year, if they beat the Rams. (And if not, then the stretch from 2003 through 2017.)

The Super Bowl era’s most impressive dynasties

Among franchises with at least two Super Bowl titles, the most impressive (nonoverlapping) spans of seasons, according to Elo ratings, since 1966

Team Span Seasons Titles Mean Elo vs. Expected
New England* 2003-18 16 5? 1711 +169.4
1 New England 2003-17 15 4 1714 +169.8
2 San Francisco 1984-95 12 4 1706 +155.1
3 Dallas 1992-95 4 3 1740 +150.7
4 Pittsburgh 1974-79 6 4 1712 +139.0
5 Miami 1972-74 3 2 1739 +138.5
6 Dallas 1968-83 16 2 1667 +125.7
7 Oakland/L.A. Raiders 1967-85 19 3 1654 +115.3
8 Denver 1996-98 3 2 1704 +103.9
9 Washington 1982-92 11 3 1653 +99.1
10 Pittsburgh 2004-11 8 2 1656 +93.9
11 Green Bay 1966-68 3 2 1688 +87.7
12 Green Bay 1995-15 21 2 1619 +81.7
13 Baltimore 2000-14 15 2 1599 +54.6
14 N.Y. Giants 1985-90 6 2 1627 +54.3

*The current Patriots’ run will be No. 1 if New England wins Super Bowl LIII.

Mean Elo is the harmonic mean of a team’s seasonal blended Elo ratings (which mixes the average, final and peak Elo during the season) over the span of the seasons in question.

Expected Elo is the mean Elo we’d expect for a generic Super Bowl contender (from a starting Elo of 1617) over the span of the seasons in question. Teams are ranked by how much they exceeded this expectation.

Source: Pro-Football-Reference.com

Among stretches of anywhere near the same length, the only other dynasty in the same neighborhood as the Patriots is the San Francisco 49ers’ run during the 1980s and ’90s. Built by Bill Walsh and quarterbacked by Hall of Famers Joe Montana and Steve Young, the Niners won their five Super Bowls in a span of 14 years (including four in the 12-year span listed as their most dominant above). That’s two fewer than it took the Patriots to get five of their own from 2001 to 2016. (Those 49ers also weren’t embroiled in various cheating scandals, but that’s a matter for another story.) But the 21st century Pats have also visited almost twice as many Super Bowls as did the Niners (who, granted, won all five they made it to in this stretch). With the chance to tack on a sixth championship in 18 years, the Patriots would solidify the most impressive stretch of football the game has ever known.

New England’s main dynasty also contains several GOAT-level mini-dynasties.

As incredible as the entirety of the Brady-Belichick era has been, you can also pick out just about any subset of it that you want, and there’s a good chance that the Patriots will be the best in NFL history over that length of seasons. For example, using the same mean-Elo approach as above, the best five-season span of the Super Bowl era4 is the Patriots’ run from 2003 through 2007. But they also own a separate, nonoverlapping five-season span from 2013 through 2017, which is the third-best such “mini-dynasty” since 1966. They also own both the best and third-best seven-year mini-dynasties, the best and fourth-best eight-year mini-dynasties, the best and fourth-best nine-year mini-dynasties, and so forth. (You get the picture.)

Pick a span of years; the Pats are one (or two) of the best

Best dynasties of N seasons during the Super Bowl era (since 1966) based on Elo ratings over that span

3-Year Dynasties 6-Year Dynasties
Team Seasons Titles Mean Elo Team Seasons Titles Mean Elo
DAL 1992-94 2 1748 SF 1989-94 2 1720
MIA 1972-74 2 1739 NE 2011-16 2 1720
NE 2014-16 2 1728 NE 2003-08 2 1720
SF 1988-90 2 1727 PIT 1974-79 4 1712
PIT 1974-76 2 1725 DAL 1991-96 3 1694
9-Year Dynasties 12-Year Dynasties
Team Seasons Titles Mean Elo Team Seasons Titles Mean Elo
NE 2010-18 3? 1714 NE 2006-17 2 1712
SF 1987-95 3 1711 SF 1984-95 4 1706
PIT 1972-80 4 1685 DAL 1971-82 2 1671
NE 2001-09 3 1683 PIT 1972-83 4 1659
DAL 1971-79 2 1676 OAK 1969-80 2 1653
15-Year Dynasties 18-Year Dynasties
Team Seasons Titles Mean Elo Team Seasons Titles Mean Elo
NE 2003-17 4 1714 NE 2001-18 6? 1699
SF 1984-98 4 1696 SF 1981-98 5 1681
DAL 1969-83 2 1668 DAL 1966-83 2 1658
OAK 1966-80 2 1653 OAK/LA 1967-84 3 1655
MIA 1971-85 2 1643 MIA 1970-87 2 1625

Teams needed at least two Super Bowl wins during the span of seasons to qualify.

Source: Pro-Football-Reference.com

Most great teams get only one truly historic period of dominance before they begin to break apart — particularly in the salary-cap era, when talent became tougher to hold on to and build around. The Troy Aikman/Emmitt Smith/Michael Irvin Dallas Cowboys, for instance, rank as our third-most impressive overall dynasty, but that run ultimately lasted only a few years: Aikman, Smith and Irvin stayed in Dallas for the rest of the 1990s, but as they got older, the rest of the roster wasn’t strong enough to compensate, in part because the cap forced the Cowboys to shed talent. The Patriots, though, have numerous nonoverlapping subsections of years that would each be the pinnacle of most franchises’ entire histories, and they’ve done it all in an era when the NFL is (theoretically) trying to promote parity.

And one of the most interesting things about the Patriots’ micro-dynasties is that many were accomplished with different styles of football, despite the constant tandem of Belichick and Brady. As my colleague Mike Salfino pointed out last week, the Pats’ playoff offenses this decade have run the gamut from some of the least dependent on running backs to some of the most. It’s a testament to the chameleon-like way Belichick and staff have been able build their teams that they’ve maintained New England’s run of dominance despite constantly shifting their strategic tendencies.

2018 might be Belichick’s most impressive coaching job yet.

Sure, we’re tired of the Patriots’ current “nobody believes in us” schtick. But it is true that this incarnation of the Patriots is comparatively underpowered, at least compared with previous versions of the team in the Brady-Belichick era. By whatever measure you want to use to account for New England’s talent level — star performances or team strength — this team looks less impressive on paper than usual.

Not only is this the worst Pats Super Bowl team since 2001, according to our blended Elo dominance metric from above, but New England also had its fewest Pro Bowlers (two) and players with double-digit Approximate Value5 (five) in any of its Super Bowl seasons over that span, and its second-fewest first-team All-Pros (one, Stephon Gilmore). In fact, there were numerous Patriot teams that fell short of the Super Bowl entirely that, according to all of the categories above, had more talent than the 2018 version. Suspensions (Julian Edelman) and off-field headaches (Josh Gordon) certainly played a role in New England’s reduced star power, but it was also a roster Belichick had to cajole more wins out of than usual.

Regardless, it worked — and it helped the Patriots extend their historic dynasty. The only thing left is to see whether Brady, Belichick and company can add yet another ring to their collection versus the Rams, the opponent it all started against.

Check out our latest NFL predictions.

Why The NFL Can’t Rely On Defense

In an NFL season marked by historic offensive production and a championship round that was conspicuously absent a top-10 defense,2 aficionados of low-scoring rock fights, filled with punts and field goals, have been left disappointed. The best defensive teams to make the playoffs were eliminated early in the tournament, with the Bears, Ravens and Texans all losing in the wild-card round. A week later, Joey Bosa and the emerging Chargers defense were dismantled by the Patriots, and the Cowboys — perhaps the best defensive team left in the divisional round based on their end-of-season play — lost to the Rams. Extracting the strong defensive teams with relatively weak offenses led to historically exciting playoff football, producing two overtime games in the championship round for the first time in NFL history. Now we have a Patriots and Rams Super Bowl pitting perhaps the greatest QB of all time in Tom Brady against the hottest young offensive mind in the league in Sean McVay.

We shouldn’t be surprised that great offensive teams have made it this far. Teams are more reliably good — and bad — from game to game and year to year on offense than on defense. Individual defenders often have wild swings in performance from season to season, and defensive units forecast to be dominant often end up being merely average. The Jacksonville Jaguars’ defense took them as far as the AFC championship a year ago, but that same defense led them to five wins this season. Meanwhile, performance on offense is generally easier to forecast, making investments on that side of the ball more reliable.

Even then, football is largely unpredictable. When an otherwise sure-handed Alshon Jeffery3 lets a well-thrown Nick Foles pass sail through his fingers for an interception to end the Eagles season, or when Cody Parkey double-doinks a partially blocked field goal to end the Bears’ playoff hopes, we are essentially cheering, or bemoaning, randomness. Most vexing for forecasters and league observers trying to make sense of things is that the plays that matter the most in football are often the most unpredictable. But again, this is particularly true on the defensive side of the ball.

Turnover margin is the canonical example. Teams that win the turnover battle go on to win their games at a very high rate. Home teams win about 73 percent of their games when they are plus-1 in turnover differential, according to data from ESPN’s Stats & Information Group, and the home team win rate climbs to more than 86 percent when it’s plus-2 or better.

Yet despite their clear importance, the number of turnovers a team creates in one season has no bearing on how many turnovers the team will create in the next. Both interceptions and fumbles are completely unpredictable from season to season at the team level. And this pattern holds true for defense in general. If we measure the stability of defensive stats from one year to the next,4 we find that compared with offensive performance, most defensive stats are highly variable from year to year.

Defensive performance is unpredictable

Share of performance across various team-level metrics predicted by the previous season’s performance in the regular season, 2009-2018

metric Share predicted
Total offensive DVOA 18.9%
Offensive passing DVOA 18.8
Defensive passing DVOA 10.0
Offensive rushing DVOA 9.7
Total defensive DVOA 9.7
Defensive rushing DVOA 8.3
Sacks 3.6
Interceptions 2.4
Fumbles 1.6

Source: Football Outsiders

High-impact plays on defense turn out to be the least predictable. And while we’re by no means great at identifying which teams will succeed on offense, offensive DVOA is about twice as good at forecasting future performance as defensive DVOA.5

For teams like the Chicago Bears, who won 12 games despite fielding the 20th best offense in the NFL, this has major ramifications. The Bears were third in the league in turnover margin and third in sacks — feats we shouldn’t expect to repeat based solely on this season’s results. (Just ask the Jags.) Casting even more doubt on their ability to field an elite defense in back-to-back years, Chicago also lost its defensive coordinator, Vic Fangio, who left to become the head coach in Denver, further destabilizing the strength of the team.

Still there is some hope for lovers of the three-and-out. While rare, there are plays a defense makes that do tend to carry over from year to year. One of the most stable defensive stats is hits on the quarterback, which has a relatively impressive year-to-year r-squared of 0.21 — better even than total offensive DVOA, which is the gold standard for stability in team metrics. Quarterback hits include sacks — 43.5 percent of QB hits end in a sack, and those by themselves tend to not be predictive — but also plays in which the passer is contacted after the pass is thrown, and that contact is incredibly disruptive to a passing offense.

When a quarterback is hit, his completion percentage is affected on a throw to any part of the field.6 Teams that can generate pressure that ends with contact on the opposing QB greatly improve their chances of causing incompletions and getting off the field. And best of all, teams that are good at generating hits on the quarterback tend to stay good at it.

Philadelphia led the league in QB hits but not sacks

Total quarterback hits, sacks and expected sacks for teams’ defensive lines in the regular season, 2018

Sources: NFL, Elias Sports Bureau

The Eagles, Jets and the Seahawks all appear to have better days ahead of them on defense. Each team racked up more than 100 QB hits in 2018. But they also experienced bad fortune, converting their hits into sacks at a rate below what we’d expect. If these teams generate similar pressure next season, we shouldn’t be surprised to see their sack totals rise just based on reversion to the mean. Meanwhile, Chicago, New Orleans and Kansas City experienced good fortune in 2018, converting their QB hits at a rate higher than we’d expect. Assuming the defensive lines return largely intact, we probably shouldn’t be surprised to see their sack totals dip next season.

Stats like QB hits are rare to find on defense. And because of the high variance in defensive performance, teams built with a defense-first mindset end up controlling their own destinies less than we might expect. When it comes to team-building, this suggests that investments on offense are better long-term bets for stability. The results this year are particularly encouraging. Lighting up scoreboards by focusing on scoring points instead of preventing them has proved to be both successful and incredibly entertaining to watch. For this season at least, defense isn’t winning anyone a championship.

Check out our latest NFL predictions.

The Rams And Patriots Have Reversed Roles Since Their First Super Bowl Meeting

One of the most wonderfully ironic moments in Super Bowl history happened just before kickoff in February 2002, when St. Louis Rams wide receiver Ricky Proehl turned to NFL Films’ cameras during warmups and declared: “Tonight, a dynasty is born!”

Proehl was right, of course. A dynasty was born that night — just not the one he was imagining. Tom Brady and the New England Patriots ended up toppling the heavily favored Rams in Super Bowl XXXVI, using it as a springboard for the greatest run of sustained success any NFL team has ever known.

The Patriots were the up-and-coming team back then, while the Rams were the established champions with the veteran, future Hall of Fame quarterback. This time around, though, the roles will be reversed for the two franchises — with the Patriots serving as the elder statesmen, while the Rams are the team on the rise. It’s a fitting turnabout, one featuring what the Elias Sports Bureau determined was the largest gap in age between both starting quarterbacks (Tom Brady is 17 years and 72 days older than Jared Goff) and head coaches (Bill Belichick is 33 years and 283 days older than Sean McVay) in Super Bowl history.

The Rams opened the betting Sunday night as slight favorites with some sportsbooks (so yes, you can say you were an underdog, Tom), though that didn’t last long. A flood of bets for the Patriots pushed the line to favor New England by 2½ points, according to the current consensus in Vegas. Here’s what our Elo ratings think about the matchup, using both the classic version from our interactive and one with the experimental quarterback adjustments we’ve been tinkering with:

OK, Elo — who ya got in the Super Bowl?

Win probabilities for Week 21 games according to two methods: standard Elo and adjusting for starting quarterbacks

Standard Elo QB-Adjusted Elo
Team Rating Win Prob. Base Rtg Starting QB QB Adj. Win Prob.
LAR 1667 47% 1656 Jared Goff +4 46%
NE 1686 53 1645 Tom Brady +42 54

Elo quarterback adjustments are relative to average, based on a rolling average of defense-adjusted QB stats (including rushing).

Source: Pro-Football-Reference.com

The Patriots still somehow have two very important components from that original Super Bowl against the Rams: Brady and Belichick. At age 41, Brady had his worst passing numbers in several years, yet he also was still a top-10 QB (at worst), a Pro Bowler and — it bears emphasizing — impossibly productive for his age. All of that came despite throwing to a revolving-door cast of receivers and a less-dominant version of longtime security blanket Rob Gronkowski. All told, Brady led an offense that still ranked fourth in scoring and eighth in expected points added, albeit with a lower per-game EPA average than any Pats team with Brady as starter since 2013.

For Belichick’s part, this season saw his Patriots improve significantly on defense, jumping from 24th in EPA in 2017 to seventh in 2018. Although New England tied for the second-fewest sacks in the league, it generated the third-most pressure (according to ESPN’s Stats & Information Group), forced the second-lowest completion percentage and generally was the best Patriots pass defense in a while. And this team was also a celebration of Belichick the (de facto) general manager: In addition to shrewd veteran acquisitions such as CB Stephon Gilmore and LB Kyle Van Noy, a large share of the Pats’ production came from draft picks made over the past few years, including DLs Trey Flowers and Malcom Brown, OLs Shaq Mason and Joe Thuney, and rookie RB Sony Michel.1 All of those pickups helped fuel a Pats roster that still relied heavily on Brady to work his magic but also blocked well and played sound defense.

The Patriots’ run wasn’t always easy, of course. The 2018 edition had the second-worst points per game differential and lowest Elo rating of the franchise’s Super Bowl-bound squads since … you guessed it, the 2001 team. But maybe that’s just further proof that everything truly has come full circle in New England. They’re certainly hoping the story ends the same way this time around.

As for these current Rams, they are not too dissimilar from their Greatest Show on Turf forebears, either. Los Angeles outscored opponents by 143 total points in the regular season (third-best in football) and got high marks in every power ranking out there, including Elo (which ranks them No. 2), Football Outsiders’ Defense-adjusted Value Over Average (No. 2), ProFootballFocus’s rankings (No. 2), Jeff Sagarin’s ratings (No. 2), Andy Dolphin’s predictive rating (No. 3) and Pro-Football-Reference.com’s Simple Rating System (No. 3). Though they never actually ranked first in Elo at any point during the season, the Rams were consistently one of the game’s top contenders all year long.

And they got that way just about as quickly as those fabled 1999 Rams, who went 4-12 the year before Kurt Warner and Marshall Faulk changed the franchise’s fortunes forever. The 2018 season culminated a remarkable two-year turnaround arc under soon-to-be-33-year-old coach Sean McVay, who took L.A. from a 4-12 disaster in 2016 under former coach Jeff Fisher to an 11-5 record last year, and now a Super Bowl. Over that span, the Rams went from an Elo rating of 1346 to 1667, a gain of 321 Elo points. Only four other Super Bowl teams in history have gained more rating points from the end of two seasons prior to the start of the big game itself — the 1998 Atlanta Falcons (+368), 1981 San Francisco 49ers (+360), 1992 Dallas Cowboys (+357) and 1971 Miami Dolphins (+339). Even the ’99 Rams had “only” gained 246 points of Elo from the end of 1997, though they do own the largest single-season gain ever for a Super Bowl team.

How did L.A. do it? The cornerstones of the 2018 team — Goff, DT Aaron Donald and RB Todd Gurley2 — were all drafted by the club from 2014 to 2016. But general manager Les Snead did his best work over the 2017 and 2018 offseasons, snagging the majority of the current team’s other starters either via the draft or in a flurry of win-now moves that mostly look smart in hindsight. The other key ingredient was coaching, where (with a few weird exceptions on Sunday) McVay has shown a fantastic knack for incorporating analytical thinking into his play-calling, and he remains the master of keeping defenses off-balance by running almost all of his plays out of the same personnel package. While there are very legitimate questions as to whether Goff or Gurley could be as successful in a different system, the pair has powered a Super Bowl run under McVay’s scheme.

Each team needed luck to get here, too. The Rams likely wouldn’t be headed to Atlanta without a blown pass-interference call that kept New Orleans from running down most of the clock in regulation, instead giving L.A. the chance to force overtime and eventually win the game. The Patriots benefited from a phantom roughing-the-passer penalty and a (legitimate) offside call that negated what would have been a game-ending interception, then rattled off what felt like a million straight third-and-long conversions in overtime. But there isn’t a single Super Bowl team in history that didn’t have big moments when fortune smiled on it. You have to be lucky and good to win a championship, and these teams fit both criteria.

Now, they’ll get a chance to battle on the game’s biggest stage. Will a new dynasty be born? Or will an old one keep rolling? Will the new Greatest Show on Turf avenge the old one? Or will Belichick draw up another brilliant game plan to shut down this latest version? Either way, it should be a fitting way to end one of the most entertaining NFL seasons in a while.

FiveThirtyEight vs. the readers

As you prepare for the Super Bowl, be sure to check out FiveThirtyEight’s NFL predictions page, which uses our Elo ratings to simulate the game 100,000 times, tracking how likely each team is to win. You can also make your Super Bowl pick against the Elo algorithm in our prediction game and make one last bid to climb up our giant leaderboard.

According to data from the game, here’s how readers did against the computer last weekend:

Elo’s smartest conference championship picks

Average difference between points won by readers and by Elo in Week 20 matchups in FiveThirtyEight’s NFL prediction game

OUR PREDICTION (ELO) READERS’ PREDICTION
PICK WIN PROB. PICK WIN PROB. Result READERS’ NET PTS
NO 64% NO 62% LAR 26, NO 23 -4.6
KC 61 KC 59 NE 37, KC 31 -7.1

Home teams are in bold.

The scoring system is nonlinear, so readers’ average points don’t necessarily match the number of points that would be given to the average reader prediction.

After a divisional weekend in which all the home teams won, both home squads lost their conference championship games for just the fifth time in the Super Bowl era. Elo tends to love home teams, especially in the playoffs, so you might think that would be bad news for its picks. (Indeed, the average probability set by the reader was closer to picking the road team than Elo’s default probabilities.) However, Elo still came out ahead on net points because more individual readers made extreme picks in favor of the Saints and Chiefs, costing the field points on average. It’s an instructive example of something we discussed back in Week 9 — that, because of the nonlinear scoring system in our contest, overly confident picks can really wreak havoc on your point totals. When in doubt, set a conservative probability! (Unless, say, you are in 59th place going into the Super Bowl and need a Hail Mary to move up the rankings. Know anybody like that?)

Congratulations are in order to reader Deryl Mundell, who leapfrogged long-standing leaderboard-toppers Neil Mehta and Greg Chili Van Hollebeke to claim first place on the season, checking in with 1,202.5 points. Deryl is also our No. 1 (identified) player on the postseason, with 294.2 points since the playoffs started. Thanks to everyone who has been playing — and the game isn’t over yet! You now have one last chance to make your Super Bowl pick. Make it count!

Check out our latest NFL predictions.

You Called A Run On First Down. You’re Already Screwed.

Throughout the 2018 regular season, the Seattle Seahawks made a conscious effort to establish the threat of the running game in the minds of their opponents. In the face of record offensive production across the NFL — driven in large part by prolific passing offenses — head coach Pete Carroll doggedly maintained that sticking with running the ball gave the Seahawks the best chance to win. Though they attempted the fewest passes in the NFL, the Seahawks went 10-6 and earned a playoff berth.

But that reliance on the run may have been Seattle’s undoing in its 24-22 loss to the Dallas Cowboys in the NFC wild-card game. In the first half the Seahawks’ running backs rushed nine times for an anemic 2.1 yards per carry. Most of those runs came in a particular sequence: rush-rush-pass. All but three of Seattle’s first-half rushing attempts originated from the rush-rush-pass play sequence. And despite the lack of success using that pattern of plays against the Dallas defensive front, Seattle opened its first possession of the second half by calling it again. The result was a punt.

The notion of establishing the run is deeply ingrained in NFL culture. Coaches and play-callers laud the approach for its ability to keep a team “on schedule” and “ahead of the chains,” both of which are football shorthand for picking up enough yards on first and second down that you stand a good chance to extend a drive. True believers think that if you abandon the run too early, you make your team one-dimensional and forfeit an important edge in toughness. You’re no longer imposing your will on a defense, and this will manifest itself in worse results overall. But is this true? Does running help a team convert more first downs and extend drives? And does the order in which you call pass and run plays matter?

To answer these questions, I looked at every play called in the NFL regular season from 2009 to 20181 and compared the result of each of the possible permutations of run and pass play sequencing2 using expected points added and success rate.3 I calculated EPA and success rate for every first-down play; then I looked at every sequence that did not absorb into a first down and extended to second down and then third down, calculating the EPA and success rate for each call. Only sequences of three plays are included in the final analysis.

Leaguewide, rushing is the preferred play call on first down, after which passing takes over as the dominant play type, especially on third down.

Over the course of the 2018 season, there was no three-play sequence that Seattle favored more than rush-rush-pass. The Seahawks called rush-rush-pass 26 percent of the time, a rate 10 percentage points higher than league average. Yet despite the high frequency with which Carroll and offensive coordinator Brian Schottenheimer used the pattern, they were not successful with it. Just 41.2 percent of their rush-rush-pass sequences ended in success. Meanwhile, on three-play sequences where the Seahawks started with a pass and mixed in a run afterward, they were successful 88.9 percent of the time (pass-rush-rush), 71.4 percent of the time (pass-pass-rush) and 50 percent (pass-rush-pass) of the time.

Rush-rush-pass wasn’t effective for Seattle

The Seattle Seahawks’ three-play sequences in 2018 by frequency, expected points added and success rate

sequence epa success frequency
Pass-rush-rush +0.56 88.9% 5%
Pass-pass-rush +0.50 71.4 4
Rush-rush-rush +0.31 52.0 13
Pass-rush-pass +0.34 50.0 12
Rush-rush-pass +0.17 41.2 26
Rush-pass-rush -0.15 38.5 7
Rush-pass-pass -0.08 34.0 25
Pass-pass-pass -0.39 21.1 10

Frequencies do not add up to 100 percent because of rounding.

Sources: NFL, Elias Sports Bureau

These results hold generally across the league as well. Pass-rush-rush is the most successful three-play sequence, followed by pass-pass-rush and rush-pass-rush.

On first down, passing will net you at least 5 yards (enough to make the play a success) 47 percent of the time, while running the ball will get you the same result just 32.8 percent of the time, 14.2 percentage points less often. On second down, the gap closes to about a 7 percentage-point advantage for passing.

Play-calling patterns that end in a pass on third down have a negative expected value across the board. If we look at each sequence in terms of EPA per play, we see that the only positive EPA values on third down are on running plays. This makes sense: If you are passing on third down, it strongly implies that the first two plays in the sequence did not end well, and you likely have a third-and-long situation.

Meanwhile, the opposite outcome is true on first and second down. There are no positive EPA rushing nodes, and all passing plays return positive expected value.

This result is the exact opposite of what we would expect to find if establishing the run via play sequences like rush-rush-pass were winning strategies. Instead of making a team less predictable, establishing the run on first and second down creates a game state that is often quite predictable for the defense. The opposing team is expecting a pass on third down because the first two plays were unsuccessful.

Surprisingly, two of the top three teams in net yards per passing attempt in 2018, the Rams and the Chiefs, actually do have success with the rush-rush-pass play sequence.

How each team uses rush-rush-pass

The frequency — and effectiveness — with which every NFL team called rush-rush-pass in a three-play sequence

team epa success frequency
Seattle +0.17 41.2% 26%
Tennessee -0.23 41.3 24
Buffalo -0.26 43.9 21
L.A. Chargers -0.13 41.2 20
San Francisco -0.37 33.3 20
Houston -0.32 38.9 18
Miami -0.50 22.6 18
Denver -0.47 32.4 17
L.A. Rams +0.28 60.0 16
N.Y. Giants +0.23 51.5 16
Indianapolis -0.03 45.5 16
Minnesota -0.28 41.9 16
Jacksonville +0.05 40.0 16
Oakland -0.72 33.3 16
Cleveland +0.37 46.7 15
Chicago -0.09 41.4 15
Pittsburgh +0.70 61.5 14
Atlanta +0.37 51.7 14
Detroit +0.00 50.0 14
Tampa Bay +0.44 47.8 14
New Orleans +0.04 41.7 14
Arizona -0.71 33.3 14
N.Y. Jets +0.19 50.0 13
Dallas +0.15 46.4 13
Baltimore +0.32 44.4 12
Carolina -0.14 40.9 12
New England +0.03 39.1 12
Washington -0.32 34.8 12
Cincinnati -0.26 47.4 10
Green Bay -0.10 40.0 10
Kansas City +1.19 53.3 9
Philadelphia +0.66 50.0 9

Sources: NFL, Elias Sports Bureau

Kansas City, the most dominant passing team in the league, was successful 53.3 percent of the time with rush-rush-pass. But the Chiefs ran the sequence just 15 times all season for a total share of 9 percent of all plays — 7 percentage points below league average — and they were mostly unsuccessful with the first two plays in the chain. When the Chiefs called back-to-back runs on first and second down, the second run was successful just 47.7 percent of the time. This suggests that the success of their third-down passes owes itself more to the strength of the Chiefs passing game and quarterback Patrick Mahomes than to the running plays that led up to them.

The story is similar in Los Angeles. Sixty percent of rush-rush-pass play sequences ended in success, and the Rams used the pattern at exactly the league-average frequency. Again, however, when the Rams called back-to-back runs to begin a sequence, the second run was successful just 46.1 percent of the time, leaving them 5.8 yards left to gain for a first-down conversion on average. The success the Rams enjoyed on third-down passing attempts appears to be independent of the rushing plays that preceded them.

While the precise order in which passes and runs are called may not matter so much — several combinations are roughly equivalent to one another according to success rate — some trends are clear. Passes are more effective when called on early downs, and runs are more effective on third down. Running on first down, while often a mistake, can be salvaged with a pass on second down. And if you’re going to rush on back-to-back plays to open a series, you should do so sparingly because it will leave your team in an obvious passing situation more often than not. Your passing attack — and QB especially — will need to be well above average to consistently convert in those high-leverage spots where all deception is gone and defenders can be confident that they know what’s coming.



Check out our latest NFL predictions.

The Draw Play Is Dying In The NFL — But It Shouldn’t Be

For decades, the draw play has been one of the NFL’s most reliable tricks to fool overeager defensive lineman. The play mimics a pass — in the action of both the quarterback and the offensive line — until the last second, when the ball is handed off to the running back. When it works, the runner can often slice through holes untouched because defenders are busy trying to evade offensive linemen for a sack of the quarterback.

The play may seem like the perfect countermeasure to keep a defense honest in the modern NFL. Yet for some reason, the draw play has been all but erased from teams’ playbooks.

As the story goes, the draw play was invented in the middle of a game to slow unblockable pass rushers. It quickly became a staple of the modern offense by the sport’s “master innovator” Paul Brown, after a desperate hand-off on a busted passing play ended up working. “You fool one guy with a trap block,” Brown said. “You fool a whole pass rush with a draw play.”

Offenses today are more pass-happy than ever before. And defenses have had to respond with more aggressive stunts and blitzes by rushers quicker and more desperate to pressure passers. So what better way to cross them up than by using a draw play? But during the 2018 season, teams ran the play just a little more than once every two games, down from well over two per game just 10 years ago.

This is despite the success rate of the play used on first or second down being better than that of all rushes by running backs on those downs.1 According to the ESPN Stats & Information Group, the success rate2 on first- and second-down draws this year is 41.8 percent, compared with 38 percent on all RB runs on those downs. And draws on any down result in longer gains on average (5.29 yards per attempt) than other running back runs (4.35).

The Los Angeles Rams called only one draw play all season. (It didn’t work.) The New Orleans Saints waited all the way until Week 10 to run their first draw play of the season — a successful one. That two of the league’s most innovative offensive coaches — Sean McVay and Sean Payton — basically ignore the play seems like a bad harbinger for its survival. But the maestro of the NFL’s best offense, Kansas City’s Andy Reid, is one of the league’s greatest proponents of the play. That makes perfect sense: He’s essentially a Brown disciple, given that his West Coast offense was originally conceived by Bill Walsh when Walsh coached on Brown’s staff with the Bengals.

The Chiefs, who will play Indianapolis in the divisional round this weekend, have run a draw 16 times this year and have had success 10 times. That success rate of 62.5 is by far the best of the 10 teams that have run more than 10 draw plays. The Chargers also have used the draw well, generating 64 rushing yards in 10 attempts, six of which graded as successful.

The draw is often thought of as a play of last resort: When teams are faced with virtually hopeless distance to convert a third down, they can use the draw to stop the bleeding before punting. But only 37 third-down draws this past season were in situations when the offense needed at least 7 yards to convert. The vast majority were used on first and second down (256 out of 307 draw plays) and out of the shotgun (253 total draw plays). Of course, the latter makes sense given that the main purpose of the play is to mimic a pass.

Reid primarily uses the draw when teams have virtually no defenders dedicated to the run, meaning no more than six defenders “in the box” at or near the line of scrimmage. That was the defense deployed 13 of the 16 times Reid called a draw this season, and the call was successful eight of those times. If defenses continue with this look, a draw could be the perfect call.

With the Chiefs offense setting records and NFL coaches looking at it for design and play-calling inspiration, there’s a good chance that teams will soon discover that one of the oldest forms of NFL deception may have even more relevance in the modern game.

Check out our latest NFL predictions.