WHAT THE RECENT ELECTION CAN TEACH US ABOUT FISHERIES MANAGEMENT

 As I began writing this blog, Election Day 2020 has passed.  Pre-election polls predicted that former Vice President Joe Biden would win the presidential race by a comfortable margin, but now, more than 12 hours after the last polls closed, the race remains too close to call.

The outcomes in six battleground states—Arizona, Georgia, Michigan, Nevada, North Carolina, and Nevada—remain unknown, and the identity of the next president remains unknown as well.  It may take another week, and perhaps one or more court decisions, before the winner is finally known.

According to the most-touted numbers, it wasn’t supposed to happen that way, and the recriminations are already beginning.  Pollsters are in the crosshairs of many journalists.  An article in The Guardian, for example, said

“The polls on the whole proved to be grossly misleading, overestimating Biden’s strength in state after state even more than they overestimated Hillary Clinton’s strength in 2016.  Major pollsters missed the mark in many places by high single digits, and polls at the more granular district level even farther off.  While polling analysts settled on tidy conclusions after 2016 about what was broken and how to fix it, it was unclear how the enterprise of polling could be salvaged this time.  In Wisconsin many polls were 10 points off and in Florida eight points.  Elsewhere, the polls looked a bit more accurate.”

Given the way polls were presented in the news before election day, with both broadcast and newspaper journalists generally predicting former Vice President Biden coasting to an easy win, with a landslide of historic dimensions not out of the question, The Guardian’s comments don’t seem unreasonable.

But people who really understand polling would disagree.

Polling is not an exact science; while we tend to focus on the point estimate—the prediction, in this case, that Vice President Biden had an 89 percent chance of winning the election, or that he was leading by 2.5 percent in Florida—that point estimate is just that, an estimate.  It does not represent a precise quantity, but is only the most likely outcome in a range of possible outcomes, any one of which could plausibly represent the actual election results.

In other words, to truly understand what a poll says, you have to understand the concept of margin of error.

On the eve of the election, Five Thirty-Eight, a firm that specializes in analyzing data, including polling data, and issuing predictions, released a story titled

“Biden’s Favored In Our Final Presidential Forecast, But It’s A Fine Line Between A Landslide And A Nail-Biter.

The article went on to explain what all the pollsters’ critics are missing—that you can’t consider the point estimate without also considering the margin of error, saying

“what’s tricky about this race is that—because of Trump’s Electoral College advantage, which he largely carries over from 2016—it wouldn’t take that big of an error in Trump’s favor to make this election interesting. Importantly, interesting isn’t the same thing as a likely Trump win; instead, the probably result of a 2016-style polling error would be a Biden victory but one that took some time to resolve and which could imperil Democrats’ chances of taking over the Senate.  On the flip side, it wouldn’t take much of a polling error in Biden’s favor to turn 2020 into a historic landslide against Trump.”

As things stand on this post-election Wednesday, we seem to be living through the former scenario, with a polling error in Trump’s favor leading to a very closely contested election, but—if predicted outcomes in Arizona, Michigan, and Nevada turn out as expected—one that Vice President Biden will ultimately win.

Five Thirty-Eight explained that the average polling error in a presidential election was about 3 percent, and then observed that

“with a 3-point error in Trump’s direction…the race would become competitive.  Biden would probably hold on, but he’d only be the outright favorite in states (and congressional districts) containing 279 electoral votes…”

And right now, that seems to be about where things are headed.

So what does a presidential poll have to do with fisheries management?  Actually, quite a lot.  Both involve data analysis.  Both deal with uncertainty.  And both are misunderstood by the general public.

Consider stock assessments.

If we look at the 2013 update to the 2012 benchmark assessment for striped bass—the assessment that should have triggered the creation of a 10-year rebuilding plan, had the Atlantic States Marine Fisheries Commission’s Atlantic Striped Bass Management Board done what its own management plan said it “must” do—we’ll notice that the female spawning stock biomass estimate was

“58.2 thousand metric tons (95% CI:  43,262-73,212) in 2012...The [spawning stock biomass] point estimate in 2012 remained just above the threshold level of 57.6 thousand metric tons (1995 SSB value) and indicates that the striped bass are not overfished.”

But that point estimate, which said that the stock was not overfished, was like the Five Thirty-Eight prediction that Vice President Biden would end up with a 2.5 percent lead in Florida—it didn’t tell the whole story.  To get that, we need to look at the parenthetical “(95% CI:  43,262-73,212).”  “CI” means “confidence interval,” and what that parenthetical tells us that there is a 95 percent probability that the actual size of the female spawning stock biomass is somewhere between 43,262 and 73,212 metric tons, and not necessarily that close to the 58,200 metric ton point estimate.

That’s an important thing to remember, because if the actual size of the spawning stock biomass fell within a substantial portion of that 30,000 metric ton range—anywhere between 43,262 and 57,600 metric tons—the stock would have already been overfished all the way back in 2012; in fact, there was a 46 percent probability that was the case.  In fact, the chance that the striped bass was overfished at that time was probably no worse—and perhaps a bit better—than the chances of Trump winning Florida in the 2020 election.

But the average person—and that includes the non-scientists on the ASMFC’s management boards—who reads a stock assessment which says that the striped bass, or any other species, “are not overfished” isn’t likely to consider the implications of a wide confidence interval; they will merely take the point estimate at its face value and believe all is well.

Such a misunderstanding of the data can cost the public dearly, weather we’re talking about an election or the health of a fish stock.

When the press talks about political polls, they often refer to each poll’s “margin of error,” expressing it as a percentage.  Like a rigorously calculated confidence interval, the margin of error warns that the point estimate, while somewhere in the ballpark, probably doesn’t precisely reflect the public’s opinions. 

The Marine Recreational Information Program, which estimates recreational fishermen’s catch, landings, and effort, measures margins of error in its estimates, too.  They call it “percent standard error,” and it provides a good measure of the precision of MRIP estimates.  But while the margins of error in political polls are fairly small—remember that in The Guardian’s article quoted above, the author was outraged that “Major pollsters missed the mark in many places by high single digits, and polls at the more granular district level even farther off”—fisheries managers would often be thrilled if their surveys had such low levels of error.

For example, 2019 landings estimates for black sea bass in the New England/Mid-Atlantic region have a percent standard error of 7.5—starting to get into that “high single digits” range that The Guardian complained about in political polls.  Break that data down to the state level—the level where most political polls, and most recreational black sea bass regulations, begin—and the best percent standard error that you find is 14.5 percent in New York and 15.7 percent in Rhode Island; it runs as high as 40.3 percent in Virginia, although most states are in the high teens/low twenties range. 

But all those percent standard errors are far higher—generally, by an order of magnitude—than the margin of error in political polls, yet fisheries managers use them to manage fish stocks on an everyday basis.

Then we cut the data even finer.  In political polls, it’s broken down into voting districts within a single state; in MRIP estimates, into two-month “waves” and/or into sectors of the overall angling community.  When you do that, the error grows.

New York, for example, has one black sea bass bag limit for Wave 4 (July/August), and a higher bag limit for Waves 5 and 6 (September/October and November/December).  Breaking things down that way takes the percent standard error from 14.5 to a still relatively tolerable 18.8 in Wave 4, a higher 23.0 in Wave 5, and a very imprecise 35.4 in Wave 6.

Other states break down the data differently, but in all cases, when state-level data is broken down into smaller units, the percent standard error will always rise.

The final big issue that is common to both election polls is uncertainty.  In the most recent election, that took the form of things such as who recently-registered voters will choose, the impacts of COVID-19 on voters’ decisions, and how mail-in voting would affect the outcomes.

In fisheries, uncertainty is broken down into things such as scientific uncertainty, relating to things like the actual size of the fish population or how it is being impacted by climate change, and management uncertainty, such as the actual level of angler effort, and how that was effected by the pandemic.

But whether we’re talking about elections or fisheries, uncertainty can be unknown, and can never be quantified.  Pollsters might worry about whether and how COVID-19 might impact voting, but they have no way to quantify such worries and include them in their polling models.  Scientists might have suspected that they were underestimating recreational landings, and that those landings were the reason that their population models repeatedly overestimated biomass while underestimating fishing mortality, but until MRIP was updated, they had no way to know for sure.

Thus, in polling as in fisheries, we know that the point estimates are imprecise, and may be higher or lower than the value that we try to measure.  And we know that uncertainty will always play a role in real-world outcomes.

In politics, we deal with those unknowns by devising alternative strategies; in this presidential election, both candidates sought “alternative paths” to the 270 electoral votes that they needed, and campaigned in as many states as possible in an effort to attract voters who might transform such alternative paths into viable routes to the White House.

In fisheries, as in politics, typical deviations from the point estimate can have real impacts on the outcome.  But in the fisheries world, unlike the political, the adverse impacts of such deviations don't get much attention. 

In 2000, a federal appellate court decided the matter of Natural Resources Defense Council v. Daley, and established the principle that a federal fishery management measure must have at least a 50 percent probability of achieving its goal in order to pass legal muster.

Of course, that means that a measure that has a 50 percent probability of not achieving its goal is acceptable, too.

So start with management assumptions based on a point estimate of biomass, a point estimate of recruitment, and a point estimate of fishing mortality, all of which might be bounded by fairly wide confidence intervals, and use that to calculate the overfishing limit (which has just a 50 percent probability of preventing overfishing), which will be reduced to account for known sources of scientific uncertainty, to come up with the acceptable biological catch, which then may, but often is not, further reduced to account for known sources of management uncertainty to calculate an annual catch limit.

Once the annual catch limit is set, adopt recreational management measures based on estimates of the previous year’s catch, effort, and landings, estimates which will, at best, have a percent standard error approaching 10 percent.  If you’re dealing with an ASMFC managed species, go one step further, and allow the states to set their own regulations which will assure at least a 15 or 20 percent standard error if set on a statewide level, or maybe a 30 or 40 percent standard error—or more—if broken down further, into wave and/or sector.

You might be forgiven for thinking that such regulations aren't very likely to hit their mark.

Then factor in the possible impacts of unknown sources of scientific uncertainty, unknown or unaccounted-for sources of management uncertainty, and managers’ all-too-frequent reluctance to adopt probabilities of success materially higher than the 50 percent standard created in Natural Resources Defense Council v. Daley (If you’re dealing with an ASMFC- or state-managed species, when that standard doesn’t apply, there’s even a real chance that fishery managers will accept a less-than-50 percent probability of success as good enough, as the ASMFC’s striped bass management board did at the start of this year).

Then ask yourself why you’re surprised when fish stocks fail to rebuild or—need I say it again—why ASMFC-managed fish stocks so often decline.

Political journalists are complaining because recent polls missed their predictions by a few percent.

If they really want cause to complain, they ought to look at some of our fisheries—particularly our ASMFC- and state-managed fisheries—to see what a real miss looks like.

In fact, I wish that they would.  If the public had a better idea about how those management bodies worked, perhaps they’d get outraged, and try to change things.