DEALING WITH IFFY FISHERIES DATA

I’ll begin this essay by making this clear:  I am not a scientist, and I am not a statistician.  My undergraduate degrees are in English and history, my graduate degree is in law.  Thus, when I need any information on science or stats, I do what anyone with any sense would do:  I rely on someone who’s trained in the relevant field.

Thus, I was very interested in the presentation that New Jersey fishery scientist Jeff Brust made to the Atlantic States Marine Fisheries Commission’s Summer Flounder, Scup and Black Sea Bass Management Board last Tuesday. 

The primary issue was whether the Marine Recreational Information Program data supported the 28% reduction in recreational black seabass landings adopted by the Management Board and the Mid-Atlantic Fishery ManagementCouncil at their joint meeting last December, once some anomalous estimates were taken into account.  Mr. Brust’s explained why such 28% reduction was probably too high, doing so in a manner that was clear and understandable to persons who, like me, have no formal training in statistics or related fields.

It was a good example of how scientists ought to deal with iffy data, and demonstrated how such data ought to be addressed not only in the case of black sea bass, but other recreational fisheries.

Mr. Brust began by reporting that data for black sea bass landings in Wave 5—September and October—had been released, which indicated that landings were less than originally predicted by Council staff.  While that new data didn’t directly bear on his core presentation, it was enough to reduce the needed landings reduction from 28% to 24.4%.

He then went on to provide examples of some of the anomalous data that has cropped up in the black sea bass harvest estimates, citing what seemed to be unreasonably low Massachusetts private boat landings in 2021 and New Jersey private boat landings in 2019, along with what seemed to be unreasonably high Connecticut party boat landings in 2019 and Virginia private boat landings last year.

Error, he pointed out, can be in either direction; data outliers can be both well below or well above the expected values.  While some critics of the management system are only quick to point out what they claim are overestimates of recreational landings, Mr. Brust described an approach that would treat underestimates and overestimates in the same manner.

He called it a “modified Thompson’s tau analysis.”  I had no idea what that meanut, but the important thing isn’t what Mr. Brust’s approach was called, or even the particulars of how it works, but what it can accomplish, which is to identify data outliers, and nothing more. 

Such outliers can result from a few different causes, with the most likely being small sample size; the accuracy of the Marine Recreational Information Program’s estimates increases with the number of anglers surveyed, so when few anglers are surveyed in a state, during a particular 2-month “wave,” and/or for a particular mode of fishing, the chances of an anomalous result increase substantially.  Although the Program’s administrators do their best to collect high-quality data, the on-the-ground reality is that some outliers will always occur.

According to Mr. Brust, the Thompson's tau alalysis can be calibrated to identify data with an 80%, 90%, or 95% probability of being outliers; as mentioned earlier, both unusually high and unusually low data would be identified.

The number of outliers in the data varies from year to year, from wave to wave, and from mode to mode; there is no clear trend, although anomalously low estimates seem to outnumber the high ones.  Thus, in the case of black sea bass for the years 2018 through 2021, there were 483 separate estimates, broken down by year, state, 2-month wave, and mode.  Of those 483 estimates, 53—about 11%--were identified as outliers.  35 were identified as anomalously low, and only 18 as anomalously high. 

Once the outliers are identified, fishery managers will still have to decide how to deal with them.  They could retain such outliers in the data series, remove them, or replace them with an alternative value, provided that the method used to determine such alternative value was both objective and statistically valid.  One possible replacement approach would use the next-closest value in the data set (i.e., if an estimate was anomalously high, it could be replaced with the next-highest value in the data set, while if it was anomalously low it might be replaced with the next-lowest value), although other replacement approaches are also possible.

The percent reduction theoretically needed to keep recreational black sea bass landings at or below the harvest limit will depend on the approach ultimately selected by fishery managers.  Mr. Brust calculated results for 19 different approaches, including doing nothing other than incorporating the new Wave 5 data into Council staff’s original calculation, which returned values for the required reduction that ranged from about 18% to roughly 23.5%, not including the 24.4% required reduction that resulted from the updated Wave 5 information.  Nine of the approaches resulted in a cluster of calculated reductions just above and below 21%, but whether or not that will ultimately be the reduction adopted by the Council and Management Board, when they meet again on February 8, will depend on the way that they decide to deal with the outliers.

Although all of the approaches used in the presentation resulted in an indicated landings reduction below the previously established 28%, it’s important to note that reductions are not a foregone conclusion of using the approach presented by Mr. Brust.  The black sea bass data for 2018 contained enough anomalously low estimates that multiple approaches to adjusting the outliers all resulted in higher estimates of actual landings.

However, over the long run, reductions in annual estimates will probably outnumber such increases, if only because there is no theoretical limit on the potential magnitude of anomalously high estimates, while anomalously low estimates will always have a lower limit, for they can never fall below zero. 

That can easily be illustrated by looking at Wave 6 (November-December) private boat black sea bass landings in New York for the years 2011 through 2016, a time series selected because the 2016 landings represented one of the largest outliers faced by black sea bass managers.  

Landings estimates during those years ranged between 6,702 fish in 2014 to 1,136,275 in 2016, with 122,342 landed in 2011, 96,323 in 2012, and 32, 417 in 2015 (because of Hurricane/Superstorm Sandy’s impacts in 2012, that year isn’t included).  The million-plus fish landed in 2016 is clearly an outlier; the 6,700 fish landed in 2014 might be an outlier, too, depending on how selective managers opted to be when defining anomalous estimates.  

The 2016 landings are a full order of magnitude, and over one million fish, higher than the next-highest year’s landings; given the numbers involved, it would be impossible to see a similar variation from the norm on the low side.  But in any given set of data, as was the case for black sea bass in 2018, underestimates may very possibly occur.

The bottom line is that the approach presented by Mr. Brust provides a realistic, statistically valid way to deal with anomalous fisheries data, and better tune management measures to what is actually going on in the recreational fishery.

Thus, the only jarring comment at the meeting came after Mr. Brust was done, when someone noted that the approach, however valuable, would probably not be used once the so-called “Control Rule” approach to managementmeasures, included in the so-called “Recreational Reform Initiative,” isadopted, probably for the 2023 season.

It’s difficult to understand why a management body might choose to displace the elegant, statistically defensible approach to recreational fishery data provided by Mr. Brust, and replace it with what, at least at this time, appears to be a far cruder approach that employs a set of pre-ordainedmeasures, largely decoupled from actual recreational landings, ignores changesin angler effort, and fails to hold recreational fishermen accountable forexceeding their harvest limit.

Perhaps such “recreational reform” will have been refined by the time it goes into effect, and will prove to be a viable approach to managing recreational fisheries, which will not cause harm to fish stocks.  Yet, at this point, it seems a shame to make such a radical and potentially perilous change, when approaches such as that outlined last Tuesday are already available, provide managers with a workmanlike way to address iffy data, and still leave recreational limits in place to safeguard the health of the resource.