CA fighting poor judging with The Pod Index

When mixed martial arts was first regulated, it was of course regulated by figures from the world of boxing. Now a new generation of regulators has come come up, from the mixed martial arts world. Andy Foster is leading the charge.

Foster trained in a wide variety of combat disciplines, and competed extensively as well, in boxing, in MMA, in grappling, and in kickboxing. His only two losses in MMA were to Brian Ebersole and Amar Suloev. And he has broad coaching experience. And he has promoted both boxing and MMA events. And he worked as a judge and referee extensively, and passes on his world-class expertise in official’s training courses.

He did exemplary work for four years as Executive Director of the Georgia Athletic and Entertainment Commission and since 2012 has served as the Executive Director of the California State Athletic Commission. 

Ask any MMA fan to list the biggest problems in the sport, and judging is on nearly everyone’s short list. In an extended interview with Ben Fowlkes, Foster details how he is fighting back.

At the last Association of Boxing Commissions convention, former amateur boxer and boxing judge Matt Podgorski made a presentation that caught Fosters rapt attention. He has been using a new tool for judge improvement since July.

I wanted something where I could gauge where my judges were at, Foster told MMAjunkie. You know, am I putting the right people in there? Just because you’ve been doing it a long time, that doesn’t necessarily mean that you’re right.

Podgorski, who now works full-time as a statistical analyst for a Chicago-area food company, suggested a system to collect all the scoring data from all the fights, and then compare the judges against one another while also examining the numbers to expose certain scoring patterns and biases. He called it The Pod Index, and all he needed was a commission willing to put it into place.

To Foster, it sounded like exactly the kind of thing he’d been looking for.

I wanted something to back me up other than just my thoughts, Foster said. I wanted some math or statistical thought process to back up what I’m doing with these assignments.

What The Pod Index promised was a system that logs every judge’s score for every fight in the state of California. With a large enough sample size, Podgorski said, it would provide a statistical picture of which judges were consistently at odds with their peers, especially in split decisions.

That doesn’t necessarily mean the judge is wrong, Podgorski said. It just means we need to investigate further. If we see a judge is the odd man out 20 percent of the time, we can go back and look at five or six fights in which they were in the minority and see what a greater number of different judges say.

To complete that process, The Pod Index engages the services of five more anonymous judges who watch the same fight with the commentary turned off, then turn in their own scorecards for comparison. Getting a larger pool of judges lends clarity to the overall picture, since, as Foster put it, a judge who ends up in the minority of a split decision could just be with two dummies.

But even if you’re with two dopes, the data will start to tell us who are the most consistent judges, Foster said. You could be right several times with the other two judges being wrong, but it’s probably not going to happen that way over and over again.

The program is not designed to call judges out or embarrass them, Podgorski said. It’s not meant to say, ‘You’re the best judge, and you’re the worst judge.’ It’s not a ranking system. It’s a diagnostic tool for the judges to get better, and that’s where the recommendations come in.

The recommendations, in some cases, are surprisingly specific and thorough. In a recent scoring review of a high-profile boxing match in California, Podgorski’s report identified one judge as having a strong work-rate preference in his scoring, which seemed to be altering his perception of close rounds. It recommended a specific bout for him to watch and score, in order to help the commission determine whether he’s too rigidly set in his ways, and whether that strong preference can either be accepted or used as a catalyst to provide him some additional coaching.

That additional coaching, according to Foster, is the whole point.

There are some judges who like doing this, and it’s a fun thing for them, and there are other judges who really take this stuff seriously, Foster said. They sit at home and watch fights and take notes and do trainings. They focus on it. And these fighters who spend all this time in the gym, spend six or eight weeks in a training camp, they diet and live by this strict discipline just to spend 15 or 25 minutes in a cage. I want the same level of dedication from my refs and judges.

Read entire article…

The Nevada State Athletic Commission has been working with Podgorski in a more limited capacity since November.