Direct marketers (a group of which I am a proud member) are very familiar with the concept of multivariate testing. For those who may not be completely familiar, multivariate testing is a process by which more than one component or offer associated with a promotion may be tested. I did a lot of this back when I was more deeply involved with direct mail. Today internet based multivariate testing is used constantly and in real time.
The key thing with a multivariate test is being able to read attributable results with a high level of confidence. For example, if you offer an audience 25% off AND buy 1 get 1 free, and sales increase that’s great right? Except how do you know if one or the other might have had the same result? Had you offered one segment of the audience one offer, a second audience segment the other offer, and a third BOTH offers, you’d be on your way to having a better understanding of what was really working as well as what was not.
Since my son and I host a baseball podcast at Almost Cooperstown we were both very interested in the proposed new rules Major League Baseball (MLB) was considering for the 2023 season. Three major changes have been implemented 1) a pitch clock, 2) banning the infield shift, and 3) allowing the pitcher only 2 throws to prevent a runner from stealing. The size of the bases will also be increased by three inches to make collisions at first base less likely as well as promote a higher amount of stolen base attempts.
We like the pitch clock, we’re not happy about the restriction on the infield shift, and the limit on throws to prevent a steal feels overly constricting. Overall, however we believe that due to these changes, fans can look forward to a faster-paced game with more stolen bases and a few more hard-hit balls finding holes through the infield.
My question is – why make all the changes at the same time and consequently how will MLB know which one had the most impact? Baseball has stumbled upon a classic multivariate test that it will conduct on the entire game at the same time.
The pitch clock will require the pitcher to throw a pitch in either 15 or 20 seconds after receiving the baseball back from the catcher or fielder, dependent on if there are baserunners. By instituting a pitch clock (which has been tested at the Minor League (MiLB) level), game times were reduced by more than 25 minutes. Most Major League Baseball pitchers currently take longer than 15 or 20 seconds to deliver their pitches so it will be an adjustment for them as well as the hitters who now will need to be ready to hit more quickly.
There will be ramifications from instituting a pitch clock. But because of the new rules on infield shifting, we won’t really be able to read some of the results of instituting a pitch clock. It’s clear baseball fans want more action and more scoring. The pitch clock immediately creates more action per minute since more pitches will be thrown in a shorter period. Great right? There are some current players who are not in favor, but in general the reaction from MLB players and managers has been positive. Today’s MLB pitchers throw harder than pitchers ever have. To be successful they need to go all out on nearly every pitch. If they don’t, they do not hang around long as MLB pitchers. Yet because pitchers will need to throw pitches at a faster pace, can they maintain that same level of effort? Even if they do will that be as effective as it has been recently? Will batting averages and scoring rise because pitchers will have to hold back to deliver pitches at a faster pace? Or will it be due to the restrictions on the infield shift?
In 2022, MLB hitters are poised to have the lowest batting average for an entire season since 1968. After the ’68 season the pitching mound was lowered from 15 to 10 inches and batting averages immediately improved. Of course, 1969 was also the first year of divisional play with the addition of 4 new teams. As a result, the player pool was a bit more diluted by adding these 100 new MLB players. Trying to evaluate the success of lowering the mound was not easy. Were the better averages and higher scoring attributable to the lowering of the pitching mound or the dilution of talent (allowing lesser pitchers to be on rosters since more pitchers were needed)? It seems MLB has a history of cloudy multivariate tests!
Should MLB batting averages rise next season, will it be due to the pitch clock? The restrictions on infield shifting? If runs scored per game goes up next season, will it be because pitchers get tired more quickly having to throw pitches at a faster rate? As with so many things, the answer is, it depends. But what’s most likely is that MLB and we probably will not know for sure. It goes my nature as a marketer to have to simply accept this.
Another new rule being instituted is the limitation of throws to first base by the pitcher. It will limit the pitcher to two throws over to first base per batter. Again, in MiLB testing, stolen base totals immediately rose and rose substantially. There will be more stolen bases and consequently more runs scored almost guaranteed. And we think this is a good thing, but it also makes it hard to measure the impact of restrictions on the infield shift.
On our podcast we’ve shared that there’s not much evidence to support restricting the infield shift. In MiLB testing overall batting average and runs scored were relatively the same before and after restricting the shift. There is some evidence that shows batting averages on very hard-hit balls, those at higher than 95MPH exit velocities, have dropped more than 100 points since teams began shifting. That’s something that fans (and MLB hitters) have noticed. When a player smashes one hard, and it goes right to the 2nd baseman standing in short right field leading to an easy groundout, the batter and fans are, well, let’s just say frustrated.
But what isn’t noticed as readily, are the weakly hit grounders that end up being a single since the ball was hit to the side on which only one fielder was positioned, and that lone fielder could not get to the ball and throw out the runner in time. To the fan, that’s much less satisfying to watch but the hit still counts and there’s now a runner on base just the same. MLB is concerned with fan enjoyment – as it should be. The objective is to have a better and more enjoyable game to watch. The new rules overall seem to be designed to do that.
Testing changes that might lead to new rules and a better product is smart and should continue. We would have preferred the pitch clock be tried for a season or two before considering restrictions on the infield shift. But that’s not the way it went and while overall MLB will likely be the better overall for the changes made, it’s unfortunate that we will not be able to accurately measure each’s impact.