Magnificent work. I really appreciated the care that went into differentiating between player effects and pitch effects. I have only very few criticisms if you will indulge me: 1) controlling for confounders is a great way of eliminating many issues, but it's only as good as the extent of control data points we have available. The canonical example is skill, which is an unobservable. Is it possible that bowlers bowled with more skill on some pitches, and this is what is really causing what we are interpreting as venue effects? As an example, consider a Bumrah bowling a lot of deliveries at the Wankhede with his wacky action. I think the main FE model wouldn't be partialing out this effect. 2) at the same time, controlling for too many features with <100k observations leads to problems of overfitting. I do worry whether that might be an issue with this approach? Could an alternative be to look at the first ball bowled in a match alone, as I think Amol Desai has done, before player effects have had a say?
Did have a separate one including bowler effects. Problem is its sparse and the effects contributed mainly to swing and turn/deviation. Both the indices remained pretty stable. I have mentioned it in a couple of places in the article.
For overfitting, I had around 200-900 observations per parameter plus thin data for any particular match/venue was handled by construction via bootstrap noise correction. I haven't seen Amol Desai's work on this. Thanks for the reference, I'll check it out!
Makes sense! I had noticed the spec with the bowler fixed effects, but as you correctly pointed out controlling for each bowler makes the data further sparse, effectively 'confusing' the model. (In fact, this is also a criticism of an alternative specification used in the Fourie-Siebrits paper that did not make it to my final draft.) Having said that, I'm convinced about overfitting not being a concern given you had sufficient observations per parameter. Beautiful article once again - I hope you continue your recent trail of work in inference along with your usual blockbusters in metrification!
Magnificent work. I really appreciated the care that went into differentiating between player effects and pitch effects. I have only very few criticisms if you will indulge me: 1) controlling for confounders is a great way of eliminating many issues, but it's only as good as the extent of control data points we have available. The canonical example is skill, which is an unobservable. Is it possible that bowlers bowled with more skill on some pitches, and this is what is really causing what we are interpreting as venue effects? As an example, consider a Bumrah bowling a lot of deliveries at the Wankhede with his wacky action. I think the main FE model wouldn't be partialing out this effect. 2) at the same time, controlling for too many features with <100k observations leads to problems of overfitting. I do worry whether that might be an issue with this approach? Could an alternative be to look at the first ball bowled in a match alone, as I think Amol Desai has done, before player effects have had a say?
Thank you Harigovind.
Did have a separate one including bowler effects. Problem is its sparse and the effects contributed mainly to swing and turn/deviation. Both the indices remained pretty stable. I have mentioned it in a couple of places in the article.
For overfitting, I had around 200-900 observations per parameter plus thin data for any particular match/venue was handled by construction via bootstrap noise correction. I haven't seen Amol Desai's work on this. Thanks for the reference, I'll check it out!
Makes sense! I had noticed the spec with the bowler fixed effects, but as you correctly pointed out controlling for each bowler makes the data further sparse, effectively 'confusing' the model. (In fact, this is also a criticism of an alternative specification used in the Fourie-Siebrits paper that did not make it to my final draft.) Having said that, I'm convinced about overfitting not being a concern given you had sufficient observations per parameter. Beautiful article once again - I hope you continue your recent trail of work in inference along with your usual blockbusters in metrification!
Fantastic piece of putting data to what goes off as popular discourse. So so well explained.
really cool analysis