It's all conjecture.
I would say that the performances you cited in 2012-2015 under different managers shows that the manager has little impact. I mean if Matt Williams was that bad a manager does that mean they would have won even more games in 2014. I doubt it. I don't think the difference between a good and bad manager is more than 2-3 games a year. If we were talking about not extending Joe Madden that would be something.