It seems a lot of people have unreasonable expectations of what an 'Ace' starting pitcher is. These expectations carry on down through each level of starter (#2, #3, #4 and #5).
A lot of smarter people than I have done similar exercises and done them more rigorously, but I'll throw up a quick post just to give an idea.
The job of a pitcher is to prevent the opposing team from scoring runs (Now, some pitchers-those who play 'real' baseball in the National League-also have the opportunity to help their team score runs...). So, it reasons that the pitchers who do that best are 'aces'. Each team has to have an 'ace' so it figures that the 30 pitchers (one per team) who prevent the most runs over the course of a season are nominal 'aces'.
Using Fangraphs I ordered each major league starting pitcher for 2010 by most runs prevented. I then separated those into groups of 30. The top 30 pitchers would be 'aces' and on down the line for the 2nd - 5th slots. All pitchers after #150 were grouped together into the 'AAA' category. I then took the simple average for each group and assume that as a 'representative' of each group. These are the results.
The average 'ace' had a season line of: 14 - 10 3.23 ERA in 32 starts, 212 innings, 65 walks and 192 strike outs.
Pitchers in this category included: Cliff Lee, Roy Halladay, Tim Lincecum, Clayton Kershaw, Colby Lewis, CJ Wilson, Brett Myers, Matt Cain and Ricky Romero.
The average #2 had a season line of: 12 - 11 3.58 ERA in 31 starts, 193 innings, 62 walks and 148 strike outs.
Pitchers in this category included: Matt Latos, Cole Hamels, Chris Carpenter, Clay Bucholz, Johan Santana, Derek Lowe, Jason Vargas, and RA Dickey.
The average #3 had a season line of: 10 - 8 3.99 ERA in 27 starts, 160 innings, 50 walks, 126 strike outs.
Pitchers in this category included: Stephen Strasburg, Ervin Santana, Daisuke Matsuzaka, Ted Lilly, Andy Pettitte, Barry Zito, Carlos Zambrano, and Joe Blanton
The average #4 had a season line of: 8 - 8 4.51 ERA in 23 starts, 135 innings, 45 walks, 95 strike outs.
Pitchers in this category included: Rick Porcello, Matt Garza, Jake Peavy, Luke Hochevar, Bronson Arroyo, AJ Burnett, Mike Leake, John Lannan
The average #5 had a season line of: 6 - 7 4.52 ERA in 18 starts, 102 innings, 35 walks, 72 strike outs.
Pitchers in this category included: Aaron Harang, Jeremy Bonderman, Kenshin Kawakami, Jon Garland, JA Happ, Kyle Kendrick, Brad Bergeson and Ben Sheets.
Adding up all the starts made by the top 5 starts gives 130 starts. That means that an average team can expect 32 starts made by pitchers other than the top 5.
These pitchers had an average season line of: 2 - 4 6.18 ERA in 8 starts, 40 innings, 19 walks, 28 strike outs.
Pitchers in this category included: Armando Galarraga, Nick Blackburn, Jamie Moyer, Vin Mazzaro, Wade LeBlanc, Dontrelle Willis, Jeff Suppan, Gil Meche, Javier Vazquez, Oliver Perez, and Rich Harden. As well as lesser knowns like: Carlos Montasterios, Anthony Lerew, David Huff, Sean O'Sullivan, Luke French, and Barry Enright.
Now I'd imagine that most people would see a pitcher with a 14 - 10 record with a 3.23 ERA and think that he was a 'good' pitcher and not representative of the best 30 pitchers in baseball.
Look at the pitchers outside of the top 5 who make starts (an average of 32 per team). They are terrible. Hopefully this shows the value of quantity of innings vs just quality of innings. When a pitcher like Roy Halladay is capable of pitching 250 innings a season he is effectively taking away 38 innings (250 minus the 212 innings expected of an ace) from these replacement level pitchers.
Speaking of replacement level, the following is a chart of the average WAR for each level of pitcher.
Ace = 5.05
#2 = 3.25
#3 = 2.28
#4 = 1.52
#5 = .77
AAA = -.01
The AAA pitchers performed almost exactly at replacement level.
Now one of the (many) problems with this study is the use of win-loss records and ERA. These win-loss records are effected by the quality of the pitcher's team. In the Ace category we have SS Sabathia at 21 - 7 and Anibal Sanchez at 13 - 12. This problem isn't relegated to this study only. W - L is a poor measure of a pitcher.
ERA is also a problem. This study makes no differentiation between NL and AL. Clearly, pitchers in the NL have an easier time of it getting to face the opposing pitcher a time or three or four while an AL Pitcher has to face David Ortiz. But, again, this problem isn't limited to this study and is a basic flaw in using ERA to compare pitchers.
I'm going to recreate the charts using more SABR stats
Level IP, BB/9, K/9, FIP, xFIP, WAR
Ace 212, 2.8, 8.2, 3.25, 3.66, 5.05
#2 192, 2.9, 6.9, 3.75, 4.05, 3.25
#3 160, 2.9, 7.1, 3.99, 3.93, 2.28
#4 135, 3.0, 6.3, 4.22, 4.39, 1.52
#5 102, 3.1, 6.3, 4.34, 4.47, 0.77
AAA 40, 4.2, 6.2, 5.37, 5.03, -.01
A couple of interesting things to note:
1st: The difference between the Ace group and the #2's is the largest jump in WAR between any two groups. This is mostly because of the dominance of a few pitchers like Lee and Halladay. It is amazing (and understated) just how much better these guys are than most other pitchers out there.
2nd: The small difference between the #2's and #3's. The walk rates and strike out rates are basically even between the two groups. And the #3 group has the better xFIP. The only reason that the #2 group out performs the #3 group is number of innings. Again, quantity of innings plays a big role along with quality of innings.
Next time your team signs a pitcher with a 10 - 8 record and 3.99 ERA in 160 innings realize just what you are getting. One of the top 100 pitchers in the league.