Size | Run Time (s) | Break-even (s/cfg.) | Coefficient of Variation | Coefficient of Variation | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
TSR vs. | All vs. | in Size (%) | in Run Time (%) | ||||||||||||
mAETG | mAETG | mAETG | mAETG | All vs. | mAETG | mAETG | |||||||||
Name | Best | TSR | All | Best | TSR | All | Best | Best | TSR | Best | TSR | All | Best | TSR | All |
SPIN-S | 27.0 | 20.6 | 19.4 | 0.2 | 2.2 | 8.6 | 0.32 | 1.11 | 5.31 | 3.3 | 5.7 | 3.4 | 4.2 | 40.4 | 75.6 |
SPIN-V | 42.5 | 42.9 | 36.8 | 11.3 | 32.5 | 102.1 | N/A | 15.93 | 11.44 | 3.0 | 6.4 | 6.3 | 3.0 | 59.5 | 121.6 |
GCC | 24.7 | 25.8 | 21.1 | 204.0 | 661.7 | 1,902.0 | N/A | 471.67 | 266.15 | 4.4 | 8.5 | 11.7 | 4.4 | 39.4 | 88.1 |
Apache | 42.6 | 34.6 | 32.3 | 76.4 | 61.8 | 109.1 | 0.00 | 3.17 | 20.55 | 3.6 | 4.0 | 5.0 | 3.6 | 11.5 | 137.9 |
Bugzilla | 21.8 | 16.9 | 16.2 | 1.9 | 3.4 | 9.1 | 0.31 | 1.29 | 8.32 | 5.2 | 6.4 | 3.1 | 5.3 | 42.6 | 74.8 |
1. | 53.3 | 42.5 | 38.6 | 24.4 | 27.9 | 179.5 | 0.32 | 10.55 | 38.68 | 3.1 | 2.9 | 2.8 | 3.9 | 21.6 | 138.3 |
2. | 40.5 | 32.9 | 31.0 | 14.7 | 23.5 | 25.4 | 1.16 | 1.13 | 1.00 | 4.1 | 7.0 | 3.6 | 4.0 | 73.7 | 86.9 |
3. | 20.8 | 18.3 | 18.0 | 0.2 | 2.5 | 3.4 | 0.90 | 1.14 | 3.33 | 6.9 | 3.3 | 1.1 | 6.8 | 32.5 | 61.5 |
4. | 28.6 | 21.8 | 21.0 | 3.1 | 7.3 | 29.3 | 0.61 | 3.45 | 29.01 | 4.0 | 5.1 | 5.5 | 4.3 | 23.4 | 323.0 |
5. | 63.8 | 54.8 | 47.7 | 134.8 | 165.6 | 655.9 | 3.43 | 32.37 | 69.05 | 1.9 | 8.8 | 3.2 | 4.2 | 95.7 | 133.3 |
6. | 34.0 | 24.2 | 24.2 | 7.2 | 9.1 | 18.2 | 0.19 | 1.12 | 456.71 | 4.5 | 2.7 | 1.5 | 4.6 | 26.6 | 54.9 |
7. | 12.5 | 9.0 | 9.0 | 0.3 | 1.5 | 2.1 | 0.33 | 0.51 | N/A | 4.9 | 0.0 | 0.0 | 8.7 | 2.4 | 4.0 |
8. | 55.6 | 44.0 | 40.5 | 45.1 | 41.2 | 249.9 | 0.00 | 13.56 | 60.32 | 3.3 | 3.6 | 3.3 | 3.3 | 23.2 | 220.4 |
9. | 26.0 | 30.0 | 20.0 | 3.0 | 3.4 | 29.8 | N/A | 4.47 | 2.64 | 5.3 | 0.0 | 1.0 | 5.3 | 2.7 | 54.0 |
10. | 60.4 | 48.2 | 44.0 | 74.3 | 60.8 | 357.3 | 0.00 | 17.26 | 70.26 | 2.6 | 2.4 | 2.1 | 2.6 | 15.1 | 89.2 |
11. | 58.3 | 46.1 | 41.9 | 23.8 | 27.0 | 240.7 | 0.26 | 13.23 | 50.65 | 2.5 | 2.2 | 1.9 | 2.8 | 20.0 | 94.5 |
12. | 54.5 | 43.2 | 40.4 | 68.0 | 53.7 | 221.1 | 0.00 | 10.86 | 58.95 | 3.5 | 2.5 | 4.1 | 3.6 | 15.3 | 151.6 |
13. | 48.6 | 37.7 | 36.5 | 45.5 | 40.1 | 60.5 | 0.00 | 1.24 | 17.28 | 5.4 | 3.1 | 2.2 | 5.4 | 12.6 | 94.0 |
14. | 51.8 | 39.6 | 36.9 | 20.2 | 23.9 | 58.1 | 0.30 | 2.54 | 12.68 | 4.0 | 3.1 | 2.0 | 4.1 | 18.4 | 71.6 |
15. | 40.4 | 31.8 | 30.7 | 4.9 | 9.8 | 19.3 | 0.57 | 1.48 | 8.65 | 3.4 | 3.9 | 2.7 | 3.3 | 18.5 | 76.4 |
16. | 33.4 | 24.3 | 24.1 | 9.7 | 11.2 | 19.7 | 0.16 | 1.08 | 38.81 | 5.3 | 3.6 | 1.0 | 5.5 | 21.2 | 35.1 |
17. | 53.4 | 42.6 | 39.2 | 46.9 | 45.8 | 335.5 | 0.00 | 20.32 | 84.21 | 3.3 | 2.9 | 3.9 | 3.4 | 14.7 | 171.8 |
18. | 57.3 | 45.9 | 42.3 | 60.1 | 52.8 | 303.5 | 0.00 | 16.23 | 70.43 | 2.1 | 2.7 | 3.6 | 2.2 | 13.6 | 108.3 |
19. | 64.7 | 52.3 | 47.8 | 168.4 | 124.2 | 823.6 | 0.00 | 38.77 | 154.72 | 2.4 | 4.5 | 2.4 | 2.5 | 47.7 | 86.1 |
20. | 71.5 | 58.5 | 53.2 | 121.1 | 86.3 | 1,133.3 | 0.00 | 55.31 | 199.05 | 1.7 | 4.8 | 1.7 | 1.8 | 63.4 | 171.4 |
21. | 51.7 | 38.1 | 36.3 | 14.3 | 18.7 | 46.2 | 0.32 | 2.07 | 15.29 | 3.9 | 3.4 | 1.3 | 3.9 | 17.1 | 89.1 |
22. | 26.2 | 36.0 | 36.0 | 6.7 | 8.9 | 12.3 | N/A | N/A | 86.24 | 4.6 | 0.5 | 0.0 | 4.5 | 14.8 | 39.4 |
23. | 15.7 | 18.0 | 12.6 | 0.6 | 0.5 | 10.1 | 0.04 | 3.06 | 1.78 | 5.4 | 0.0 | 7.5 | 6.0 | 2.8 | 52.2 |
24. | 58.3 | 47.0 | 42.8 | 40.6 | 39.6 | 304.5 | 0.00 | 17.03 | 63.38 | 2.7 | 2.6 | 2.3 | 2.6 | 22.8 | 128.1 |
25. | 65.7 | 53.1 | 48.3 | 61.7 | 53.1 | 507.8 | 0.00 | 25.64 | 93.94 | 2.0 | 2.1 | 1.7 | 7.7 | 21.0 | 104.0 |
26. | 42.0 | 32.9 | 30.7 | 15.5 | 22.0 | 71.2 | 0.71 | 4.93 | 22.18 | 2.8 | 5.9 | 2.5 | 3.3 | 26.6 | 75.8 |
27. | 45.8 | 36.5 | 36.0 | 4.6 | 8.2 | 10.8 | 0.39 | 0.63 | 5.15 | 4.6 | 2.0 | 0.4 | 4.6 | 24.5 | 60.5 |
28. | 68.5 | 55.4 | 50.7 | 170.4 | 121.6 | 1,522.3 | 0.00 | 75.95 | 296.75 | 2.1 | 2.9 | 2.4 | 2.1 | 27.4 | 119.9 |
29. | 38.4 | 31.6 | 29.7 | 38.9 | 41.0 | 89.3 | 0.31 | 5.79 | 25.41 | 4.1 | 3.1 | 3.3 | 4.1 | 17.7 | 117.0 |
30. | 45.8 | 20.4 | 19.3 | 10.3 | 17.5 | 30.5 | 0.28 | 0.76 | 12.23 | 6.9 | 4.3 | 4.9 | 6.8 | 113.2 | 129.0 |
Sum | 1,546.1 | 1,257.4 | 1,155.4 | 1,533.1 | 1,910.4 | 9,501.8 | 1.31 | 20.40 | 74.40 | 0.6 | 0.8 | 0.6 | 0.9 | 16.8 | 37.6 |