An alternate proposal for scoring system reform has been submitted by ISU member Australia for consideration at the 2004 ISU Congress to be held in the Netherlands next June. This proposal consists of an extensive set of revisions to the current 6.0 system, with the aim of meeting the dual goals of improved honesty and integrity in judging, and an improved method of evaluating skating performances. The proposal incorporates the best of the current 6.0 system with many of the desirable concepts under discussion in the CoP approach, but with none of the latter systems many serious flaws.
Under Australia's proposal the panel of judges would be significantly altered to bring greater immunity to misconduct or error on the part of the judges. For Championship events, panels would consist of 11 judges and a substitute judge, plus the referee and assistant referee. The marks given by the judges would be combined using a double trimmed mean, meaning the two highest and two lowest marks would be dropped and the remaining seven marks averaged. For non-championship events either seven or nine judges would be used and a single trimmed mean taken to determine the results. In this approach, judges who give excessively biased marks will have their marks omitted from the average that determine the results. Seven marks will go into the averages, instead of the five used in CoP, resulting in increased statistical accuracy compared to CoP. Further, the impact of a single biased judge that might slip through the double trimmed mean is reduce by 40% in this proposal compared to CoP.
A second aspect of the proposal is to eliminate block judging and deal making by limiting panel representation on a geographic basis.
Six geographic regions are specified in the proposal, four in Europe, one for the Americas, and one for Asia, Africa and Oceania. Membership on panels would be limited to a maximum of two from each region. Even if both judges from a single region evaded the double trimmed mean they would still make up only about 1/4 of the marks averaged, so block judging would be effectively eliminated from the scoring process.
Rather than using the ordinal method or OBO to combine the judges marks, results would be based directly on the double trimmed mean. This approach was taken to prevent place switching in intermediate results, preserve the truest indication of victory and margin of victory in the results, and to allow skaters in the long program the opportunity to win the event after a poor short program performance, without requiring "help" from any other skaters.
The relative value of each part of an event would remain the same as currently; e.g., 1/3 for the short program and 2/3 for the long program. Scores in each part of an event would be scaled by the same weighting factors as today, and then added.
This approach to combining parts of events maintains a mathematically rigorous balance between the parts. It also prevents a skater from building up such huge lead in the short program that it cannot be overcome by another skater winning the long program. These two aspects of the proposal offer major improvement over both the current 6.0 system and CoP.
Australia's proposal retains the 6.0 scoring scale and the main marks for Technical Merit and Presentation, but includes significant changes to how those marks are applied to evaluating a skating performance. The proposal uses completely "reflective" judging in the use of the marks. Reflective judging is a hybrid of absolute judging (or judging "in the moment") and relative judging (comparing each performance to the others).
Judges have been encouraged to use reflective judging for many years, but the limitation of marking to the nearest 0.1 combined with the huge increase in the number of competitors in an event over several decades has prevented that approach from being used completely effectively. For example, in a championship event with 30 competitors there are only 30 marks between 3.0 and 6.0, the marks one would expect those skaters to span. This means there is only one mark for each competitor and thus marks must be saved, or marks that do not accurately reflect the individual Technical Merit or Presentation marks must be given.
Australia's proposal calls for marks to be specified to the nearest 0.05 points. This doubles the number of individual marks and increases the number of total marks by a factor of four. Under this proposal marks do not have to be saved, completely reflective marking can be used, and the point differences from one place to another become statistically meaningful.
The proposal, however, also calls for the judges to assign their marks not only by taking into account the appropriate individual marks on a nearly absolute scale, but also by considering where their total marks place the skaters relative to all the others. One of the greatest strengths of the current 6.0 system is that by considering placements, the place each skater receives is ten times more statistically meaningful than just marking in the moment alone. In the CoP approach, which does not use relative comparison of the skaters, the individual scores are statistically nearly meaningless. Rather than being a distraction, considering placements in a relative comparison is essential to obtaining the best possible scoring accuracy, and is unequivocally the fairest way of scoring an event using human judges.
Under the current 6.0 system each whole number is described in terms of the quality of a performance; e.g., 1.0 for very poor, 2.0 for poor, through 6.0 for perfect. In reality the scores reflect both the difficulty of the program attempted, and the quality with which it is performed. In the proposal submitted, the meaning of each marks has been more precisely defined to include both a standard of difficulty and quality.
Both the Technical Merit and Presentation marks are broken down into several sub-marks in this proposal. In singles, for example, the Technical Merit mark is subdivided into three sub-marks for Jumps, Spins, and Sequences and Connecting Moves. The Presentation mark is subdivided into two sub-marks for Harmonious Composition of the Program and Choreography.
In singles, jumps would account for one-half of the Technical Merit mark (0 to 3.0), while spins would be one-quarter (0 to 1.5), and sequences and connecting moves the final quarter (0 to 1.5). In pairs, lifts, jumps and throws would make up one-half the Technical Merit mark, spins and death spirals one-quarter, and sequences and connecting moves one-quarter.
In this approach, the contribution of each type of element to the skater's score is rigorously established. For singles, jumps would make up 25% of a skater's total score, compared to CoP where jumps make up about 42% of the total score. Spins under Australia's proposal would make up 12.5% of the total score compared to 8% under CoP. With the publication of these sub-marks all would know exactly the relative contribution of each type of element to the total score, and where each skater stands for each type of element.
For the second mark, each sub-mark would make up half the Presentation mark (0 to 3.0 for each). The sub-mark for composition would include
The sub-mark for choreography would include
In order to make consistent use the five sub-marks, the technical committee would be called upon to provide specific examples of the marks appropriate for common examples of program content. The purpose of this is to insure consistency in the marks from competition to competition and to assist in implementing completely reflective judging.
This proposal does not completely reject the value of using some technology in the judging process. Instant replay would continue to be used, and it is expected in championship competitions an automated note taking system would be used to completely record the content of programs and the quality of execution of the elements. The information recorded in this way would be provided to the skaters to allow them to analyze the judges' assessments of their performances, and could be used to evaluate the performance of the judges.
The proposal submitted by the Australian federation offers many improvements to the process of judging skating competitions without introducing any of the problems associated with the CoP system. It is sufficiently similar to the current 6.0 system to guarantee it will function as expected without the need to extensively retrain all judges, which means it can be painlessly adopted for use in all countries for all levels of competition. It is sufficiently new and innovative that it can bring real reform to the problems and concerns that face the judging of figure skating. Specifically, it offers the following benefits compared to the current 6.0 system and the proposed CoP system.
Copyright 2003 by George S. Rossano