For nearly a year the Glacier Falls FSC (GFFSC), located in Anaheim, California, has been working on adapting, and adopting, the new judging system for use at its competitions at all event levels.
At its annual Summer Classic Open Competition in August, 2004, the club introduced the use of instant replay into local competition. The hardware used in this replay system was engineered to support future use of the new judging system, as this test was just the first step to bring the new judging system into use at the club. Since then, software to run the new judging system was written and combined with the video replay software tested in August.
A second engineering test for this project was recently conducted at the West Coast Open Championships hosted by the Tri-Valley FSC (TVFSC), located in Valencia, California. The purpose of this test was to exercise the scoring system software outside the laboratory, in a real competition environment, and to obtain feedback from local judges. Event results were not calculated in this test, as that was not the purpose of the effort.
This report describes this recent test, and the resulting lessons learned on the road to putting the new judging system into use within the club. As in the August test, this project is not an official project of U.S. Figure Skating, and is being conducted by the Glacier Falls FSC, at the expense of the author, who is currently the club president.
For this test, hardware to support three judges and one caller, with real-time video and instant replay, was installed at the rink.
In August, the installation was fairly simple (even though it involved hardware for seven judges), since GFFSC’s home rink is fairly spacious compared to the typical local ice rink, and the benches there can be removed from the hockey boxes. At TVFSC, the hockey boxes are of standard size, with fixed benches running the length of the boxes, in the middle of each. With the addition of tables and chairs, the installation was crowded, but still manageable with some effort.
The competition videographer, Hurd Video, again provided the video feed for the real-time video display and instant replay. In this test, the video camera was located at the end of the same hockey box used by the scoring system equipment, and the video camera and scoring system equipment rack used the same power circuit.
Initially it was found that the video equipment experienced RF interference and an adjustment to the video installation had to be made to eliminate the problem.
The current test demonstrated that the system architecture is flexible enough to support a variety of configurations in different rinks, and reinforced the expectation that careful planning must be given to the installation before arriving at the rink. It should also be obvious that the installation must be completed far enough in advance to work out unforeseen installation issues. The installation plan must also be coordinated with the competition videographer, to prevent problems with their activities.
One judge commented that the difference between the perspective offered by the video camera and the direct view of the ice was somewhat distracting, but there is little a host club can do about that. It is unlikely that the typical club would have the room to place the video camera in the center of the panel looking out over the judges, as is the standard configuration at ISU events, and the perspective would still be different for most of the judges. Further, it is unlikely the videographer would want to be located up there. Of course, if it could be arranged, there is no reason not to do it that way.
Given that a fundamental design goal of this project is to minimize system cost and complexity, the position is taken that a different view from the video camera obtained for free, wherever it is located, is vastly better than no video feed at all; particularly since it is a fundamental premise of the project that instant replay is critical to equitable use of the new judging system.
For this test the author served as the caller. For some events, the skaters were asked to provide a list of the elements in their programs using a standardized form. Other events were called on the fly. The skaters appeared to have no problems filling out the form they were provided.
Prior to this test, it was hoped that local competitions could be judged using only one caller on duty at a time to prevent manpower demands from putting too great a strain on a host club. After calling a couple of No-Test events in a row, however, that hope went down in flames.
The conclusion from this test is that the caller is a necessary evil and that the caller must have an assistant, with the operative words necessary and evil. Evil, because even with three persons involved, the recent Olympic Games showed that mistakes can be made that affect medal results. Necessary, because the only other option currently available (no caller) is even worse. Those who think that a caller-less competition can be run under the new judging system are not doing the skaters any favors by promoting that point of view.
It was also clear from this test that having the judges call their own elements and also enter GoEs was too great a distraction during a performance, and put too great a burden on the judges. We reject the ISUs notion that the judges are too darn stupid to call the elements themselves. However, asking the judges to call the elements and judge is simply asking them to do too much, and so a calling process is needed for that reason, as well as to synchronize the data entry from all the judges.
Calling events on the fly is extremely stressful, and far more difficult for the lower levels than the upper levels. Based on this and previous tests, we conclude the following:
- ALL skaters MUST provide a trick list for each of their programs, to be submitted to the host club with their competition application. The Registrar would forward these forms to the chief caller for the competition, to allow advance preparation of a set of trick lists for each event. Applications would not be considered complete without trick lists.
- Even for local competitions, the caller should be someone who has passed the Technical Specialist Test. Utilization of the caller on panels should follow the same time guidelines as for judges.
- An assistant technical specialist is needed to serve as an extra set of eyes for the caller to minimize the frequency of calling errors. Any qualified judge or coach is capable of performing this job at a non-qualifying competition, and since there are not going to be all that many official technical specialists available to work competitions at first, this may be the only choice. Indeed, availability of technical specialists may be the fundamental limitation to rolling out the new judging system nationwide, and not the speed with which judges can be trained.
- A better way of doing the caller’s function needs to be worked out, but until it is, a qualified caller and assistant are essential, even in the most basic local competition.
To avoid the cost and complexity of replay technicians and additional hardware, the system uses a self-serve approach to selection of video clips. This worked well in the August test, and is currently the approach that will continue to be taken for the judges. For the caller, however, that is not the case. The callers are just too busy doing their job to also manipulate selection of video clips.
To provide the caller instant replay, which is even more important than for the judges, the system has been modified since this test to automatically save a video clip whenever the caller initially identifies an element.
Our current estimate for system hardware that supports seven judges and one caller, and sufficient spare parts for quick resolution of hardware failures, is $18,500. If a large group of clubs (16-20) share this cost and use the system 40 days a year, the system pays for itself in four years with a daily use fee of $125 per day, or about $1 per start. On top of this, of course, are shipping expenses and cost for the computer tech who is needed to set the system up and keep the whole thing running.
For most areas of the country there are enough clubs in reasonable geographic proximity for this approach to work. For some areas, however, this is not the case (Alaska and Hawaii, for example). In these areas the economic burden on clubs may be too great. How this problem is resolved remains to be seen, but a pure paper system is not the answer. One approach may be for Headquarters to subsidize a few systems, or subsidize shipping of systems to such areas of the country. A semi-paper system is another possibility.
In the course of this project, several approaches to a pure paper system (and worse yet, a caller-less paper system) have been studied. At best they sort of work, but not very conveniently, and all of them in our view are a gross disservice to the skaters. (By a pure paper system, we mean a system where the judges and caller use paper scoring sheets and the accountants enter the marks into their computers in some way.)
In developing the software, however, a "minimalist" approach was identified that works tolerably well, though at the cost of depriving the judges of instant replay. This approach has been tested in the laboratory, and will be tested in a rink in the next phase of this project.
This minimalist system consists of one "caller" computer, with instant replay, networked to the existing accounting computer(s), and appropriately configured software.
The caller operates this computer in the same way as the full-up hardware system and also literally calls (speaks) the element identifications to the panel. If needed, a throat mike and speakers can be attached to the caller computer (or inexpensive intercom hardware) so the caller doesn’t have to shout at the top of his lungs to be heard.
The judges fill out personal score sheets with the GoEs of the elements and the PCs. After each performance, the caller once again announces the complete list of elements for the performance and then the sheets are given to the accountants, who enter the judges’ marks entered into their computer(s). The called list of elements is automatically taken from the caller computer into the accountant computer(s) over the network.
Software to enter the marks expeditiously has been tested in the lab. The result of timing test are that marks from a panel of nine judges on a senior level free skating event can be entered in less than five minutes for an individual skater, meaning results for an event would be available a few minutes after the completion of the event. For lower level events, with a panel of seven judges, the marks were typically entered in less than two minutes in the laboratory tests. There appears to be no problem with the accountants keeping up with the event in this approach.
Since the accountants already have the necessary computer hardware, the cost for the minimalist approach is currently about $1750 for the caller computer with instant replay, and about $100 for the networking hardware to connect to the accounting computer(s) if it isn’t already available within a club. The incremental cost to upgrade an existing computer to the specifications of a caller computer could be as low as $750 depending on what computer hardware a club has available; however, it would also be prudent to have on hand sufficient spare hardware to correct failures of the caller computer during a competition, which will increase the cost.
Now that most hardware and software issues have been resolved, the next step is to educate local skaters and coaches within the club. Once this educational activity is completed GFFSC is planning a mock competition as a dry-run for real competition. Planning for a subsequent club competition using point based scoring for all events has begun, and information will begin to become available on the club web site later this fall, starting with event requirements and descriptions.
It would also be useful and interesting to have a shadow panel judge a local competition with this system; however, such a test is difficult to arrange.
The current baseline approach for events in the planned club competition using the new judging system is as follows:
- Seven judge panels.
- One event referee.
- One technical specialist.
- One assistant technical specialist.
- One computer specialist to set up and maintain the hardware.
- Complete computer hardware system with instant replay.
- Judging system hardware networked with existing accounting computers.
- Video feed to be provided by competition videographer, as is.
- No replay technician or hardware. Automatic save of video clips for the caller. Self-serve video clips for the judges.
- Skaters required to provide a trick list for each performance when they submit their entry.
Compared to current practices, this approach makes us of judging system hardware networked to existing accounting computers, with the addition of three persons on duty for each event.
The Glacier Falls FSC and the author thank the Tri-Valley FSC, Hurd Video, and the local judges who participated in this test for their cooperation, suggestions and feedback.
Copyright 2004 by George S. Rossano
4 October 2004