Status of COMPMNGR's Judge's Handheld Marking Device System

First I'd like to thank the organizers of the Hill Country Classic Dance competition, David and Shelly Vance, and the organizers of the Texas Challenge, Peggy Heeney, Rosendo Fumero, and Phillip Stephens, for allowing me to try COMPMNGR's judge's handheld marking system at their competitions. I'd also like to thank the Texas Challenge scrutineer, Leroy Walters, for his help in keeping the system up and running. Since there appears to be a great deal of interest in such systems, I thought it would be worthwhile to report on the status of the COMPMNGR system.

A little history might be in order. An East Coast amateur dancer named Bill Smyth was designing such a system around an early PDA (personal digital assistant) around ten years ago and approached me about providing an interface with COMPMNGR. I developed an interface but never got to test it since, for reasons I don't remember, he dropped the project. At that time PDAs were pretty pricey, so a complete system based on such would probably have been prohibitively expensive anyway. As a sidelight for you scrutineers, that PDA interface was the origin of the "PLC" file method of backing up the judge's marks. Over the last couple of years the price of PDAs with wireless networking capability has dropped substantially. More important, some brands come with a version of the Microsoft Windows operating system which allowed me to use a version of the Microsoft C++ compiler to write a little marking system program for them.

The COMPMNGR system consists of a wireless router, a notebook computer with wireless networking capability, and several PDAs also with wireless networking capability. Three PDAs were used in the system at the Hill Country Classic. Suffice it to say, the system did not work flawlessly. The biggest problem was that PDAs would randomly lose their connection to the network. Out of about 750 opportunities, connection was lost twenty or so times. In the event of a lost connection a PDA did not receive heat information from the notebook computer running the COMPMNGR program. That problem seemed to occur randomly. Another problem was in the COMPMNGR software. It required that either all judge's marks come from PDAs or all transcribed manually from paper marking sheets into the COMPMNGR scrutineer's data entry box. In spite of these problems there appeared to be enough promise to justify continued development.

So I modified COMPMNGR to allow a mix of PDA marks and paper marks, made some other modifications based on what I learned at the Hill Country Classic, and bought six more PDAs for a total of nine. I also bought plastic cases for the PDAs and mounted them to clipboards which also hold paper marking sheets. That way, in the event of PDA failure or judge confusion, the judge could quickly switch to paper marking while I rebooted the PDA. That was the system which was tested at the Texas Challenge.

Things went somewhat better at the Texas Challenge, although by no means perfectly. The problems fall into four categories: software, hardware, procedures, and training. The software actually performed pretty well with one exception. Partway through the Friday Matinee session we discovered that if a heat contained a "scratched" entry COMPMNGR would not send all the heat information to the PDAs. However, I repaired that problem by the end of the day so testing was continued on Saturday and Sunday. I am now pretty happy with the reliability of the software, although I may make a few more modifications to make it easier to use and trap some user errors.

The second problem area was hardware. Once again, there was a problem with PDAs losing connection with the network. On Friday the wireless router was located on a dinner table in the corner of the ballroom. Saturday and Sunday the router was mounted on the dais, higher and more centrally located. The higher location seemed to help somewhat but certainly didn't solve the problem. It appeared that "dropouts" were more common during awards breaks, when the judges would walk around the ballroom carrying their PDAs. That, plus the fact that the wireless connection is so much more reliable in my usual (home) test environment, leads me to speculate that interference from other networks around the hotel is at least partially responsible and that a more powerful wireless router might help the problem. (I had bought one of the cheapest models for early testing and had never replaced it.) Also, one of the judges, Patrick Johnson, told me that a later generation of PDAs provides more reliable wireless communication than the generation used in the COMPMNGR system. I plan to buy a more powerful router model before the next test, but can't afford to replace all the PDAs.

The third problem area had to do with procedures. Any hardware system can suffer component failure. To keep things going smoothly fallback procedures are required. In the COMPMNGR system the fallback procedure is paper marking. When paper marking is used the master of ceremonies usually announces a heat by calling out the competitor's numbers. If there are several competitions on the floor at the same time, which is common in pro/am events, the judges write down the numbers of competitors competing against each other. For example, in heat 112 numbers 101, 103, and 105 are competing against each other; 110 and 120 are competing against each other; and 130 is uncontested. In the COMPMNGR marking system the COMPMNGR computer sends such heat information to the PDAs just prior to each heat (to allow for late entries and scratches) and the judge doesn't have to write down these numbers if his or her PDA received the heat information. But the judge doesn't know if the information was received until too late unless the correct procedure is used. That procedure requires the COMPMNGR computer to send the heat information before the master of ceremonies announces the heat number. If a judge's PDA has not received the information when the heat number is announced he or she knows that he/she must write down the competitor numbers for paper marking. So it is absolutely essential that the scrutineer, the master of ceremonies, and the judges coordinate their efforts.

The fourth problem area was training, or lack thereof. Some judges are computer literate, and some are proud of the fact that they are not. Moreover, PDA screens are small and packed full of information and "buttons"which make the PDAs hard to manage for those with poor eyesight or an unsteady hand. If the scrutineer, the master of ceremonies, and the judges are not comfortable with the system and procedures the system will not work well and the competition will not run smoothly. Probably the best training method would be to hold a mock competition involving the scrutineer, the master of ceremonies, and the judges. The mock competition would familiarize those folks with the procedures as well as the software user interface.

In summary, when the marking system is working properly it is way cool to watch and may even speed things up slightly under some circumstances. A couple of judges said they preferred the system to paper marking as it gave them more time to watch the dancers, but some judges didn't like the system.  However it is expensive and is not as robust as the traditional paper marking system since in the paper marking system the judges have a lot of leeway in how they mark, and if they go too far afield the scrutineers and chairman of judges can fix things quickly. The marker system requires more training and coordination for things to go smoothly.

As to the future, I plan to make a few user interface improvements in the software and test the system a little more. Then I hope some enterprising and energetic young people will assemble their own PDA systems using the software so I can get out of the business of providing the hardware system and support at competitions. My interest is solely in COMPMNGR and its supporting programs.