Status of COMPMNGR's Judge's Handheld Marking Device System
This report contains four sections:
Summary
Description of test system
Observations
Where to go from here
You might also want to read the status report on the second test, conducted at the Texas
Challenge, and the system user's manual.
COMPMNGR's handheld judge's marking system was used for the third time last weekend at Randy Ferguson's Dance Houston. I'd like to thank Randy and Audra for allowing me to test the system.
The bad news is that the wireless networking connection was still unreliable for significant periods of time, mostly in the evening sessions. The good news is that during most of the day sessions the networking was fairly reliable, enough so that I could reach the following conclusions, which are pretty much the same as after the second test. My main conclusion is that the use of judge's hand held making devices, while "way cool" to watch, provides very little real benefit to the competition organizer. Of course, being "way cool" will be sufficient for some. Just below are lists of advantages and disadvantages.
Their advantages are:
●They prevent judges from making some common
errors, such as marking an entrant twice or failing to recall the correct number
of entrants from a semi-final round.
●They may speed things up very slightly under some
circumstances since runners are not required and the scrutineer doesn't have to
manually enter the marks into COMPMNGR.
●Some the majority of judges seem to
like using the system; some said they prefer the system over paper
marking as it gave them more time to watch the dancers.
Their disadvantages are:
●All judges require some training. At least at this point in time, some
judges do not like computers, are inexperienced with them, and require extra
training.
●The scrutineer and master of ceremonies also require some training.
●The marking devices are small and can be difficult to use for
persons with
poor eyesight or shaky hands.
●They may make it more difficult to handle last minute changes such as
scratches, write-ins, and judge substitutions.
●Even a marking system based on the
least expensive PDAs (personal digital assistants) having wireless networking
capabilities is expensive. (The PDAs used in the test system cost about $300
apiece.)
●They require constant monitoring by a technician.
●Systems based on the least expensive
PDAs may not be all that reliable.
As far as COMPMNGR's support for handheld judge's marking system, I have taken it about as far as I care to, although I do plan to make a few more improvements to make the system more reliable and user friendly. When those improvements are completed I will provide the software (a program which runs on Windows based personal digital assistants and a COMPMNGR interface to communicate with them) as an option to COMPMNGR users. I never intended to sell or rent the hardware for such systems, and certainly do not want to travel to competitions in order to provide setup or support services. It is my hope that some young, energetic, and highly computer literate scrutineers will be interested in doing that. So if you know of such persons please steer them to me; I'll be happy to work with them as long as I can do so while sitting in my office smoking a cigar. The following material is intended primarily for such interested parties.
The hardware
The test system hardware consisted of:
●An Averatec 3300 notebook computer running COMPMNGR under Windows XP
●A D-Link WBR-1310 wireless router, connected by wire to the computer, and
a Hawking Technology HSB2 signal booster set to provide a 500mW signal (about 10
time the usual signal strength)
●Nine HP iPAQ rx3115 PDAs running the handheld marker program under Microsoft® Windows
Mobile™ 2003 Second Edition software for Pocket PC
The software
As explained in the earlier test review article, some years ago I provided COMPMNGR with an interface for a handheld marker system to be developed by another party. That marker system was never completed. The interface is very simple in concept. COMPMNGR writes a text file describing an upcoming heat (competitors, judges, dances, etc.) with the judge's marks set to 0 as place markers; these are called 'OUT' files since that is their file name extension. The separate marker system program running concurrently with COMPMNGR on the scrutineer's computer has two tasks: (1) it reads OUT files and sends the information to the handheld markers, and (2) it receives marks from the handheld markers, inserts them into the OUT file, and when all marks are inserted renames the file with the same root name and the extension 'INP'. COMPMNGR imports the INP files into scoresheet records, tabulates placements, and posts the placements in its on-screen program.
Over the last few years the price of PDAs with wireless networking capability has dropped substantially. Moreover, some brands now come with a version of the Microsoft Windows operating system which allowed me to use a version of the Microsoft C++ compiler to write programs for them. So I bought a PDA and wrote a little marking system program called HandheldMarker for it. I also wrote a program called MarkerServer to run concurrently with COMPMNGR on the scrutineer's computer to handle the OUT and INP files. When initial tests indicated that HandheldMarker and MarkerServer were working correctly, I integrated MarkerServer into COMPMNGR as a separate thread of execution (actually as a modeless dialog box).
Both MarkerServer and HandheldMarker use the WinSock API (the Windows implementation of the Unix sockets TCP/IP technology) to communicate over the network. In fact, I wrote a single program module, a C++ file incorporating a couple of C++ classes, to handle both the server task in MarkerServer and the client task in HandheldMarker. The sockets experts among you will want to know that I used a "non-blocking" sockets technique, a separate thread of execution for each socket.
For diagnostic purposes, in this test we attempted to keep all nine PDAs "connected" to the scrutineer's computer at all times, even when only a smaller number were actually being used for judging. To be precise, the MarkerServer module acting as server on the scrutineer's computer was in communication with the HandheldMarker program running as client on the PDAs. In the following, a "loss of connection" means the loss of communication between a PDA and the scrutineer's computer. In all cases of loss of connection, it was necessary to reboot the PDA before a connection could be reestablished.
Probably the most important observation was that, in many of the cases of lost connections, all or a majority of the nine PDAs lost their connection at about the same time. Further, the PDAs losing connection were ones that were not at the time being used for judging and so were inactive. It appeared that the inactive PDAs lost their connection about six to eight minutes after the connection was made.
The HandheldMarker program has a feature allowing the user to tap a button to "ping" the MarkerServer module and thereby send a message requesting MarkerServer to reply with a message to confirm that HandheldMarker is still connected. When the ping feature was used about every five minutes the inactive PDAs did not lose connection. The fact that a number of PDAs lost connection after a period of inactivity suggests that there is some kind of "time-out" mechanism built into the PDAs or into the sockets implementation, although I could find no documentation to indicate that is the case. Nor do any of the wireless settings screens on the PDAs refer to such a time-out.
It was observed that the problem with lost connections was much worse during periods with larger multi-dance heats having more judges, such as occurred in the evening sessions. Then even the PDAs being used for judging lost connection. It is possible that having more judges contributes to the problem, possibly due to timing conflicts. But it is also true that during these periods there tended to be larger time gaps between heats, so that a problem due to time-outs would be more likely to occur.
In the two previous tests it was thought that the connection losses were usually random in time. Generally this was with a smaller number of PDAs, so it is possible that there was a time-out pattern that was not observed.
Following the competition I added a test mode to COMPMNGR so that it would ping the PDAs requesting a response from them at an interval I could adjust. With a five minute interval no PDAs lost connection over a three hour period. With a six minute interval all PDAs lost connection within five intervals, most but not all during the first interval. This tends to confirm that there is some sort of time-out mechanism that would explain at least some lost connection problems during the competition. Fortunately this is easy to work around; it is simply necessary to have COMPMNGR ping the PDAs at an interval somewhat less than five minutes.
It has been speculated that electromagnetic interference from other systems (e.g. cordless phones, cell phones, other wireless networks) causes loss of connection. In this test the wireless router was equipped with a signal booster in an attempt to overpower such interference. As best we can tell, this made no difference, so it is not clear whether electromagnetic interference is a factor in lost connections.
In a number of cases it was clear that loss of connection was due to judge error, for example, pressing the off button (turning the PDA off) or the record (voice recorder) button. In other cases it appeared that a judge tapped the wrong on-screen button and entered a program area reserved for the technician.
For the first hour or so of the competition the scrutineer was using a computer with an old version of Windows XP, one which had not been upgraded through service pack 2, and there were problems with COMPMNGR crashing as well as with lost connections. Switching to another computer reduced the problems, although COMPMNGR did crash twice more during the weekend.
There are other possible causes of lost connections. I have spent over forty years programming computers and am well aware that there is always a possibility of bugs in my source code. There could also be bugs in the Windows implementation of the sockets protocol or other Windows system services. And there could be problems with the wireless network setup or hardware. Presumably such problems would show up in "laboratory" testing The most I can say about these possibilities is that there has been extensive testing to determine.
I plan to make a few more modifications in both the HandheldMarker program and the MarkerServer module. The most important of these is to add a periodic timed "ping" to keep the PDAs connected during periods of inactivity. The other modifications are to make the system easier to use. I would prefer not to spend any more entire weekends testing the system at dance competitions; instead I hope to find a few competitions at which I can selectively test during evening sessions similar to the ones that have given the worst problems in the past test.
As I said in the summary, I don't intend to provide marking system hardware or support services. I am hoping others will do that. There are two ways a person interested in developing the hardware portion of a COMPMNGR compatible marking system could proceed. The first is to use the HandheldMarker program and the MarkerServer module I have developed. The second is to write his/her own programs to perform the functions handled by the HandheldMarker program and the MarkerServer module, namely handling the 'OUT' and 'INP' files required by the COMPMNGR interface. Obviously in either approach it will be necessary to acquire the hardware components. I would suggest starting small, say with only one PDA just in order to get the software working with the hardware. I will be happy to work with anyone developing a COMPMNGR compatible marking system, even to the point of providing relevant sections of my source code. Also, interested persons are welcome to work with me at any competition which allows me to use my test system.