ARES/RACES HTML/JavaScript Membership Database

Philip Karas, KE3FL

If you are joining us after having read the article in Worldradio you can skip down to Figure 1 and pick up from there with the detailed explanations and examples of different database searches. Otherwise, you should start right here.

ARES and RACES have a need for membership databases. I've seen databases maintained in all sorts of programs, from straight ASCII files to spreadsheet programs and full-blown database management systems (DBMS). All have their advantages and disadvantages. I've worked on and developed a number of different DBMS, from simple ASCII files to relational DBMSs such as ORACLE. I designed the database, developed, and wrote a C-based user interface for the National Corrosion Database (NCD-developed in ORACLE and using their QSL-C tools) when I worked for NACE International in conjunction with the NACE-NIST Corrosion Data Center.

I have been rather disappointed with the cumbersome nature and size of RDBMSs and I have come to the conclusion that many databases need not be in such systems. The NCD was not one of those. When I was asked to develop an ARES/RACES Membership Database (ARMD), around 1994, I felt that ease of use and completely free software were the main issues, free software being the most important.

In 1983/4 I developed a search engine and a design to use "standard" ASCII files as database files. When the ASCII file is arranged in a pre-determined way, this search engine could return all, or part of, the data in a database record when given the search criteria. At that time, it was a DOS command-line-based program. I found that in 1994, many people did not like DOS command-line-based programs any longer. So I wrote a C-base menu program to help with the user interface but it never was used very much. While all members of our local ARES/RACES organization received a free copy of the DB and the software, no one, other than myself, really knew the power of, or even how to use the "Pfind" (c) search engine. This was not for want of offering classes on the system either.

When I started learning Perl, Java, and JavaScript in early 2001, I thought that only Perl and Java could be used to host a database and it would have to reside on a server. Needless to say, we did not want our database on a server. It would do no good to be there during an emergency that caused communication lines to fail. Keeping it floppy-disk size (1.4 Meg) was important for distribution and, again, using some kind of free software was important.

While playing with the new level 1 DOM (Document Object Module) and DHTML early in 2002, I made an intuitive leap from being able to read an HTML file's innerHTML to being able to read any HTML file's innerHTML. Once that leap was made, it became a matter of experimentation to come up with the best form of the "HTML" file to be read so that we could use it as a database. Now, I could see a way of using my "older" Pfind technology, re-written in JavaScript and HTML.

The form I've selected works rather well. I've tested it out using our own CARET membership database as well as two other files that contain QST table of contents-type information for 1993 - 1999, and 2000 - present. All files load quickly when they are on a client's computer using one of the "newer" DOM level 1 browsers, IE 5.0, NS 6.0. (This technology can be rather slow when used over the Internet. I once had to wait 15 minutes for the data from the 150K QST database to download to my computer so I could search it. But then, this delay would be present no matter how I wrote the HTML database file, either for use in a JavaScript Pfind system, or as a straight HTML file. The Internet was just very slow the day I was testing.)

The really big advantage of this "new" system is that it uses something everyone knows how to use: an Internet browser. It also uses my old-standard Pfind ASCII database file. While I could have designed a fully-blown database file structure and used classes and methods to operate on the data, I felt that I simply did not have the time for such an endeavor. Also, for the kind of information we want to save and retrieve, the Pfind structure still works very well. Simply because it is almost 20 years from its first use does not mean that using an "old" idea is out. After all, the data has not changed very much. I found that I was able to put in HTML anchors to look up people by call sign and I was also able to put in the HTML code needed to show photos of members, all in a Pfind-type ASCII database file.

I implemented a simple version of the Pfind key-word searching feature and I allow the returning of the found data in thee ways. The program can return the full database record, just the names and address lines, or just the line of interest. All of these returns have theirs uses. (see figure 1)

 


Figure 1
 
The Simple Key-word Search:

By "simple," I mean that the present key-word search is not identical to that implemented in the original Pfind program. While we do not need to understand the difference between the original and the JavaScript implementation, it is important to understand the kind of key-word search that is implemented in the HTML/JavaScript program. In this program, the key-word search takes all words entered and places them into an array of words to be found in the record field. The present implementation does not look for correct order of the words found. It only looks for each word and, if they are all found someplace in the record field, it is considered a match. They do not have to be on the same "line" nor do they have to be in the order entered.

This kind of search has its limitations, but for the most part it will not matter, for this kind of data. An example might help in understanding the limits of this kind of a search. Let's say we want to find all people who have 80 meter CW capability, so we search on "cw 80" and this will match all records that have lines that look like: (see figures 2 - 4)

C: cw 160, 80, 40, 30, 20

But, and here's the catch, it will also match the record that has the following line in it:

I: Interests: I'm 80 years old and love hiking and doing 20 meter cw.

Figure 2
 

Figure 3
 
Figure 4

Neither order of the words nor context are considered, so both lines will cause a match and, in fact, the words do not even have to be on the same line to cause a match. I will simply say that this was not the case in the original Pfind program written in C, but it was far easier to do things this way using JavaScript than to implement the single-line, order-important key-word search used in the original Pfind program.

I thought long and hard before I settled on this type of limited key-word search. I also believe that if I can see no real difficulties I will first try it and see if it works. If it does, I go with it. So, I went with the easy way to save time and I have yet to find a problem, but you should know that the possibility for the return of some "strange" records does exist.


Types of Data Returns:

One can understand returning the full data record; after all, everything is there: Name, address, interests, e-mail address, equipment, etc. (As shown in figures 3 and 4)

Returning just the address lines means we can "cut" and then "paste" these lines into a word-processing program to print out address labels if needed. (see Figure 5)

 

Figure 5

And last, returning just the line of interest can return things like all the e-mail address lines so that one can form a membership e-mail list. (see Figure 6 & 7)


Figure 6
 
Figure 7

Bottom Line:

There are advantages and disadvantages to using this type of "database" system. One advantage, as stated before, is that the client will know how to use the main program since almost everyone has an Internet browser on their computer. I took a poll of the members at a recent club meeting. Of all of those who had and used computers, the vast majority of our membership, I found that all of them had one of the newer DOM level 1 browsers on their computers. In fact, at that time, I was the only one who did not, since I was still using NS 4.75 on my laptop computer. (I have since upgraded to IE 5.50, the best I could do on my slower Windows 95 Pentium.)

The main disadvantage is that every now and then a new version of a browser comes out that is partly brain-dead, meaning that it no longer recognizes an old standard JavaScript method. This has happened recently with NS 6.0 - 6.21 but, fortunately, the present problem was not with innerHTML. This disadvantage will probably only be due to a "bug" in the new implementation of JavaScript and should be able to be fixed with either a patch or an update.

Whether or not the selection of an Internet browser and JavaScript as the basis of our ARES/RACES database is a good decision, remains to be seen.

If you would like to play with a JavaScript Pfind search engine, one is available on my web site to search past QST article titles located at: http://www.qsl.net/ke3fl. It's near the bottom of the page. Look for "qst."

73 de KE3FL

Phil Karras
AEC Carroll County MD
ORS, OES, VE,
Life Member
MA Physics
Sr. Software Consultant
Syntek Systems Corp.

 

Appendix A - Pfind Database file structure:

<!-- CARET DataBase File -->
*<a name="KE3FL"></a>
Philip Karras/KE3FL
3305 Hampton Court
Mt Airy  MD  21771-7201
Z: 21771-7201   Philip Karras/KE3FL
C: Carroll
M: membership good to 4/1/03
I: <a onClick="document.location='ke3fl.jpg';" onMouseOver="Click();" onMouseOut="Base();">
I: <img src="ke3fl.jpg" width=75 height=91 alt="ke3fl.jpg">
I: </a>
S: ------------------------------------------------------------------------
H: 301-831-5073 (145.41, auto: 55)
W: 301-208-6660 X32
@: ke3fl@arrl.net
T: Ticket             Extra, 1992
S: ------------------------------------------------------------------------
A: ARRL Appointments  ORS, OES, AEC, VE
A: ARES Equipment     60 AHr, 30AHr batteries
E: Emergency Antenna  160, 80, 40, 30, 20, 17, 15, 12, 10, 2,    , 445,
E: Emergency Field    160, 80, 40, 30, 20, 17, 15, 12, 10, 2,    , 445,
E: Emergency Power    160, 80, 40, 30, 20, 17, 15, 12, 10, 2,    , 445
E: Emergency Power    14 days any one freq.
O: Other Emergency    First aid, CPR, water safety (certs not current)
O: Other Emergency    German language
C: CW                 160, 80, 40, 30, 20, 17, 15, 12, 10
F: FM                                                      2,    , 445,
S: SSB                160, 80, 40, 30, 20, 17, 15, 12, 10
M: Mobile                      40,     20,             10, 2,    , 445,
P: Packet                                                  2,    , 445,
I: Interests- Building & testing antennas/repair electronics
I: Interests-         (4HT, 4mobile) CB, and (4) FRS Radios
I: Interests- QRP building, have: 20, 30, 40, & 80 meter rigs
S: ------------------------------------------------------------------------
C: Call-UP:
S: ------------------------------------------------------------------------
*<a name="AB3CDE"></a>
Harold McOrt/AB3CDE
124 Binary Way
Westminster MD 21157
Z: 21157   Harold McOrt/AB3CDE
C: Carroll
M: membership good to 4/1/03
S: ------------------------------------------------------------------------
H: 301-123-4567
W: retired
@: ab3cde@arrl.net
T: Ticket             Extra, 1945
S: ------------------------------------------------------------------------
A: ARRL Appointments  ORS, OES, VE
A: ARES Equipment     2 - 30 AHr batteries
E: Emergency Antenna       80, 40,     20,     15,     10,    2,      445 
E: Emergency Field         80, 40,     20,     15,     10,    2,      445 
E: Emergency Power         80, 40,     20,     15,     10,    2,      445 
E: Emergency Power    ? 14 days any one freq.
C: CW                          40, 30, 20,   , 15,     10
F: FM                                                  10,    2,      445,
S: SSB                     80, 40,     20                                 
M: Mobile                                              10,    2,      445,
I: Interests- I'm 80 years old and I enjoy 40 meter cw!
S: ------------------------------------------------------------------------
C: Call-UP:
S: ------------------------------------------------------------------------
*
<!--endOfList...0

empty
*<a name="YourCallSign">
Your Address
Goes Here
etc
Z: (enter Zip-code and Name/call)
C: Carroll
M: membership good to 4/1/0?
S: ------------------------------------------------------------------------
H:   (145.41, auto: XX)
W:
@: # (e-mail address) (***** empty ***** bad *****)
T: Ticket             Novice, Technician, General, Advanced, Extra, year?
S: ------------------------------------------------------------------------
A: ARRL Appointments  ? ORS, OES, AEC, VE
A: ARES Equipment     ? batteries
E: Emergency Antenna  160, 80, 40, 30, 20, 17, 15, 12, 10, 6, 2, 221, 445,
E: Emergency Field    160, 80, 40, 30, 20, 17, 15, 12, 10, 6, 2, 221, 445,
E: Emergency Power    160, 80, 40, 30, 20, 17, 15, 12, 10, 6, 2, 221, 445,
E: Emergency Power    ? 14 days any one freq.
O: Other Emergency    ? First aid, CPR, water safety (certs not current)
O: Other Emergency    ? German language
C: CW                 160, 80, 40, 30, 20, 17, 15, 12, 10, 6, 2, 221, 445,
F: FM                 160, 80, 40, 30, 20, 17, 15, 12, 10, 6, 2, 221, 445,
R: RTTY               160, 80, 40, 30, 20, 17, 15, 12, 10, 6, 2, 221, 445,
S: SSB                160, 80, 40, 30, 20, 17, 15, 12, 10, 6, 2, 221, 445,
M: Mobile             160, 80, 40, 30, 20, 17, 15, 12, 10, 6, 2, 221, 445,
P: Packet             160, 80, 40, 30, 20, 17, 15, 12, 10, 6, 2, 221, 445,
B: BBS                160, 80, 40, 30, 20, 17, 15, 12, 10, 6, 2, 221, 445,
I: Interests-
S: ------------------------------------------------------------------------
C: Call-UP:
S: ------------------------------------------------------------------------
*
S: ========================================================================
S: *******************  New members since January 2001  *******************
S: ========================================================================
* ***end***end*** *
-->

Explaining the Database Structure

More information about a Pfind-database, its structure, and data returns from searches, can be found in the on-line help supplied with the Pfind-database application.

The Pfind free-form ASCII database structure only requires the data be placed in an ASCII file like this:

<!-- CARET DataBase File -->
*
Data Record 1 goes here
*
Data Record 2 goes here
*
Data Record 3 goes here
*
Data Record 4 goes here
*
* ***end***end*** *
-->

You can use the optional command: "endOfList" to "hide" data from the Pfind database searches. In the example above I used:

<!--endOfList...
and 
-->

This hides the database record form. I used for the ARES/RACES data, both from the browser and from the search functions. The <-- starts an HTML hide and the --> ends the HTML hide. The "endOfList" hides everything after it from the Pfind database searching functions, and last, the ...0 was simply used to indicate to me that this was the first time I had tried to hide this data from Pfind.

To build this type of database you only need a text editor. The "free-form" part means that you can place the data in the database any way you want.

If you stick to a structure of some pre-determined kind, the Pfind functions will allow you to do more effective searches; even so, Pfind only allows basic logical AND searches.

In the example I gave of the ARES database, I placed a number of things between the database record delimiters (the asterisks: *). I'll try to explain why I did what I did because there are some special features built in to the Pfind search functions.

First, if you use a colon ":" as the second character of a line, then Pfind recognizes this line as a database field line. In a Pfind type database, each line is a database field and each field is one line long.

In the demo DB I used:

@: for e-mail addresses

P: for phone numbers

C: for the county

E: for some type of emergency equipment or capability, etc.

You can also see that I used E: a number of times so I also put in a larger Ďfield nameÓ as in:

E: Emergency Antenna 160, 80, 40, 30, 20, 17, 15, 12, 10, 6, 2, 221, 445,

E: Emergency Field 160, 80, 40, 30, 20, 17, 15, 12, 10, 6, 2, 221, 445,

E: Emergency Power 160, 80, 40, 30, 20, 17, 15, 12, 10, 6, 2, 221, 445,

In a traditional database, if I do a search for all members with the ability to setup an emergency 40-meter station then I would probably only get people who had saved Ď40Ó in the "emergency field station" database field. This is not the case in a Pfind free-form ASCII database. One reason is that I only allow one letter to be used for the field name. In my example I used " E: Emergency Field" to indicate an Emergency Field operation:

Let's say we have four different members with the following field data:

E: Emergency Field 80, 40,

E: Emergency Field 40, 20, 2

E: Emergency Field 10, 2, 440

and

E: Emergency Field 2, 440 for four different members.

If we now search on "E: field 40", the records for all four of these members will be returned as hits. (A space between things to search for means logical AND. The space is NOT preserved as part of the search text.)

One refinement of the search could be using:

"E: field 40,"

Then, only the first two would be returned. In this case you would ALWAYS have to place a comma after a 40-meter wavelength but not after using 440 MHz to be able to see a difference between "40," and "440".

The reason for this is that the code is designed using the internal JavaScript methods and I decided not to design methods and functions to implement a true field-specific database with all its search capabilities.

The rule of thumb for Pfind searches is simply this: if the items being looked for are on the line (remember, a line is a field), then that line is a match. Case doesn't matter, position doesn't matter, and order doesn't matter.

A search can return record data in three ways:

  1. the matched line (field)
  2. the entire record
  3. the address

The street address is a little different in a Pfind database. Pfind treats all lines as address lines if they do not have a colon as the second character. This means that when we do a "return address lines only" search, we will get all lines that do not have a colon in the second character position as address lines. Also, no delimiters will be placed on the page so you can cut-and-paste the returned addresses into a word processor to be printed on labels if you want.

Another feature of a Pfind database is that everything on a line starting with an asterisk is ignored. Pfind considers that any line with an asterisk as the first character is a data-record delimiter line, not a data (field) line. Since it is used only to separate one data record from another, nothing on this line will be returned as data from any search. So, in my DB example I used this line to hold an HTML anchor for each amateur in the DB. HTML does not show anchors on the screen and Pfind doesn't return delimiter lines so this worked out well. The anchor I used held the person's call sign, all uppercase.

Another feature is that when a database file is read in, all HTML anchors are found and an HTML drop-down list is made of the anchor names. In this example, it was the amateur call-signs.

 

Back to top