Figure 1: Three-level architecture
This document explains what software you need to run AntWorld; where to download OAWS from, how to install and configure it, and how to recompile it if you make changes to the source code.
This document described installation process for the version of AntWorld updated in March 2002. As of this version, the installation process is designed primarily for use with a web server or servlet engine using file layout conforming to the Web Application format specified by in Java Servlet API Specification 2.2, such as Jakarta Tomcat. Previous versions of AntWorld have been designed for use with more old fashioned web servers, such Sun's Java Web Server 1.0.3, or Apache with an early version of JServ. You can find the deployment instructions for older versions of AntWorld here.
During an AntWorld session, AntWorld software in three possibly different places will be used:
Organizational Level
Each OAWS servlet has its own function: creating user accounts, initiating quests, recording user's judgments, etc. In the past, one of the OAWS servlets, known as the Organizational Ant Proxy Server (OAPS) has a special role: all user's requests were routed by his LAPS through the OAPS in order for some guiding information (ant icons) to be inserted. The OAPS was also used to record all the URLs visited by the user. Now this feature is obsolete, and it won't be discussed in further details. Normally, LAPS does not need to forward page requests thru OAPS anymore; instead, it uses other servlets in a more conventional way.
AWS Level
The AntWorld Database Server runs with Sybase SQL server, which as far as I know, can be obtained for free only for Linux platform. But if your site already has bought a Sybase SQL server and runs it on any machine on your local network, then you probably can also use these instructions to set up Organizational AntWorld Server on any UNIX machine. I tested them on Solaris 5.8. Using the installation scripts on this UNIX machine, you will also set up the the AntWorld Database Server on the computer that runs the SQL server; this may be the same machine on which the Organizational AntWorld Server runs, or a different machine, only accessible via isql, bcp, and JDBC.
If you desire to install AntWorld servers on a Microsoft Windows machine, you may encounter some problems, since the distribution includes not only Java code and Perl scripts, but also some C shell scripts. You may have to use your Microsoft Windows expertise and re-write them as MS Windows "batch files"; or you can install Cygnus Tools, which come with UNIX-like bash shell, and re-write scripts into bash. In any case, it won't be a particularly pleasing experience.
If you are using a servlet engine such as Tomcat or JRun, it is often considered advantageous, although not absolutely necessary, to use a web server such as Apache as an intermediary. Instead of having the servlet engine (Tomcat) directly listening to the requests from the internet, you can set up Apache to receive requests and to forward them to the servlet engine. This guide does not cover this technique; refer to the manuals for yoru servlet engine for instructions.
If you don't have a servlet engine (such as Jakarta Tomcat) and Jakarta Ant, download them from the following sites, and install them in the standard way:
As of March 2002, Sybase makes Adaptive Server Enterprise 11.9.2 for Linux available for free download. This is the version this guide has been tested with. If you run Linux, and don't have a Sybase SQL server yet, you can download ASE 11.9.2 for Linux from linux.sybase.com, and install it in the standard way, according to the instructions suppplied. The instructions will tell you which RPM files you need to download (3 packages are mandatory:
sybase-ase-11.9.2-3.i386.rpm sybase-common-11.9.2-3.i386.rpm sybase-doc-11.9.2-1.i386.rpm), and how to use "rpm" to install them in /opt on your machine.
Once you have installed the doc package, you will have lots of documents, in PDF format, in /opt/sybase-11.9.2/doc/ . Read "CONTENTS" first.
Same Sybase site also offers ASE 11.0.3.3 for free download. The difference from 11.9.2 is that 11.0.3.3 is offered "for development as well as deployment", while, 11.9.2. is only offered for "development". Other versions of Sybase SQL server, for Linux, Solaris, and other platforms, can be purchased from Sybase. For example, at the Rutgers site, we are using Sybase's 1997-vintage Adaptive Server Enterprise, release 11.5.
Once signtool is installed, make sure that it's in your PATH. You can achieve this, for example, by setting up a soft link:
ln -s /opt/signtool/signtool /usr/local/bin/signtool
unzip -r ant.zip
This will create the directory code, with a number of subdirectories. They contain the source code for LAPS and OAWS, scripts, stored procedures, and data for ADBS, as well as various installation scripts. They will be discussed later on.
When you installed the Sybase SQL server (ASE 11.9.2, or any other server) using the srvbuild program, you had to supply the "server name" for the "Adaptive Server", or agree to the default name suggested to you. Typically, the name is the same as the internet of the machine where the server is installed (which would be localhost, if the machine is not networked); sometimes it is "SYBASE". If you are not sure what the server name is, you can find it in the file /opt/sybase/interfaces. You need to know what the name is, because you have to supply it to programs such as isql in order to connect to the SQL server.
These instructions assume that when the Sybase SQL server was installed, its main directory (which is also the home directory of the UNIX user sybase) was made available as /opt/sybase. If this is not the case, you can set up an appropriate soft link, e.g. by typing, as root,
cd /opt ln -s sybase-11.9.2 sybaseAlternatively, you can change the file names in init0.sh and init1.sql to match your system layout.
Unless you have created large database devices already, you should decide now where the SQL server will store its data. If you want to follow our example and store them under /opt/sybase, you can start with creating a few subdirectories, by running the script code/util/init0.sh as user sybase. (Or you can run it as root, and then transfer ownership to to sybase with
chown sybase /opt/sybase/dev /opt/sybase/dump)
Now, you need to create the database antdb; the database devices where this database will be stored; and the users antserv and antoper that will access this database. The commands to do all that may be found in one of the scripts whose names match code/util/init1*.sql. I suggest using init1_linux.sql; copy it to init1.sql before customizing and using it. You should first edit your file init1.sql to replace "oawspwd" everywhere with whatever password you want to give to the AntWorld database accounts. You may also change the sizes allocated to database devices, if the sizes given in this file are too large or too small for you; however, if you do so and change a disk init for a device, don't forget to change all create database or alter database statements that use this device! (And don't get confused with various space units that Sybase uses in different contexts).
Once you have customized init1.sql as desired, you can feed the entire file to the SQL server using /opt/sybase/bin/isql:
cd code/util/ emacs init1.sql # Customize allocation commands and passwords as needed /opt/sybase/bin/isql -U sa -P saPassword -S serverName -i init1.sqlOr you can run isql interactively:
/opt/sybase/bin/isql -U sa -P saPassword -S serverNameand feed the SQL commands to isql one by one, detecting any errors as they happen. The latter approach may be safer.
Before you started working with certificates, you need to make sure that you have set a password for protecting certificates in your Netscape browser. This can be done by going to Security | Passwords.
Now, you actually need to get a Netscape code-signing certificate (if you don't have one already). There are several ways to obtain such a certificate, as explained below.
signtool -G Certificate_Namewhere Certificate_Name is the name of the new certifcate to be created.
To prevent conflicts, you must not be running the Netscape web browser at the moment when you run signtool -G, since the Signing Tool will modify your ~/.netscape/cert7.db.
Write down the name you have given to your new certificate; you will need to later put it into your AntWorld build file. To check whether it has been successfully created and now sits in your cert7.db, you can run
signtool -l
Note for end users: if the applet and JavaScript code at your server have been signed with a "self-signed" object-signing certificate, this means that normally a web browser won't recognize it as a valid certificate, unless you are running the web browser right there on the server, in the user account in which the certificate was created. Therefore, in order to be able to use AntWorld from a web browser anywhere else, you must, before starting your Netscape browser, copy the 3 files from the .netscape directory of the user who did the installation process (those are key3.db, cert7.db, and secmodule.db) to your personal Netscape directory. The user's personal Netscape directory is ~/.netscape on UNIX; in MS Windows, it is something fairly deep under Program Files\Netscape in MS Windows -- if in doubt, find first where your existing key3.db is.
A line defining a parameter has the form
parameterName = parameterValuewith an optional semicolon after the value. Depending on the parameter, its value may be interpreted as a number or a string. If the value is a string, it may (and --when confusion is possible-- must) be quoted using double quotes, e.g.
ServletPrefix = "http://myhost.mydomain.com/antworld/servlet/"
DbDriver = com.sybase.jdbc2.jdbc.SybDriverOlder versions of jConnect used different driver class names, e.g. com.sybase.jdbc.SybDriver
DbUrl = jdbc:sybase:Tds:aplab:4100/If you are running the SQL server on a different machine from the one where the OAWS is running, you should modify the DbUrl parameter appropriately. For example, in Rutgers, we run the server on the computer named "aplab", so the value of DbUrl there is DbUrl = jdbc:sybase:Tds:aplab:4100/
DbServer = localhost
DbLogin = antserv DbPassword = oawspwd
Note that the configuration file does not contain any parameter that stores the name of the Sybase database used by AntWorld (antdb). We don't need to store it because it is the default database of the user antserv.
ServletPrefix = /antworld/servlet/ AuxPrefix = /antworld/aux/If desired, you can specify the host name and port number explicitly, e.g.
ServletPrefix = http://my.host.domain:8080/antworld/servlet/ AuxPrefix = http://my.host.domain:8080/antworld/aux/
OapsHost = aplab.rutgers.edu OapsPort = 6060
# For 2 megabytes: CacheKBSize = 2000
Note that to use this option, you must make sure that the JVM that runs your servlets is allowed to allocate sufficient memory for the heap. For example, in JWS 1.0.3, the web server is started (in /etc/rc3.d/S42http, or wherever you've copied your starting script) by a java command with no -mx option, which means that the Java heap size is limited to the default 16 megabytes. I would advise you to modify the JWS startup script to use at least -mx24m (24 megabytes). Increase this amount accordingly if you are using a large page cache (as specified by the CacheKBSize option in your ant.cfg.
CacheFileMaxLife = 60 #60 minutes
| Property Name | Default Value | Meaning |
|---|---|---|
| tomcat.home | /usr/local/tomcat | The main directory of Jakarta Tomcat. If you have the shell environment variable TOMCAT_HOME set correctly, you can remove the tomcat.home property from build.xml entirely, and Jakarta Ant will use the environment variable instead. |
| tomcat.lib |
${tomcat.home}/lib (for Tomcat 3.*), or ${tomcat.home}/common/lib (for Tomcat 4.*) |
The directory, under ${tomcat.home}, which contains the standard JAR files coming with your servlet engine, such as servlet.jar. |
| tmp | /tmp | The directory which will be used for temporary files during the installation process. |
| app.name | antworld |
The application name. It affects the location of the application files
under ${tomcat.home}/webapps. There is little reason to change it; one
situation when you may want to do it is when you want to run several
versions of AntWorld on the same machine. If you decide to change it,
it will change the URL under which AntWorld is accessed (the default
is http://you.host/antworld/), and you will have to change the parameters
ServletPrefix and AuxPrefix in ant.conf
accordingly. Note: During the installation process, Jakarta Ant will be copying files into this directory (${tomcat.home}/webapps/${app.name}). If it does not exist, it will create it first. Therefore, you may want to check whether the existing file permissions on ${tomcat.home}/webapps/${app.name} or ${tomcat.home}/webapps will allow it to do it. If, for example, the webapps directory is owned by root, but you are going to run the installation process as user antinstall, you may have to first login as root, create subdirectory webapps/antworld, and transfer its ownership to user antinstall. |
| dist.down | ${tomcat.home}/webapps/ant/download | The download directory. This is the place where AntWorld deployment process will place some files that end users may want to download, such as antscape.zip or antworld.tar.gz, which teh end user may download to set up LAPS on his workstation. This directory is not used during the normal operarion of AntWorld. |
| sign.cert | n/a | The name of the Netscpae code-signing certificate that you will use to sign code with Netscape Signing Tool. See Preparing your Netscape code-signing certificate, above. |
| sign.pass | n/a | The password that you have set for protecting certificates in your Netscape browser. |
| netscape.classpath | /usr/lib/netscape/java/classes/java40.jar | The location of a JAR file that comes with the Netscape browser and contains Netscape LiveConnect classes. It is used to compile the AntWorld console applet. The location depends on where Netscape is installed on your machine. |
| build.compiler | n/a | Tells Jakarta Ant what Java compiler to use. I have set the value of this property to "classic" in my build.xml, for greater compatibility between the applet byte code produced during the compilation and the older Java Virtual Machines that may occur in users' browsers. (I am not even sure whether it is this property, or the attribute target="1.1" that I have in the javac command, or both, that are necessary to ensure that the applets run in you garden variety Netscape 4.7, but I suggest that you keep both!). |
| netscape.classpath | /usr/lib/netscape/java/classes/java40.jar | The location of a JAR file that comes with the Netscape browser and contains Netscape LiveConnect classes. It is used to compile the AntWorld console applet. The location depends on where Netscape is installed on your machine. |
antThis will cause build for the default target, sign, which will be preceded by its prerequisites: prepare, and compile.
The LAPS distribution is built separately, using target local. To do it, type
ant localThis will compile the LAPS code and install the distribution files, ready for downloading to the end users' workstations, in ${dist.down}.
To produce Javadoc documentation of the source code, use target javadoc. Type
ant javadocThis will generate Javadoc documentation for public, protected, and package-visbility members of all AntWorld classes, in ${tomcat.home}/antworld/javadoc/
With most servlet engines (including Tomcat and JRun), it is usually safer to always restart the servlet engine after you recompile any class files. In Tomcat, for example, this can be done as follows:
cd $TOMCAT_HOME/bin ./shutdown.sh # wait for a minute to make sure that Tomcat has completely stopped ./startup.shDepending on how Tomcat is run on your machine, you may or may not have to act as root to restart Tomcat.
The AntWorld web application having been deployed, you can now access it in your web browser, by going to
http://your_host.your_domain:8080/antworld/From that page, you can follow the "Start an AntWorld Quest" link, but in so doing, you will only get a runtime error from the AntWorld server, since at this point it still lacks some essential data in the database. If you have run ant javadoc, you can also follows the "Javadoc" link, to browse the Javadoc documentation for the AntWorld source code.
cd code/util ./maketab.sh
The comments in the script explain every operation that it performs. If the script malfunctions and you see error messages (for example, because some configuration file was not found, or the SQL server is not currently running, or the SQL server password stored in ant.conf is incorrect), you may choose to run the commands from the script one by one.
Once the database have been initialized, you can try to run AntWorld! Simply go to
http://your_host.your_domain:8080/antworld/(the URL above assumes that your Tomcat runs on port 8080), and follow the "Start an AntWorld Quest" link. You should get to the AntWorld welcome screen, where you can register as an AntWorld user, and start your first AntWorld quest.
$ISQL -S localhost -X -U antoper -P oawspwd -i $CMDIR/dailyDump.sql > $CMDIR/dump.out
cp -a dumpScripts /opt/sybase
cd /opt/sybase/dumpScripts dailyDump.shYou will see a profusion of SQL server messages, telling you about souccessful database backup (or otherwise).
0 3 * * * /opt/sybase/dumpScripts/dailyDump.sh >> dumpScripts/out.txt 2>>dumpScripts/err.txt(Note that the command in crontab are in the sh syntax).
If you don't want to back up the database at all, but only need regular log truncation, you can edit dailyDump.sql to remove dump database commands and leave only dump tran.
0 1 * * * /some-path/code/util/crawl.sh > /some-path/code/util/crawl.log 2> /some-path/code/util/crawl.errYou have to specify the correct path, of course.
As with dailyDump.sh, you may want to test the script before scheduling it.
The crawler script may stall because of the database log been full. This can be detected by reading the file /opt/sybase/install/server_name.log, and noticing lines like this:
2 task(s) are sleeping waiting for space to become available in the log segment for database antdbAlternatively, you can log into the SQL server using isql, and issue the command sp_who, which shows the list of currently running SQL server processes. If it shows processes with the status of LOG SUSPEND, it indicates the same problem. The solution is to truncate the database log at once; this can be done via isql by using the same truncate command as in dailyDump.sql, namely
dump tran antdb with truncate_only
cd code/util importQ.sh --input dump.tar.gzThe dump file contains only page URLs, but not their actual content; therefore, the importQ.sh script, once it imports the data into the SQL server, will start indexing programs It will import all quest info, and will then start indexing programs (MakeInst, MakePsi), which may take several hours on this amount of data. It is OK to interrupt them (you may have to do it anyway, if they stall because of running out of log space); crawl.sh can be used later on to finish indexing of those pages.
Or write to me (Vladimir Menkov) at vmenkov@aplab.rutgers.edu; I will try to respond to your questions by improving this guide.