This recipe demonstrates how to keep a single webpage up-to-date using WebJob. Payload and Delivery (PaD) technology will be used to turn the webpage into a self extracting executable. WebJob will then be configured to download, unpack, and execute this PaD file. The motivating factors behind this recipe are: 1) it's desirable to have a way to keep webpages up-to-date, 2) having the ability to automatically detect and repair damaged or corrupted content makes webpages more resilient to defacement, and 3) minimizing user activity on production systems helps to establish well-behaved profiles -- an important factor in identifying abnormal activity. Cooking with this recipe requires that you already have an operational WebJob server. If this is not the case, refer to the instructions provided in the INSTALL file that comes with the source distribution. While this recipe was written for UNIX environments, the concepts should be portable. You need to have at least the following programs available on your production system: diff, chmod, mv, rm, sh, and webjob (1.4.0 or higher). This recipe assumes that for a given client, say client_0001, you wish to periodically inspect a webpage and repair or update it as necessary. 1. Obtain a copy of the webpage you wish to monitor as it would be deployed on the production Web server. To keep things simple, let's assume that your webpage has the content shown here: --- webpage.html --- Welcome!

Welcome to my world!

--- webpage.html --- sed -e '1,/^--- webpage.html ---$/d; /^--- webpage.html ---$/,$d' webjob-pad-update-webpage.txt > webpage.html Next, create webpage.html.pad and copy it to client_0001's commands directory on the WebJob server. pad-make-script --create webpage.html > webpage.html.pad chmod 644 webpage.html.pad scp -p webpage.html.pad you@your.webjob.server.net:/integrity/profiles/client_0001/commands/ The server should now have an integrity tree similar to the one shown here: integrity | - incoming | - logfiles | - profiles | - client_0001 | - baseline | - commands | - webpage.html.pad 2. At this point, the WebJob server has been configured. Now, we'll focus on the production server. Log into the production server, and create upload.cfg as shown below. Be sure to set URLGetURL, URLPutURL, URLUsername, and URLPassword as appropriate. Install this config file in /usr/local/integrity/etc. If necessary, create /usr/local/integrity/run with: mkdir -m 755 -p /usr/local/integrity/run --- upload.cfg --- ClientId=client_0001 URLGetURL=https://your.webjob.server.net/cgi-webjob/nph-webjob.cgi URLPutURL=https://your.webjob.server.net/cgi-webjob/nph-webjob.cgi URLUsername=client_0001 URLPassword=password URLAuthType=basic RunType=snapshot OverwriteExecutable=Y UnlinkOutput=Y UnlinkExecutable=Y GetTimeLimit=0 RunTimeLimit=0 PutTimeLimit=0 URLDownloadLimit=10000000 TempDirectory=/usr/local/integrity/run --- upload.cfg --- sed -e '1,/^--- upload.cfg ---$/d; /^--- upload.cfg ---$/,$d' webjob-pad-update-webpage.txt > upload.cfg install -m 600 -o root -g wheel upload.cfg /usr/local/integrity/etc 3. Next, create update-webpage.sh as shown below, and install it in /usr/local/integrity/bin. This script is actually one big command -- look at the last line. The WebJob portion of this command downloads, and executes webpage.html.pad using upload.cfg as its config file. The PaD portion of this command unpacks the webpage, compares it to the deployed page, and if they are different, updates the deployed page by replacing it. Finally, the run status and any differences are posted back to the WebJob server. Before running this script, be sure to set PAD_FILE, JOB_HOME, and WEB_PAGE as appropriate. --- update-webpage.sh --- #!/bin/sh PAD_FILE=webpage.html.pad JOB_HOME=/usr/local/integrity WEB_PAGE=/usr/local/www/htdocs/webpage.html PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:${JOB_HOME}/bin DODIFF="( echo \"Checking webpage...\" 1>&2 && diff -uP ${WEB_PAGE} ${JOB_HOME}/run/%payload )" UPDATE="( echo \"Updating webpage...\" 1>&2 && mv ${JOB_HOME}/run/%payload ${WEB_PAGE} && chmod 644 ${WEB_PAGE} )" FINISH="( echo \"Cleaning workdir...\" 1>&2 && rm -rf ${JOB_HOME}/run/%payload )" ${JOB_HOME}/bin/webjob -e -f ${JOB_HOME}/etc/upload.cfg ${PAD_FILE} ${DODIFF} \|\| ${UPDATE} \; ${FINISH} --- update-webpage.sh --- sed -e '1,/^--- update-webpage.sh ---$/d; /^--- update-webpage.sh ---$/,$d' webjob-pad-update-webpage.txt > update-webpage.sh install -m 755 -o root -g wheel update-webpage.sh /usr/local/integrity/bin 4. At this point, you are ready to test everything out. Run the update script as follows: /usr/local/integrity/bin/update-webpage.sh If it completes successfully, you should find four files on the WebJob server in its incoming directory. The files listed here will give you an idea of what to look for: client_0001_20021111210945_update-webpage.sh.env client_0001_20021111210945_update-webpage.sh.err client_0001_20021111210945_update-webpage.sh.out client_0001_20021111210945_update-webpage.sh.rdy Inspect these files. Their content should be similar to that shown below. The .rdy file is simply a lock release mechanism. Therefore, it's content has not been listed. --- client_0001_20021111210945_update-webpage.sh.env --- CommandLine=webpage.html.pad ( echo "Checking webpage..." 1>&2 && diff -uP /usr/local/www/htdocs/webpage.html /usr/local/integrity/run/%payload ) || ( echo "Updating webpage..." 1>&2 && mv /usr/local/integrity/run/%payload /usr/local/www/htdocs/webpage.html && chmod 644 /usr/local/www/htdocs/webpage.html ) ; ( echo "Cleaning workdir..." 1>&2 && rm -rf /usr/local/integrity/run/%payload ) JobPid=20094 KidPid=8539 KidStatus=0 KidSignal=0 KidReason=The kid exited cleanly. Hostname=your.production.server.net SystemOS=i686 Linux 2.4.18-grsec-1.9.4 JobEpoch=2002-11-11 18:08:06 PST (1037066886.700503) GetEpoch=2002-11-11 18:08:06 PST (1037066886.980652) RunEpoch=2002-11-11 18:08:07 PST (1037066887.625140) PutEpoch=2002-11-11 18:08:08 PST (1037066888.640114) --- client_0001_20021111210945_update-webpage.sh.env --- --- client_0001_20021111210945_update-webpage.sh.err --- Extracting payload... Delivering payload... ( echo "Checking webpage..." 1>&2 && diff -uP /usr/local/www/htdocs/webpage.html /usr/local/integrity/run/webpage.html ) || ( echo "Updating webpage..." 1>&2 && mv /usr/local/integrity/run/webpage.html /usr/local/www/htdocs/webpage.html && chmod 644 /usr/local/www/htdocs/webpage.html ) ; ( echo "Cleaning workdir..." 1>&2 && rm -rf /usr/local/integrity/run/webpage.html ) Checking webpage... Updating webpage... Cleaning workdir... DeliveryStatus='0' --- client_0001_20021111210945_update-webpage.sh.err --- Note that the following output implies that the file webpage.html was not deployed on the production server prior to the invocation of the update script. This should lead you to conclude that the update script can also be used to deploy new content. --- client_0001_20021111210945_update-webpage.sh.out --- --- /usr/local/www/htdocs/webpage.html Wed Dec 31 16:00:00 1969 +++ /usr/local/integrity/run/webpage.html Mon Nov 11 18:08:08 2002 @@ -0,0 +1,8 @@ + +Welcome! + +
+

Welcome to my world!

+
+ + --- client_0001_20021111210945_update-webpage.sh.out --- 5. Once you are satisfied that all is well, add the following cron job to the production server's crontab -- it runs every five minutes. --- crontab.5min --- */5 * * * * /usr/local/integrity/bin/update-webpage.sh > /dev/null 2>&1 --- crontab.5min --- From this point forward, the webpage will be automatically repaired if the deployed content differs from the PaD content for any reason. When you want to update the webpage, simply put a new PaD file on the WebJob server. As an added benefit, the diff output preserved in the WebJob upload can be used as a negative or reverse patch. Applying this patch to the page extracted from the PaD file allows you to reconstruct the original webpage as deployed. Thus, there is an inherit quarantining property built into this scheme.