WebJob downloads a program or script from a remote WebJob server
and executes it in one unified operation. Any output produced
by the program/script is packaged up and sent to a remote, possibly
different, WebJob server. WebJob is useful because it provides
a mechanism for running known good programs on damaged or potentially
compromised systems. This makes it ideal for remote diagnostics,
incident response, and evidence collection. WebJob also provides
a framework that is conducive to centralized management. Therefore,
it can support and help automate a large number of common
administrative tasks and host-based monitoring scenarios such as
periodic system checks, file updates, integrity monitoring,
patch/package management, and so on.
WebJob Call Flow
This is a high-level view of how a typical WebJob transaction works:
Highlights and Advantages
WebJob was written in C and has been ported to many popular
operating environments such as AIX, Cygwin, FreeBSD, HP-UX, MacOS
X, NetBSD, OpenBSD, Linux, Solaris, and Windows NT/2K.
In incident response and evidence collection scenarios, WebJob
does not need to be "installed" on client machines. In
many cases, it can be run from a floppy, CDROM, or network share.
This means that WebJob can be configured such that it is minimally
invasive to the target system. This is important when trying
to collect evidence of an attack on live systems.
In system management, monitoring, and auditing scenarios where
persistence is required, only a single binary and a few
configuration files actually need to reside on client machines.
Logistically, this can be a big time saver in terms of software
deployment and maintenance.
The tools that actually do what you need to have done
are managed in one location, namely the WebJob server.
Thus, scripts and programs can be kept in a state of continual
readiness. Effectively, this increases your ability to adapt
and respond to unforeseen events.
Client-Server data can be exchanged safely and securely
using SSL encryption and certificate authentication.
All harvested data is aggregated in one location -- the
WebJob server.
WebJob only requires an outbound TCP connection -- typically
on port 443. A WebJob server never initiates communication with
a WebJob client. This eliminates an entire class of network-based
attack vectors.
WebJob does not diminish the client's security posture
because it is strictly a client side application and it runs in
the security context of the user invoking it. In other words,
the WebJob client does not accept inbound requests, and there are
no inherent SUID/SGID issues.
WebJob's GET, RUN, and PUT timers ensure that runaway jobs are
terminated once user-specified time limits have been exceeded.
WebJob scales horizontally. In other words, a single WebJob
server can handle multiple clients, and multiple servers within
a single-tiered framework create additional capacity.
WebJob scales vertically. In other words, WebJob servers
can be configured as clients to create a multi-tiered framework.
WebJob provides the ability to digitally sign and verify jobs
using its built-in DSV support. It also supports GPG signature
verification using a client-side GetHook.
WebJob servers support a number of powerful features such as
config file overrides, triggers, and hooks.
WebJob includes PaD, which allows you to turn config files
and tar balls into jobs.
WebJob does not limit what you can do.
Drawbacks and Issues
Because WebJob is a general purpose tool, it cuts both ways.
In other words, an attacker could use it to infiltrate and execute
malicious tools.
WebJob can't be completely trusted on a compromised host even
when statically compiled -- think kernel patch. The best you
can hope for is to detect a breach before such a patch is put
into effect. This could potentially be done by running host
integrity checks on a frequent basis. By the way, if you suspect
a kernel patch, your only true recourse is to take the system
down and inspect it from another vantage point.
To support batch processing, WebJob stores authentication
credentials on the client system. Therefore, one must take
measures to prevent and/or detect spoofing and replays. This
becomes an issue as soon as the client is compromised.
WebJob can't protect client-server exchanges when used without
encryption and mutual authentication.
WebJob in Action
To date, WebJob has been successfully used to:
Automatically harvest argus, ifconfig, lsof, netstat, ndd, patch, ps, tcpdump, (name your utility), etc. data Automatically update cron tabs, DNS records, password files, snort rules, web sites, (name your application), etc. Automatically update system binaries when their MD5s do not match expected values Conduct massive searches for credit card numbers, social security numbers, and suspect hashes Deploy FreeBSD, Linux, Solaris, and Windows packages Drive GUI-based Windows utilities via AutoIT scripts Harvest evidence and diagnostic information from hundreds (300+) of systems in parallel Harvest system information to perform security audits or compliance verification Implement a Virtual Evidence Locker (VEL) Implement and maintain a Poor Man's Compile Farm (PMCF) Implement and maintain a distributed malware test harness Perform integrity monitoring with FTimes Periodically perform administrative tasks on a 950+ node Content Delivery Network (CDN) and the list goes on and on...
Several of these items have corresponding recipes in the Cookbook.
Check them out here.
WebJob Framework
Here's the layout of a typical, single-tier WebJob framework:
|
