By Erick Engelke
October 1, 2007
Updated October 30,2008
This document gives an introduction to emerge, a campus-wide
emergency notification system.
Background
Unfortunate events at other universities have led us to
investigate solutions to quickly disseminate emergency information, with the
need to have an implementation before such a need arises locally.
Emergencies come in different sizes: such as a chemical leak
which should lead to evacuation of a building or wing, to a suspected hostage
taking where everyone should vacate the campus.
Fire alarms are a blunt notification system. They are
effective at notifying most people and effecting evacuation of one or more buildings.
However, people are not informed of the nature of the disaster, and they will
often wait outside or go to another building. Neither result would be
useful if our goal is to encourage people to leave the area.
Our goal, as a campus, is to spread the information using as
many channels as possible. The focus of this document is one method:
electronic notification via computer networks.
Computer Networks
There are approximately 17,000 computers attached to the
campus computing network excluding the residences and wireless laptops.
|
Environment |
Number of Computers |
|
ADS |
1,689 |
|
Nexus |
4,300 |
|
Artsfaculty |
350 |
|
Xhier |
457 |
|
Unmanaged |
11,000 |
|
Wireless |
up to 2,816 online |
|
Residences |
up to 6,000 |
A technology which would reach all of the managed
environments (ADS, Nexus, Artsfaculty and Xhier) would amass at least 6,400 screens.
Since these systems all have central management, configuring them to install
the appropriate messaging software is easily achieved.
There are approximately 11,000 unmanaged computers on
campus. We could encourage their owners to install the same software, but
we have no guarantees it will be done.
The wireless and residence systems are also important, and
we can envision ways of alerting them, but that is beyond the scope of this
document.
So we return to just the managed computers for now.
Ideally, our system would have the ability to send text,
graphics and hyperlinks to additional information.
There are a number of technologies with messaging
capabilities, but most have unacceptable limitations. Our software must:
- not be impeded by firewalls or routers
- not require an additional login by the user
- not allow SPAMming of unwanted messages by anyone in our community or outside
- support all major desktop platforms (Windows, Linux, Mac)
- be
scalable enough to support at least 6,400 computers and grow as needed
- be very fast – we need to notify people very quickly for
their safety
- be
very reliable, the system should resume automatically after problems like power
outages, network issues, etc., and must support redundancy in case of hardware
failures.
It would also be nice if the system would be versatile
enough to support messaging to subsets of the campus population, such as to
announce the need to vacate a particular building, or to warn of an imminent
power outage.
Emerge: A Notification System
I designed emerge, a system which meets these goals.
Client computers are issued a program which is active as long
as users are logged in. These clients connect to the central emerge
server and maintain a heartbeat so both sides know if communications become
disrupted. The connection is reestablished automatically after power
outages, network outages or other disruptions.
The server has a web interface for campus officials to issue
warning messages. After entering a UWuserid and password, anyone can send a test message to
themselves to verify everything works, or selected people are easily able to
send messages to any subset of the campus. Computers are organized by IP
subnet, building/room location (from DNS), similar DNS names, and active
directory organizational units. A few
special accounts can message the campus as a whole.
The message takes the form of a web page which pops up on
the user’s computer almost instantly.
The Messages
The web page delivered to clients can contain graphics,
text, and hyperlinks to more information.
Each page should be tidy and the message clear. Also,
a date and time should be clearly displayed, because the message may not be
noticed by some users until after the emergency has passed.
A small supply of possible web page layouts should be
designed earlier and thus an appropriate one will be ready to be branded with
the particular message when an emergency arises.
The browser supports HTML 3.2 and limited CSS. Prior
creation will allow one time to mesh the desired look and feel with the
necessary experimentation with the browser to reach desired results. A
sample web page with the UW look and feel is http://globe.uwaterloo.ca/php2
.
The web pages must be served from a suitably powerful web
server, because large numbers of emerge clients will be connecting very
quickly, and a less powerful server could easily be overwhelmed.
Emerge Server
The emerge system needs a central server running some form
of Unix, Apache, PHP5 and PHP extensions including sessions and MySQL. It will have the DNS CNAME:
emerge.uwaterloo.ca
A backup server should exist with the same software
installed and the DNS CNAME: emerge2.uwaterloo.ca If
the main server goes down, the backup server will take over all clients, the
only question is how quickly they will fail over.
If the primary server does an orderly shutdown and closes
its network sockets, the failover to the secondary machine will happen in 10
minutes, assuming the primary machine doesn’t restart (the primary is attempted
first a few times).
If the primary machine crashes without an orderly shutdown,
the average failover time is 30 minutes for existing sessions (maximum of one
hour), and 10 minutes for new sessions.
The development server was a 1.5 Ghz Pentium 4 with 512 MB RAM running FreeBSD
6.2. For each client computer, there is an agent running on the server
with an RSS of 908 kB. All the clients are
usually waiting in calls to select().
There are only four exceptions where the agent is busy:
- receiving
a new connection, doing the DNS lookup and files creation
- closing
a connection and teardown of files
- transmitting
emergency messages
- processing
a heartbeat which occurs every 50 minutes
The emerge server could be converted to another unix platform if desired. It
really is a question of whether IST will maintain another FreeBSD server.
The platform dependant code is limited to two C programs. In addition to
BSD socket functions, the system uses named AF_UNIX SOCK_DGRAM messaging to
communicate internally.