Tuque – An
Introduction
Last Updated February 25, 2011
The ability to push new and updated software, and the ability to remove it later, has been a cornerstone in the success of the Nexus environment.
Using GPOs (Group Policy Objects, the usual means of software distribution in an Active Directory environment), we have been able to keep workstation software updated to common version levels. As a result, users can move between machines and maintain file interoperability, and data files are compatible between different users. Also user training is maintained for the single common version, and security releases can be dispatched efficiently.
Users save time and frustration by not having to maintain their own software. Likewise, IT staff can operate more efficiently because they spend less time on individual machines, installing and fixing software.
In April 2010, Nexus had 2,146 GPOs. That number is misleading; many GPOs are duplicated to share the installation workload over multiple servers. However, there are over 1,150 distinct MSI based applications distributed on Nexus.
We have a great deal of experience distributing packages by GPO. Most of the Nexus workstations have either all or most of their software installed automatically.
We have also encountered the limitations of GPO based installations. Some packages are too big for MSIs on which GPOs depend. Sometimes software has to be installed or uninstalled in a certain order, this is hard to configure with GPOs. We have experienced timeouts and the resulting failed installations. GPOs do not adequately track their results. Some software installs better as an EXE file than an MSI (Microsoft Installation Package), and GPOs do not handle EXEs well. The way the files are read from CIFS servers (file servers) is also inefficient and scales expensively and awkwardly.
A particularly annoying problem from the user perspective is that GPOs will install when a computer reboots – often taking significant time while the user might be waiting. It is not possible to schedule the installation to a time more considerate of the user.
Finally, for office users we would like a self-service model where specified users can install software on demand. Rather than walking to the CHIP help desk to purchase software, a user could click on a web site which automatically installs the software in a preapproved method.
There are many products available which can manage software installations. Each has benefits and shortcomings.
Microsoft System Centre appeared to be a strong candidate. The Engineering Computing staff members were initially enthusiastic about System Centre and its features. However, we have since become disillusioned with it.
Possibly one of the greatest criticisms is the security risk it represents because it is designed to run with privileges extending to many workstations, all while running millions of lines of code that have had frequent security bulletins. In the System Center model, code on the potentially compromised server is all that keeps out the bad guys. Soon we will explain how we can do better than that.
As reported in the Active Directory consolidation report, we should be wary of systems that depend on a high degree of privileges. A compromise through any of these vectors would gain total command of the Nexus environment. In addition to granting enterprise privileges to the hacker, Systems Center can be used to run any desired software on the client.
Some functions of Systems Center are already being tracked successfully through campus systems such as ONA (network asset management), Audit (Windows asset management), and software metering is already done on Nexus. In many of these cases, our local implementations have more features and may be preferable to the Systems Center alternative. Also, these systems can be combined in new powerful ways, such as how ONA makes use of the Nexus login data. This is synergy.
System Center, like many other Microsoft products, has synergy too; it is positioned to sell more products. Laptops and netbooks, for example, must be upgraded to a higher version of Windows than the manufacturers installed version. Netbooks have very limited disk space, making this is an unreasonable strategy and will render them useless. And while we have purchased campus agreements to allow this, it still makes little sense to go through the hassle of reinstalling all these systems needlessly.
Tuque is a new software distribution system from Engineering Computing. It can install software in almost all the same ways as Systems Center, but it is designed for our needs and is extremely flexible while being significantly more secure.
Tuque can be used to manage the software lifecycle (install, uninstall, upgrade) on all Windows computers, including netbooks and other devices which need not necessarily be in the Nexus domain. The client operating system does not need to be upgraded or replaced, and people who may not wish to join a domain, or devices which are incapable of doing so can all gain the benefits of managed software. Systems Center requires all workstations to be in the domain.
Tuque has the concepts of software packages, workstations, and subscriptions which logically join workstations to software. The current state of each subscription is always known, so we know whether the software install succeeded, failed, is pending or is already in progress. On large systems, managing such knowledge is essential. If a computer or hard disk needs to be replaced, the previous software can be installed identically to a previous state.
The subscriptions can include date and time fields, so a package can be scheduled to be installed at a particular time. This avoids the boot-time installation issues of GPOs. Software can also be given a priority, so packages will always be installed in the preferred order – as is often required for patches which resolve issues in presumably-prior installations.
Subscriptions can have prerequisites which Tuque installs first automatically, and antirequisites which Tuque uninstalls first, the latter are referenced as replacements and are used to upgrade to new versions.
The Tuque management interface is entirely Web driven. Software can even be managed via a cell phone if desired. However, it can also be extended in interesting ways because we can programmatically enhance it – we can add Web scripting for common or difficult tasks, much like ONA (Open Network Administrator) has been customized for our on-campus needs.
Software subscriptions can be applied at domain OU levels or by individual workstations, so one could apply a package subscription to every computer in a faculty or department, or select a handful of computers throughout the domain structure.
Office users and laptop owners can be empowered to use Tuque’s simplified unprivileged web interface to manage software on their computers. This feature permits management capabilities of preapproved software without having the user perform the installation or even know a privileged password.
It would be feasible to add billing to an account number. This could generate an Email to the software licensing person with the subscription information, an account number and a UWuserid used to sign the request.
The subscription allows for many powerful management options. Security updates for software can be automatically delivered to all who need them. Userids owning workstations using software can be identified to determine how popular a given package is, and create a list of people who may be contacted if a new version becomes available.
Tuque reduces the number of servers required compared to GPOs. GPO software installations, when combined with antivirus software scanning, create a lot of small inefficient CIFS requests to the servers to look for viruses in all the files being accessed. These requests slow down the servers and increase the odds of a timeout causing an installation failure. The workaround we use with GPOs is to copy GPOs multiple times, each with a new server name – that’s why there almost double the number of software GPOs compared to applications.
Tuque uses a more efficient bulk HTTP download strategy which can be serviced quicker, and where we can set reasonable limits to the number of connections. If the software cannot be downloaded due to server load, the Tuque client simply retries in a few minutes. Once downloaded, the antivirus software efficiently scans the local copy of the software.
We can perform a barebones operating system install using Sysprep, RIS or Windows 7 Microsoft Deployment Tool (MDT), then layer on the necessary software.
Tuque keeps track of the state of each install and displays that information on web pages. We are able to script any number of changes – since we know and can control all aspects of the software delivery process. We know which machines have particular versions installed and we can make upgrade purchase decisions. With Tuque we are in control.
Tuque
addresses two security issues raised by System Center. First, unlike
Systems Center, a compromise of the Tuque server does not grant domain or
workstation privileges, partly because Tuque is not privileged on the domain or
on workstations. Secondly, compromising the Tuque server does not grant
one license to install malware on clients; the clients all cryptographically
check that the software was digitally signed by a trusted administrator.
That signing is done totally outside the domain’s privilege structure. In other words, the only software which the
Tuque clients will install is software that has been authorized by someone we
trust. A complete compromise to the server has little effect on the
workstations; they will just ignore requests made by the bad guys.
As with any
web server, there is the risk of a hacker changing the web server’s source code
to steal passwords, however the passwords used are not for privileged accounts,
they could not be used to do anything very powerful.
Systems
Center is also a big system requiring significant configuration and
maintenance. It will take many man hours
to evaluate and still more to effectively learn, share, and maintain. It will require consultants to configure, and
training courses to understand. Instead, Tuque is lightweight, capable of
running on a VM using stock apache and mysql already
frequently served on campus and can be backed up with existing mechanisms. Training can be accomplished in an hour. Like ONA, it can be tweaked if people find
any aspect complicated. The main
development work is already done, now are ready to deploy.
Tuque is
elegant (about 2,000 lines of PHP). It
represents a low risk of abandonment because Erick’s software tends to be used
for years, and the packages it describes could be easily moved to a surviving
product.
Probably
the most telling evidence is that Engineering Computing, who have years of
experience managing automated software deployments, are starting to use Tuque.
There are three types of users of Tuque: installation creators (people who lay out software for distribution), system administrators, and power users.
An installation creator creates a software distribution package, these are usually files from a CD or DVD and a script to install them. This person then ZIPs up the files, compressing them to reduce the number of bytes copied to each machine, and then digitally signs the compressed image to indicate himself as the person who vouches for the image.
The creator then copies the ZIP image to a web server and makes an entry on the Tuque server that describes the software (name, details, location of zip file, etc.). This is a complete package now.
From that point on, a system administrator can match the package to machines, selecting install or uninstall. The design of Nexus (group memberships for people, OUs for workstations) indicates which computers this administrator can manage. All the administrator does is make these connections. The Tuque driver on the workstation does the rest.
Power users can also be given access to Tuque. A UWuserid can be associated with machines, indicating that user has privileges to install or uninstall software.
Once an administrator or power user links a workstation to software, the workstation discovers this, usually within a few minutes. It reads the instructions for the next app to install, waiting for the user to logout if so requested. Tuque then downloads the package, reads the cryptographic signature, tests if the creator is currently trusted, and stops if there is any suspicion.
If Tuque is satisfied with the integrity of the package, it unzips the data and runs the install script or uninstall script, as requested. Reboots can be specified, but they wait until the user is logged off.
The web server for the zip files can be a single server or a cluster. The number of concurrent requests can be adjusted using the standard web server options.
Everything is done through standards: zip, http/https, public key infrastructure, LDAP, SQL, etc.. Most of the complex operations are done by time-tested implementations such as Apache, MS Authenticode, InfoZip (open source but part of Windows), etc. They are just arranged in new, clever ways, and the bits are accessible so we can enhance them as desired.