Embedded Web Servers For Distributed Management

Note: this white paper was originally written in early 1998 by John Ousterhout while he was at Sun. At that time we were experimenting with TclHttpd and Scotty to create a web-based interface to SNMP-managed devices. Since that time, TclHttpd was released as open source and has been used for a variety of embedded applications.

Over the past few years the World Wide Web has revolutionized the way that people access and view information. Web browsers such as Netscape Navigator and Microsoft Internet Explorer provide a simple and consistent way to access information in a variety of formats provided by millions of people all over the world. Links embedded in Web pages provide a powerful mechanism for connecting related information, so that users can browse from topic to related topic at their convenience.

In this white paper we describe how the Web can be used for management applications by embedding Web servers directly into devices or services to be managed. This approach provides easy remote access, simplifies the development of graphical user interfaces, and makes it easier to integrate management tools with other applications. We have developed an embeddable Web server based on the Tcl scripting language that is ideal for these applications, along with a client-side browser plugin module to enable enhanced user interfaces.

Problems with current approaches to management

Building software to manage devices and services has proven to be difficult and expensive. Whether managing an embedded device such as a network switch, energy control system, or manufacturing robot, or managing a server application such as a transaction processing system, management tools must address several complex issues.

First, the management tools must provide remote access to the device or service. Typically a single management tool will be used by many people to control many devices or services. Ideally, each user should have complete access to all the devices or services from their desktop. In the past, this has required the development of custom network protocols. Each manufacturer uses its own protocols, so it is difficult to integrate the management tools from different manufacturers.

Second, management tools require graphical user interfaces (GUIs) to simplify their use. Unfortunately, GUI development has traditionally been very expensive and time-consuming, so management tools often have weak interfaces. Furthermore, the APIs used to develop GUIs are platform-specific, so management tools developed for Windows cannot be used on Unix platforms or vice versa; sometimes the tools are not even portable among different flavors of Unix.

Third, management tools need to be flexible. It should be easy to extend management tools to support new devices or services, and it should be easy to integrate management tools with other applications. For example, management tools for network switches should be able to communicate with a database system that stores information about the network configuration, and management tools for energy control systems should have easy access to weather information for predicting future demand. Unfortunately, many of today's tools are not very flexible: they are based on compiled languages and distributed in binary form, which makes it hard for customers to extend the tools and integrate them with other applications. Each manufacturer uses different APIs and GUIs, which makes it hard for different management tools to work together.

The Web solution

The Web offers simple solutions to many of the management problems described above. It uses a single protocol, HTTP, for communicating between browsers and the Web servers that store Web pages. Thus browsers and servers from different manufacturers can easily work together, just by supporting the standard HTTP protocol. Furthermore, the HTTP protocol allows browsers not only to fetch information for display, but also to invoke actions on the Web servers using "post" requests that include parameters.

The Web also provides a standard approach to graphical user interfaces based on the HTML document format. Browsers are available for virtually all platforms, so GUIs based on HTML are highly portable.

Lastly, the Web makes it easy to integrate diverse applications and information sources. A Web page from one source can refer to other pages using links (Uniform Resource Locators, or URLs) and can also invoke actions on various servers.

Embedded Web servers

Many device and service vendors have recognized the opportunity offered by the Web and are looking for ways to replace their existing management tools with Web-based tools. In this approach, Web browsers provide a universal interface for management applications. They communicate with the devices and services using the same HTTP protocol used for normal Web accesses.

In order for standard Web browsers to communicate with devices and servers, each device or service must contain an embedded Web server. The embedded Web server runs as part of the device or service, implements the server side of the HTTP protocol, and maps URL references to procedure calls in its associated device or service. To query the status of the device or service, a user might click on a link that refers to a page hosted by the embedded Web server. When the server receives the request, it invokes a procedure associated with the URL, which synthesizes a page containing the requested information. Different URLs correspond to different kinds of information. For example, a network switch might offer one URL that returns information about recent traffic patterns through the switch and another URL that returns configuration information about the switch's ports.

To control a device or service, a user can fill out an HTML form and submit it to the embedded server. For example, to disable a port on a network switch, a manager might fill out a form that contains the name of the port. When the form is submitted, it will be posted to a URL in the embedded server and the information provided by the manager will be included as parameters for the post request. When the server receives the post request, it invokes a procedure associated with the request, passing it the parameter from the request that indicates the name of the port. The procedure uses the parameter to disable the particular port.

In order to use the Web for management a new kind of Web server is needed. Such a Web server must have the following properties:

It must be portable and modular. It must run on a variety of platforms with minimal support from the underlying operating system and must be able to execute in an existing application such as a database system.
It must integrate with the device or application in which it is embedded. For example, it must be easy to associate URLs with particular procedures in the device or application, so that a fetch or post of the URL causes the procedure to be invoked.
It must be customizable, so that its functions can easily be tailored to the needs of a particular management application. It must be possible to customize and upgrade the Web server remotely, ideally without restarting the associated device or service.
It must have a small memory footprint, so that it can be used in inexpensive devices.

Existing Web servers fail to meet most of these criteria. They are designed as large stand-alone applications that are not intended to be embedded in other applications or devices. They typically have large memory footprints (many megabytes) and place significant demands on the underlying operating system. Many servers are tailored for particular operating systems and cannot easily be used in other environments. Most servers are difficult to customize, requiring large configuration files that provide relatively little power. Finally, most commercially-available servers have high per-unit prices that make it impractical to deploy them in large volumes.

The Tcl solution

We have developed a new kind of Web server that is ideally suited for embedded applications. The server is implemented as a script in the Tcl scripting language, and it uses the existing Tcl mechanisms for customization and integration. The resulting server has many attractive properties:

Tcl has already been ported to a variety of platforms including PCs, Macintoshes, and virtually all flavors of Unix, as well as several embedded operating systems. The server runs immediately on any platform that already supports Tcl. Tcl requires very little support from the underlying operating system (e.g. it does not require a threads facility) and it has a modular I/O system that can easily be retargeted; these factors simplify the task of porting Tcl to new platforms.
The Tcl interpreter was designed from the start to be embedded other applications. It has been used successfully in device controllers, databases, visualization systems, and hundreds of other applications. There are simple interfaces between Tcl and C code that make it easy to pass information between the Tcl interpreter and its enclosing application.
Tcl is a highly dynamic language that makes it easy to customize and extend applications. For example, it is easy to implement customized logging facilities in the Tcl Web server.
Tcl allows new code to be downloaded into an application on the fly to change its behavior. For example, in management applications this feature can be used to change policies or performance monitors without restarting the device or application. It is even possible to upgrade the Tcl server software on the fly.
Tcl contains a powerful and flexible security model called Safe-Tcl, which can be used to restrict the kinds of operations that can be performed on a device or application.
The Tcl Web server is surprisingly fast. Tcl contains a sophisticated high-speed event-driven I/O system that allows the Web server to handle dozens of requests per second, which is more than enough for management applications.
The Tcl Web server has a modest memory footprint that allows it to be used in a variety of devices and applications.
The Tcl Web server is fully functional: in addition to supporting the full HTML 1.0 protocol, it includes modular packages for CGI scripts, server-side includes, and image maps. These packages can be used if needed or omitted to reduce the footprint. Support for the HTML 1.1 and 1.0 keep alive extensions is now included in the server.

Enhancing client-side GUIs with the Tcl/Tk plugin

Although the Web simplifies many of the issues associated with remote management, it constrains the graphical user interfaces that can be constructed on the client side. HTML supports only simple forms based on textual input and requires server involvement to handle nontrivial user interactions. To get around these limitations we have developed a Tcl/Tk plugin module for Netscape Navigator and Microsoft Internet Explorer. The plugin module allows Tcl/Tk scripts to be incorporated into Web pages and executed in the browser. This permits more intuitive and graphical user interfaces than are possible with HTML alone. For example, with the plugin it is possible to create animations, graphs, and charts. It is also possible to create more interactive user interfaces that respond locally to user input. Tcl and Tk enable exceptionally rapid development of these interfaces, typically 10x faster than with other approaches to GUI development, and Tcl/Tk scripts are portable among all major platforms.

The Tcl/Tk plugin makes it possible to program both the client and server sides with the same language, which provides additional productivity improvements. This also enables interesting interactions between the browser client and the Tcl server where Tcl scripts are sent back and forth between the client and server to invoke complex actions.