High Frequency Server Monitoring
in the Browser using WebSocket
Bergmans Mechatronics LLC
Update: The collectdViewer system is now available at collectdviewer.com
WebSocket is a high-speed, low-overhead communication interface between browsers and servers which has the potential to enable a new generation of dynamic and responsive browser-based applications. This article explores possible applications for WebSocket in the field of Software Development and IT Operations ("Devops"). It was inspired by attending my first Devopsdays conference in Mountain View in June 2011 where I was able to get a sense for the complexity of setting up and maintaining large-scale software systems.
This article presents a simple demonstration of server performance monitoring in the browser using WebSocket. In this demo, performance data is updated in the browser at a rate of once per second, without polling or the use browser plug-ins. The article also describes several concepts that could be implemented to meet the real-world requirements of developers and operators. Finally, the article concludes with a brief discussion of possible future activities which might be pursued to enable the Devops community to benefit from this technology.
Introduction to WebSocket
WebSocket is a standardized interface for continuous, bi-directional and low-overhead communication between Web browser clients and back-end servers. A WebSocket connection has lower latency levels and lower bandwidth requirements than AJAX/Comet and related design patterns. As a result, WebSocket allow developers to readily create dynamic browser-based applications that would be difficult or impractical to implement using the HTTP request-response model.
The advantages of WebSocket are achieved without browser plug-ins in many modern browsers, including the Safari browser for iOS devices, and some WebSocket implementations include an emulation capability for legacy browsers.
Cloud Server Monitoring using WebSocket
The properties of WebSocket make the interface ideally suited for dynamic web applications in which large amounts of time-critical data must be transmitted to and displayed in the browser. A potential application which could require this type of capability is remote monitoring of servers used in large-scale software systems.
To illustrate the feasibility of the use of WebSocket for this type of application, Kaazing Corp. (kaazing.com) commissioned Bergmans Mechatronics LLC to develop a simple WebSocket-based Amazon Elastic Compute Cloud (EC2) performance monitor.
The resulting system, shown schematically in Figure 1, operates as follows. Two Server Monitor Agent programs execute on the EC2 server at a rate of once per second. One agent executes the top shell command and the other executes the vmstat shell command. The results of these commands are transmitted as a Streaming Text Orientated Messaging Protocol (STOMP) message over conventional TCP socket to an ActiveMQ message broker on the server.
|Figure 1 Demonstration System Schematic
The message broker forwards these results to a Kaazing WebSocket Gateway that in turn transmits the STOMP messages via WebSocket to the browser. The browser presents this data in text format and also plots the CPU time-history data in a canvas element (Figure 2). The demo can be viewed live at http://18.104.22.168:8000/lab/server_monitor.html.
|Figure 2 Browser-based Monitoring of Cloud Server CPU Usage
In this demonstration, controls in the browser allow the user to start and stop a CPU-intensive load program on the EC2 instance. This load program is written in C and consists of an empty infinite while loop. As seen in the screenshot in Figure 2, this program consumes up to about 40% of the instance CPU when it is activated.
Other notable facts about this demonstration:
- The WebSocket interface permits significantly higher browser update rates than the 1 Hz rate shown in this demonstration. For example, one system developed by Bergmans Mechatronics updates data in the browser at a rate of 20 Hz (http://labsocket.com).
- The Kaazing WebSocket Gateway provides WebSocket emulation for those browsers without native WebSocket capability, including Internet Explorer 6. As a result, applications built using the Kaazing Gateway can be accessed by a wide variety of popular legacy browsers.
- The monitoring agents in this demonstration were written in PHP but could easily be written in almost any other language.
Addressing Real-World Requirements
The simple demonstration presented above illustrates the basic feasibility of using WebSocket to monitor server performance from a browser. A more practical browser-based server monitor should however include additional capabilities to provide users with greater understanding of the condition of the servers used in an application. Three ideas to meet this requirement are as follows:
Use of Existing Data Collection System
To meet the data monitoring requirement of system operators, many additional system parameters will need to be displayed in the browser. Rather than writing entirely new server-side monitoring software to generate this data, the use of an established server data collection system such as collectD is recommended. A monitoring agent on the server would then periodically extract data from the collection system database (RRD files in the case of collectD) and transmit the data via WebSocket to the browser. Alternately, in a more advanced configuration, monitoring agent modules capable of transmitting data via WebSocket could be incorporated directly into the data collection system.
The use of an existing data collection system would not only reduce the amount of software development required for browser-based monitoring, it would also give users a high degree of confidence in the displayed data since the data is collected using a system that is trusted and proven..
Improved Graphical Display
While time-averaging of data is effective for showing general trends in a data set, this approach can mask the presence of anomalies which might be of interest. Many graphical representations can be used to reveal these anomalies. One such representation is the temporal histogram. John Rauser’s presentation at Velocity 2011 examines the limitation of time averaging and the benefit of the temporal histogram. This type of histogram is shown at t=11:28 in the video of his presentation http://velocityconf.com/velocity2011/public/schedule/detail/20280.
Other desirable features to add to the display include:
- Use of tooltips to reveal additional information about specific data points and to possibly allow system operators to anotate the data
- Ability to adjust horizontal and vertical scales
- Ability to choose variables for horizontal and vertical axes
Many of these features (and more) already exist in browser-based data display applications, so implementing these features in a monitoring system using WebSocket should be feasible. Implementation of these display features might even be relatively simple if existing front-end software could be modified for use with WebSocket.
Multi-User, Multi-Server Monitoring and Control
Large-scale applications typically operate on multiple servers and could have a requirement for multiple operators to monitor and even control the system. The publish-subscribe messaging architecture of the existing demonstration system, enabled by the use of the ActiveMQ message broker, can be readily expanded to accomodate these extra system components, as shown in Figure 3. In this configuration, a central data aggregation and analysis module would probably be necessary to produce a complete view of the status of the application. Additional authorization and authentication logic would also be necessary to regulate user access to the system functions.
|Figure 3 Schematic of Notional Multi-Server, Multi-User Monitoring and Control System
As shown here, WebSockets can be used to enable simple browser-based server monitoring with a display update rate of once per second. Ideas for expanding this concept, from higher display update rates to a multi-server, multi-user control and monitoring system, have also been presented.
Specific areas for future development from which the Devops community could benefit include:
- Using the current demonstration system in essentially the same form, with customizations for monitoring system-specific parameters.
- Creating a WebSocket-based browser interface to an existing server data collection system.
- Developing a Multi-Server, Multi-User monitoring and control system.
Please feel free to contact me if you would like to explore any of these ideas. Comments and questions about this article are also welcome in the Comments section below.
About Bergmans Mechatronics
Bergmans Mechatronics LLC, founded by John Bergmans in 2003, performs WebSocket application research and development and training. In addition, BML develops custom data acquisition and control system software. The firm’s client base includes companies in the software, industrial, medical, and defense sectors.
2004-2016 Bergmans Mechatronics LLC