Active versus Passive Monitoring for Web Sites

According to Gartner, there are 5 dimensions of an APM solution¹:

1. End-user experience monitoring — the capture of data about how end-to-end application availability, latency, execution correctness and quality appeared to the end user.
2. Runtime application architecture discovery and display — the discovery of the various software and hardware components involved in application execution, and the array of possible paths across which those components could communicate, which together enable that involvement.
3. User-defined transaction profiling — the tracing of events as they occur among the components or objects as transactions move across the paths discovered in the second dimension, generated in response to a user's attempt to cause the application to execute what the user regards as a logical unit of work.
4. Application component deep-dive monitoring — the fine-grained monitoring of resources consumed by, and events occurring within, the components discovered in the second dimension.
5. Analytics — the marshalling of a variety of techniques, including behavior learning engines, complex-event processing platforms, log analysis, and multidimensional database analysis to discover meaningful and actionable patterns in the typically large datasets generated by the first four dimensions of APM

Note 1 Application Performance
Monitoring Is the First Step to Manage the Hybrid Cloud Application Portfolio
19 December 2011 ID:G00227693
Analyst(s): Will Cappelli

Focusing specifically on the end-user experience monitoring, nirvana would be the ability to monitor every router hop from the customer to the datacenter; the entire components of the user’s screen; monitoring the web server they land on and the entire application path from the web server to the database and back out.

The reality is that is impossible to instrument the Internet, client equipment, data center infrastructure, application code and hardware all at the same time. 
 
 
 
Even if you could provide that level of instrumentation, it would prove useless in determining the actual quality of all end- users’ experience from just the sheer amount of data.

When it comes to diagnosing how a site is performing, an accurate quality assessment can be made by monitoring the customer's activity, errors and response times. But what methods to monitor the end users are available?  Which are the best - Active or Passive?

A comprehensive strategy can be made with active monitoring (synthetic or scripted), passive monitoring (network-based packet capture, known as End User Experience Monitoring).

Synthetic Monitoring

Synthetic monitoring’s can be considered “Active Monitoring” because you are acting as the client. The earliest beginnings of Synthetic monitoring stem from the data center operator or person in charge hitting their own site from their desktop and declaring it “alive”.  onstantly hitting your own web site isn’t a good use of a person’s time, so scripts were written to continuously hit the site. However, there are valid reasons to have this mechanism in place.

Some of the benefits that synthetic testing provides are:

• Consistently determining if the site or page is up or down. Consider the time when a page is not generally available to the public or if there are no end users hitting that particular page or aspect of the site.
• Synthetic tests can use various protocols (doesn’t always have to be just HTTP/s) that may be difficult or impossible to monitor on the wire (like RMI, Flash, Silverlight, Citrix ICA, FTP, or other binary protocols that cannot easily be decoded).
• Synthetic scripts can be executed from specific locations – using that locality to determine region-specific problemsActive monitoring is looking at a website from outside-in and will allow for monitoring of timing and errors specific to external web components like mash-ups (like a Google map link)  or third party hosted elements (like the Customer Support Person who drops down on the screen) that are not really hosted from the data center of the web owner.
• Scripted monitoring can be pointed to your web site or others to show how similar scripts act against ‘golden’ or competitive web sites.

Some of the drawbacks to this active method of testing are:

• Can only test the same thing over and over and on a periodic basis.
• The test is not real – you can’t buy shoes every five minutes on a real web site.
• A script cannot test everything. Humans are amazing at attempting odd combinations in real life.
• Scripting does not provide a real time sampling. Periodic sampling does not provide a good indication of what everyone is experiencing on a web site. If Synthetic scripts test a transaction every five minutes and the site is taking thousands of hits per second, thousands of errors or unhappy users would not be reflected by the script's results.

Products like BMC’s TM-ART provide a comprehensive method to script many types of transactions that would be found in a web environment.

Network-based packet capture

Products like BMC's - End User Experience Management (EUEM) are a Network-based packet capture "Passive" monitoring solution.

Typically, End User Experience Monitoring (EUEM) or Real User Monitoring (RUM) is a passive technology that records and monitors user interactions with a website.   Monitoring the users’ actual activity on a website or an application provides visibility into performance, errors and volume of the web property. For the sake of this discussion, we will focus on true passive monitoring which is via network packet capture.

Some of the benefits that real end user monitoring testing provides are:

• Typically sees all users all the time.
• Typically "Inside your four walls".
• Provides a real view into what all users see.
• No overhead – imposes no load on the servers or application.

Some of the drawbacks to a real end user monitoring method of testing are:

• Can't see CPU/Memory/Network bottlenecks.
• Requires a device be located in a central collection spot.

Conclusions

When the question comes up – “Which is better, Active or Passive monitoring of a web site”? The real conclusion is that both are unique and provide advantages and some disadvantages. Current implementations of web properties include many variations like mash-ups, externally hosted content, Flash or Ajax, for example. The combination of active and passive monitoring provides a more comprehensiveview of website performance and user experience then each separately. Therefore it is recommendable to use both active and passive technologies to monitor web performance.