I’m writing a series of posts about Generalizing Apdex. This is #10.
In my post on Separately Configurable Thresholds in Apdex-G, I began work on a general description of Apdex zones and thresholds. The language I proposed would allow Apdex to be used to report on metrics for which “quality” increases or decreases monotonically with the value of the metric. Response times and VOIP MOS scores are typical monotonic metrics: smaller response times are always better, higher MOS scores are always better. Many metrics have this property. Arthur Schneiderman writes:
Properties of good metrics
The first requirement for a good metric is that it should be a reliable proxy for stakeholder satisfaction. In other words, improvement in the metric should link directly to improved stakeholder satisfaction. This linkage should be clear and uncomplicated. It should also be what mathematicians call monotonic—i.e., improvement in the metric should always produce improved stakeholder satisfaction …
With over 30 application performance management (APM) tool vendors offering scores of products, buyers face hundreds of confusing choices. Compounding the problem, the lack of a common taxonomy, or standard APM nomenclature, makes cross-vendor product comparisons especially challenging.
To address this challenge, NetForecast has developed an APM tools framework anyone can use to define APM requirements and map them to vendor offerings. The APM framework is a comprehensive description of application management functions and features. Building on essential infrastructure management concepts, it provides a common nomenclature that makes it easy to compare APM tools. The framework also places Apdex elements in their appropriate contexts.
Good performance is delivered with tools that directly support APM processes and best practices. The APM framework lets you describe and prioritize the functions and features appropriate for your environment, and helps guide your APM strategy.
The May 12, 2010 Apdex webinar material is available. Service Level Management (SLM) is the art and science of keeping application services running properly once in production. In this live webinar Peter Sevcik, Executive Director of the Apdex Alliance, described how Apdex works and provides a key performance indicator (KPI) that supports each layer [...]
The Apdex method can play a useful role in any SLM process. But it is not the whole solution. Apdex is certainly a very useful tool when you have collected measurements and need a simple way to show how well those measurements reflect your goals. But first you have to decide what those goals should be.
This aspect of SLM is a broad subject that I’ve explored previously, for example:
Today I was looking for a more systematic way to put Apdex in context, to focus on where Apdex can — and cannot — help. In the process, I discovered the SERVQUAL “gap” model, first defined in a 1985 paper, A Conceptual Model of Service Quality, by Parasuraman, Zeithaml, and Berry. There’s a link to a book and a (2.3Mb) copy of the original paper below. For a more concise and readable introduction online, I recommend the Theory Of The Gaps Model In Service Marketing, published by the Marketing Association of Australia and New Zealand.
Webinar Date: Wednesday, May 12, 2010
Webinar Time: 12:00 noon EDT (1 hour)
Now available: Recording and slides
Overview
Speaker: Peter Sevcik, Executive Director Apdex Alliance
Service level management (SLM) is the art and science of keeping application services running properly once in production. The key to successful SLM is the ability to use metrics that are linked to the business.
My previous post focused on the contribution of the Database Administrator (DBA) to application performance. Even so, application performance depends upon many factors, some of which are beyond the control of even the most dedicated DBA. So if you were thinking of relying on your DBA to fix everything, this week’s performance principle provides is intended as a wake-up call:
Expecting a DBA to guarantee the performance of any application that uses the database is like asking a piano tuner to guarantee a flawless performance, regardless of the pianist.
The annual Computer History Museum Fellow Awards program publicly recognizes individuals of outstanding merit who have significantly contributed to advances in computing technology or applications, and to the evolution of the information age. Fellows may have worked in such diverse fields as hardware, software, networking, computer science, business, education, public service, or journalism, but they have one thing in common: their contributions have had a direct influence on computer history, and ultimately, they have changed our lives.
Each year around this time, a “who’s who” of the technology world assembles at the museum in Mountain View CA for a banquet and ceremony to induct a new group of Fellows. This evening, among the six new Fellows chosen in 2009, Don Chamberlin will be honored. Don was a co-inventor of SQL, the world’s most widely-used database language, and one of the managers of IBM’s System R project, which produced the first SQL implementation and seeded the development of much of IBM’s relational database technology.
In recent years, much has been written about the value of use cases and scenarios for capturing functional requirements; by comparison, their usefulness for performance management has received scant attention . An application scenario defined for performance management purposes:
Involves a known fixed workload
Runs in the normal production environment
Runs against the production databases
Is instrumented to record response time
Because standard application scenarios are application/program instances with defined behaviors, their use of computing resources is also (relatively) predictable. In a sense, they are “benchmark” programs, since they perform a similar function. Normally however, performance benchmarks are designed to mimic a particular type of workload on a component or system, and are used to measure system capacity and throughput when processing a typically broad mix of applications. Standard application scenarios, in contrast, can be designed to measure a system’s responsiveness for a single precisely-defined set of processing needs.
If Sir Isaac Newton were stating the laws of computer systems performance, his first law would surely have been: The graph of performance continues in a straight line unless the force of some external event causes it to change.
Not knowing what changed is a serious impediment to problem diagnosis.
How does performance suddenly become “abnormal”? Of course, the answer is, it doesn’t–at least, not on its own. There is always an external cause. So to fix a performance problem, we must find the cause–usually more of something, such as:
Increased processing volumes
More data in a database
More customers
A new application competing for resources
Increased competition from existing applications on the same servers
Increased interference from other traffic on the network.
Apdex is a simple formula that converts many performance values into an easy to understand 0-to-1 performance index. Defining the target application response time T and accurately interpreting the result (the score) requires some methodology–and that methodology should reflect how you plan to use Apdex. There are three ways to use Apdex: to do [...]
