When attempting to determine whether an application is successful, one of the hardest things to quantify is User Experience or UX. Developers typically rely on qualitative measurements based on surveys or personal experience to evaluate the UX of software. But there’s a more accurate and straightforward way to see how successful your UX is: Apdex Score.
Apdex is a model for measuring UX on a scale that places the response time of an application against a benchmark and assigns a score ranging from “excellent” to “unacceptable.”
While surveys and personal experience can be useful tools in indicating how well an application performs from a user’s standpoint, they don’t give you any quantitative indicators. On the other hand, Apdex can tell you whether an app’s UX is running well or needs to be improved using a set of pre-defined criteria.
In the rest of this article, you’ll learn more about what an Apdex is, the advantages of an Apdex, and how this score is calculated.
What is Apdex?
Apdex is an abbreviation for Application Performance Index, which was created as an open standard for measuring the satisfaction with the response time of web applications. It is designed to provide a numerical scale to measure the success of an application as measured by its response time based on a set of samples.
By using Apdex, you can set benchmarks for measuring the performance of an application so that changes can be made to improve User Experience.
Measurement Criteria
An Apdex Score for a transaction or event is classified based on three basic criteria:
- Satisfied – the action has completed within the acceptable guidelines.
- Tolerating – the action completed, but not within an ideal amount of time.
- Frustrated – the action either did not complete at all or only completed after an unsatisfactory amount of time.
Variables
The key variables in Apdex consist of the following:
- T = the amount of time specified for a successful transaction.
- Satisfied Count = the number of transactions or activities which process equal to or less than T.
- Tolerating Count = the number of transactions that take within four times the Satisfied Count.
- Frustrated Count = the number of transactions or actions which do not process within the above guidelines.
- Total Samples = the total number of transactions or activities within a given timeframe.
A satisfied score is any score that is less than or equal to “T.” A “tolerating” score is equal to “4T,” or four times the amount of time that one would expect a process to complete. Any transaction or process that takes between “T” and “4T” is considered to be “Tolerating.” Any action which takes longer than “4T” is considered “frustrated.”
The Formula
To calculate an Apdex score, one adds the multiple of satisfied transactions to the multiple of tolerating transactions multiplied by 0.5, plus the number of frustrated transactions multiplied by 0 (you can technically leave out this number, as it will always be 0; however, it is here for clarification of the model), and divide this total by the total number of transactions.
Apex can be represented by this formula:
Image Source: https://en.wikipedia.org/wiki/Apdex
The Apdex formula provides a measurement that falls between the range of 0 and 1. A 0 represents an unacceptable result (wherein a user would be completely unsatisfied in all instances), and 1 represents a perfect excellent response time (where the user receives a satisfactory result in every instance).
The closer you are to 1, the better your UX.
Example Case
Let us assume a transaction should take 1.2 seconds to complete. Our “T” would, therefore, be set at 1.2. Let’s say 100 transactions have been processed during a specific time period. In this example, 75 of them processed in less than this time, so our satisfied count is equal to 75. Our Tolerating index is “4T,” which calculates to 4.8. In this case, imagine that we have 20 actions that took between 1.21 seconds and 4.8 seconds. Our tolerating count would then be 20 * 0.5, which is 10. The remaining 5 transactions took over 4.8 seconds. We multiply 5 * 0, which is, of course, 0.
To get our score, we then enter the numbers into our formula:
(75 + (20 * 0.5) + (5 * 0))/100
Our score is therefore calculated as 0.85.
Interpreting Results
If we have a sufficient “N” (or TotalSamples), we should be able to calculate a clear Apdex score, which will fall between 0 and 1.
The criteria for evaluating the results can be seen in the following table:
| Value | Rating |
| 1 – 0.94 | Excellent |
| 0.93 – 0.85 | Good |
| 0.84 – 0.70 | Fair |
| 0.69 – 0.49 | Poor |
| < 0.49 | Unacceptable |
Any score of “unacceptable” assumes that users will simply give up and no longer use the process. This is the last thing you want. In the case of poor or unacceptable scores, you need to implement changes to your UX as soon as possible, so you don’t lose potential and existing users.
Where Can You See Your Score?
So, now you know how to calculate your Apdex score. But imagine if you have hundreds, thousands, or even millions of transactions and events coming through. It would be challenging to calculate your Apdex score manually.
Airbrake Performance Monitoring software can help. Airbrake APM is specifically designed for developers and coders interested in monitoring their application’s total health. We provide several tools that give you the context needed to keep your application’s health in tip-top shape - including calculating your Apdex score.
Already have an Error Monitoring account with Airbrake? Perfect! The same notifier you installed for Airbrake Error Monitoring also supports Performance Monitoring, and all Airbrake plans include 50K free performance events each billing cycle. Airbrake currently supports Performance Monitoring for Rails, Node.js (Express.js), Go, Django, and Flask projects, with more languages and frameworks being added soon. All you need to do to see your Apdex Score is head on over to the Performance tab of your Airbrake dashboard.
If you don’t already have an account, sign-up to try Airbrake for free for 30 days of unlimited errors and unlimited performance events!
