The software industry is always changing, with new advances in technology happening on a regular basis. While the internet has transformed almost every industry around the globe, its progression is not always easy. With new software being released to the market every day, companies are often required to invest in new technology. Unfortunately, not all developers are aware of the potential consequences that can arise from neglecting performance and load testing. This is where software service level agreements (SLAs) enter the picture.
What is a Software Service Level Agreement?
A software service level agreement is an agreement between a client and a service provider that specifies the level of support to be provided with regards to availability, performance, and other quality-related metrics.
Why do development teams need an SLA?
SLAs are important to development teams because they provide a baseline standard for performance and operational quality. A development team can use SLAs with their clients to set clear expectations regarding the software they create and provide a framework for measuring their performance. A development team can use metrics, thresholds, and expected performance to measure their own success.
How can you define Baseline Performance Metrics?
Web performance is increasingly important for both your user experience and your bottom line as users abandon sites after a few seconds. You must set appropriate performance SLAs to meet your user’s expectations. Thankfully, there are multiple places to get inspiration from on defining performance SLA parameters.
The three best places to obtain SLA expectations are from internal teams, industry benchmarks, and from your competition.
Internal Teams
Obviously, utilizing your own teams to define your SLAs is the simplest and most convenient technique. Because they’ve already traveled that road before, they’re intimately familiar with what’s possible and what is realistic.
Industry Benchmarks
A quick Google Search can yield performance metrics your industry is using. Another avenue of gaining this information is to examine marketing/SEO agencies as they regularly publish industry data.
Look to Your Competition
As developers, we should strive to make our products the best out there. One way to accomplish that is by beating the competition, therefore it is imperative to take a look at the competitors’ metrics. Other teams may prefer to define their own, but benchmarking against 3rd parties is a perfectly acceptable method as well.
You can check their page speed in an easy, non-invasive manner by running a free speed test against their site to gain insights in their performance metrics. However, be cautious not to go overboard. Too many virtual users may cause their applications to fail, alerting their IT department and potentially resulting in a blocked IP Address. To make matters even worse, if you went too far you could inadvertently cause a DDoS attach, which is illegal. Should this happen, your day would go from bad to worse.
Identifying your application’s performance SLAs will not only make sure you’re on track with your product, but it will also maintain user trust in your product. With all that said, be sure to tailor your performance SLAs to meet the needs of your customers.
How do you create a Performance and Load Testing SLA Plan?
Most Software Quality Assurance (SQA) teams are required to test the performance of applications that are under development. These tests are used to determine the impact of new features on an application’s performance, identify bugs that may cause error conditions, and to determine the readiness of an application for use by end-users.
SLA Testing Factors to help determine application readiness
Testing of the service, and software in general, is often a significant part of verifying any Service Level Agreement (SLA). The SLA for Performance and Load Testing typically consists of 3 parts: type of software being tested, expected load on the application, and required availability. These 3 parts are described below:
Type of software being tested :
Testing will vary depending on the type of software being tested. For example, mobile applications typically require different testing methods than web applications. Testing methodologies may include functional testing, regression testing, and load testing.
Expected load on the application :
The expected load varies depending on what kind of application is being tested (e.g., CPU utilization, memory usage, transaction volume). For example, a mobile app may be expected to handle hundreds of concurrent users with the majority of transactions completing in less than 10 seconds. In comparison, a web application may be expected to handle hundreds of concurrent users with the majority of transactions completing in less than 2 seconds.
Required availability :
The required availability of the application is also dependent on the type of software being tested (e.g., percent of time the software must be available, maximum allowable downtime). For example, a mobile app may be required to be available 100% of the time, whereas a web application may only need to be available 99.9% of the time.
What are the steps to performance and load test the SLA agreements?
Performance and load testing of services may be performed under the supervision of both the development team and their client. This process provides verification that milestones in development are met, and expected SLA are being achieved. Here is the typical process for testing SLAs:
1. Establishing a performance baseline before implementing any new functionality or services.
2. Verifying new functionality or services doesn’t impact your baseline.
3. Conducting tests when changes are implemented to ensure the addition is not impacting load time.
4. Utilizing a framework to define the test cases and the expected load.
5. Keeping track of all test results in one place for easy reference.
6. Having a clear plan of action to deal with issues that arise during load testing.
7. Benchmarking software components against your SLA requirements.
8. Continue to monitor each component and ensure degradation does not occur.
What documentation is in an SLA?
The Introduction
The first section of an SLA is known as the introduction. This section contains language that defines and describes the agreement. It should include information about the parties involved, a description of the service levels to be provided, and what processes are in place for escalation and problem resolution.
The Scope
The next section that should be included in an SLA is known as the scope. This section details what will and will not be included in a service agreement. For instance, if a service provider cannot offer custom development services, it may be important to disclose this information early. What equipment will or will not be covered by the SLA? Will the service provider supply replacement parts or software? Will they offer end user training?
Responsibilities
A service level agreement outlines the responsibilities of the company providing services and those receiving them. Responsibilities can include issues such as backup storage, internet connectivity, system monitoring, updates to software patches, security measures taken to protect data, and more.
Availability and Performance Guarantees
A service level agreement can guarantee that a system will be available at certain times. This guarantee may be set on a per second or per minute basis. For instance, a company might specify in an SLA that their website must be available 99% of the time with a maximum allowable downtime of 50 minutes per quarter. This section may include the number of hours that a customer can be offline before some sort of compensation is due.
In short, these are the actual numbers that describe how available a system or service will be, and what type of performance can be expected. These metrics should include specifications regarding the number of transactions per second, maximum response times for critical functions, average response times for various functions, maximum memory usage, and any other key metrics that are important to the service provider or customer.
Response Times
Sometimes SLAs outline the amount of time a service provider has to respond to service requests or whenever there is a query. As with all industries, Customer Service is vital to Customer Satisfaction, especially when you depend on a SaaS company for critical business functionality. You want your business partners to respond quickly whenever there are issues, especially when they are on their end.
Resolution Times
Most SLAs include a clause that states how long it takes for a customer to receive a resolution from the service provider if their system is down. In most cases, if a customer contacts support and does not receive an answer within one hour, some sort of compensation will be provided to acknowledge the lack of responsiveness on the part of the company providing services.
Signatures
Both parties come together to sign the SLA which binds them to the terms.
Conclusion
A software service level agreement is an incredibly useful document for defining expectations between customers and developers of applications. It can also serve as a guide for development teams to measure their own performance. SLAs are something that should be included in all software applications, particularly those designed for mission critical use.