Zephyr vs. Xray Comparison

Deciding between Test Management Tools is difficult when there are many excellent free tools and paid test management tools are on the market. The three we will be comparing today are Zephyr vs. Xray, along with our free AccelaTest. Although each test case management tool has excellent options, deciding the right software tool for your QA team and development teams is a matter of cost, your software development process, and the features your team requires.

Summary of Feature Level Comparison

Before getting into the details of each of the test case management tools, we will analyze all the features of the test management tools and how they will help your software testing efforts.

What is each of the tools going to cost?

Xray vs. Zephyr both require a monthly subscription to use the test case management tool. In contrast, the comprehensive test case management features in AccelaTest are, and always will be, completely free.

The pricing for each test case management tool is as follows:

• Zephyr pricing starts at $10 per user per month
• Xray pricing starts at $10 / year
• AccelaTest’s price is always $0.00 per user per month.

Additionally, both of these solutions do offer a free trial. In the pricing arena, AccelaTest is the clear winner for being the only no-cost solution that supports your entire testing process.

How do the tools rank as a Test Case Management Solution?

Test Case Management tools should allow you to perform test case writing, test plans and support the software development process. Fortunately, all three contenders deliver on their ability to manage test cases and include the ability to reuse test cases.

The most notable differences between the tools are a lack of Session-based exploratory testing for Zephyr and AccelaTest and the inability to reuse test repositories for Xray.

As the core features offered by all the tools in this arena are roughly the same, and all teams may not need the different features, we declare a tie.

How do the tools rank on the ability to Manage Test Runs?

The ability to create a test plans report, and a test runs report is vital for many teams. Fortunately, once again, all three contenders deliver on this both the ability to manage test executions and reports. Examples of these reports include:

• test case execution report
• test execution burndown chart
• overall requirement coverage report
• historical requirement coverage report
• top defects impacting testing report

The most notable difference between the tools is the inability to quickly set test results in Zephyr and Xray. AccelaTest is still in active development and working towards adding more features, which is documented below.

Once all of the enhancements are complete, AccelaTest will be the clear winner in this category.

How well do the tools handle Defect Management?

Only Zephyr offers a standalone defect management component. However, Xray and AccelaTest offer integration with other tools for issues, such as Jira Issues through Jira plug-ins. Due to the same functionality offered by all of the solutions, we declare Defect Management a tie.

What are the 39 Xray vs. Zephyr Feature Level Comparison Points?

In this feature level comparison of Zephyr and Xray, we will be comparing these tool’s key features: Requirement Management, Test Cases, Execution of Test Cycles, Bugs or Defects, Integration Capabilities, Certifications, Support, Reporting, and finally finish up with Pricing.

There are pros and cons to every testing tool, so let’s get to it.

Requirement Management
Hierarchical requirements support
Requirement and Defect Linking
Test case management
Add attachments to test steps
Duplicate tests between projects	2
Create, view, edit, clone tests within project	2
Reusable test repositories
Session-based exploratory testing
Gherkin/Cucumber (BDD) support	1
Summary / Pre-Conditions
Test Sets and Execution Management
Test execution and results audit history
Modify tests during test execution	1
Report defects from test runner	1
Assign tests to testers
Test Case Parameterization
Ability to Clone, Edit, and Delete Test Runs
Associate test repository to any test execution set
Set test result (pass/fail) status in one click.
Bug management
Standalone issue management module
3rd party issue tracking integrations	1
Ability to report issues directly (email, Slack, Teams)
Integrations
Integration with Test Automation Tools	Built In1
Jira Issues	1
Zapier
API
Slack integration
Webhooks
Customization
Filter information hierarchically (examples: Test names, Test tree, Bug details)	1
Custom workflows
Custom tags or fields
Permission-Based Privileges
Reporting
Customizable graphs, dashboards, and metrics	1
Scheduled report delivery	1
Additional features
Import & Export all data	2
Search (ability to add criteria)
Installation on-premises or Cloud	Cloud	Both	Both
Task Board
Security and certifications
Soc 2
ISO 27001
Service and Support
Support
Pricing
Price	Free Test Management Tool Start Immediately	Zephyr Pricing Starts at $10 Free Trial	Xray Pricing Starts at $10 / year Start Subscription

Legend:
¹: Functionality is on roadmap to be implemented.
²: Email support to activate for your project.

Test Case Management Tool Comparison – FAQ

What is AccelaTest?

AccelaTest is different from many popular tools. We have made all core functionality our competitors charge money for entirely free. You read that right. We offer the ability to efficiently manage test plans, test case writing, test executions, test results, and reporting (test plans report, test runs report, test case execution report, etc.) at no cost. We earn revenue from our hybrid automated testing approach.

Our hybrid automated testing approach combines API automation, real-time insights on REST API health, and manual testing under one testbed. This empowers manual testers to create an automation suite while reducing the time it takes to execute test cases.

What is Xray?

Xray is another comprehensive test case management application built to monitor testing progress and track progress. Xray was built with integration with Jira in mind. It can integrate with Jira Issue types through a Jira add-on.

Xray offers many impressive reports that allow teams to know the exact status of their project:

• Test Plans Report
• Traceability Report
• Overall Requirement Coverage Report
• Historical Requirement Coverage Report
• Test Executions Report
• Test Runs Report

Many agile teams use Xray to boost productivity and ensure the highest quality software is delivered. Users have found great success using this solution.

What is Zephyr?

Zephyr is owned by Smartbear and has been built to heavily integrate with Jira through the use of Jira plug-ins. In addition to Jira, Zephyr also integrates through many other third-party plugins available on the market like Bamboo, Jenkins, and many others.

Similar to Xray, Zephyr offers many quality reports:

• Traceability Report
• Test Execution Burndown Chart
• Test Executions Report
• Top Defects Impacting Testing

Your QA team could join the many other happy users who have found much success with Zephyr.

Zephyr vs. Xray Conclusion

Hopefully, you now have a better understanding of Xray vs. Zephyr, in addition to our Free Test Management tool. As you have seen, Zephyr and Xray have similar features and AccelaTest will continue adding new features. This makes the cost of test management tools a significant difference. The test case management features of AccelaTest are forever free while Zephyr and Xray will continue to cost you monthly. Open your free account now or discover all the features offered here.

Zephyr and Xray Resources:

• Zephyr website: https://smartbear.com/test-management/zephyr-enterprise/features/agile-testing/
• Xray: https://www.getxray.app/test-management