Deciding between Test Management Tools is difficult when there are many excellent free tools and paid test management tools are on the market. The three we will be comparing today are Zephyr vs. Xray, along with our free AccelaTest. Although each test case management tool has excellent options, deciding the right software tool for your QA team and development teams is a matter of cost, your software development process, and the features your team requires.
Summary of Feature Level Comparison
Before getting into the details of each of the test case management tools, we will analyze all the features of the test management tools and how they will help your software testing efforts.
What is each of the tools going to cost?
Xray vs. Zephyr both require a monthly subscription to use the test case management tool. In contrast, the comprehensive test case management features in AccelaTest are, and always will be, completely free.
The pricing for each test case management tool is as follows:
- Zephyr pricing starts at $10 per user per month
- Xray pricing starts at $10 / year
- AccelaTest’s price starts at $0.00 per user per month.
Additionally, both of these solutions do offer a free trial. In the pricing arena, AccelaTest is the clear winner for being the only no-cost solution that supports your entire testing process.
How do the tools rank as a Test Case Management Solution?
Test Case Management tools should allow you to perform test case writing, test plans and support the software development process. Fortunately, all three contenders deliver on their ability to manage test cases and include the ability to reuse test cases.
The most notable differences between the tools are a lack of Session-based exploratory testing for Zephyr and AccelaTest and the inability to reuse test repositories for Xray.
As the core features offered by all the tools in this arena are roughly the same, and all teams may not need the different features, we declare a tie.
How do the tools rank on the ability to Manage Test Runs?
The ability to create a test plans report, and a test runs report is vital for many teams. Fortunately, once again, all three contenders deliver on this both the ability to manage test executions and reports. Examples of these reports include:
- test case execution report
- test execution burndown chart
- overall requirement coverage report
- historical requirement coverage report
- top defects impacting testing report
The most notable difference between the tools is the inability to quickly set test results in Zephyr and Xray.
How well do the tools handle Defect Management?
Only Zephyr offers a standalone defect management component. However, Xray and AccelaTest offer integration with other tools for issues, such as Jira Issues through Jira plug-ins. Due to the same functionality offered by all of the solutions, we declare Defect Management a tie.
What are the 39 Xray vs. Zephyr Feature Level Comparison Points?
In this feature level comparison of Zephyr and Xray, we will be comparing these tool’s key features: Requirement Management, Test Cases, Execution of Test Cycles, Bugs or Defects, Integration Capabilities, Certifications, Support, Reporting, and finally finish up with Pricing.
There are pros and cons to every testing tool, so let’s get to it.
Requirement Management
Hierarchical requirements support
Requirement and Defect Linking
Test case management
Add attachments to test steps
Duplicate tests between projects
Create, view, edit, clone tests within project
Reusable test repositories
Session-based exploratory testing
Gherkin/Cucumber (BDD) support
Summary / Pre-Conditions
Test Sets and Execution Management
Test execution and results audit history
Modify tests during test execution
1
Report defects from test runner
1
Assign tests to testers
Test Case Parameterization
Ability to Clone, Edit, and Delete Test Runs
Associate test repository to any test execution set
Set test result (pass/fail) status in one click.
Bug management
Standalone issue management module
3rd party issue tracking integrations
Ability to report issues directly (email, Slack, Teams)
Integrations
Integration with Test Automation Tools
Jira Issues
Zapier
API
Slack integration
Webhooks
Customization
Filter information hierarchically (examples: Test names, Test tree, Bug details)
Custom workflows
Custom tags or fields
Permission-Based Privileges
Reporting
Customizable graphs, dashboards, and metrics
Scheduled report delivery
Additional features
Import & Export all data
Search (ability to add criteria)
Installation on-premises or Cloud
Cloud
Both
Both
Task Board
Security and certifications
Soc 2
ISO 27001
Service and Support
Support
Pricing
Price
From $0/mo
Zephyr Pricing
Starts at $10
Free Trial
Xray Pricing
Starts at $10 / year
Start Subscription
 |
 |
 |
||||
|---|---|---|---|---|---|---|
Requirement Management |
||||||
| Hierarchical requirements support | ||||||
| Requirement and Defect Linking | ||||||
Test case management |
||||||
| Add attachments to test steps | ||||||
| Duplicate tests between projects | ||||||
| Create, view, edit, clone tests within project | ||||||
| Reusable test repositories | ||||||
| Session-based exploratory testing | ||||||
| Gherkin/Cucumber (BDD) support | ||||||
| Summary / Pre-Conditions | ||||||
Test Sets and Execution Management |
||||||
| Test execution and results audit history | ||||||
| Modify tests during test execution | 1 | |||||
| Report defects from test runner | 1 | |||||
| Assign tests to testers | ||||||
| Test Case Parameterization | ||||||
| Ability to Clone, Edit, and Delete Test Runs | ||||||
| Associate test repository to any test execution set | ||||||
| Set test result (pass/fail) status in one click. | ||||||
Bug management |
||||||
| Standalone issue management module | ||||||
| 3rd party issue tracking integrations | ||||||
| Ability to report issues directly (email, Slack, Teams) | ||||||
Integrations |
||||||
| Integration with Test Automation Tools | ||||||
| Jira Issues | ||||||
| Zapier | ||||||
| API | ||||||
| Slack integration | ||||||
| Webhooks | ||||||
Customization |
||||||
| Filter information hierarchically (examples: Test names, Test tree, Bug details) | ||||||
| Custom workflows | ||||||
| Custom tags or fields | ||||||
| Permission-Based Privileges | ||||||
Reporting |
||||||
| Customizable graphs, dashboards, and metrics | ||||||
| Scheduled report delivery | ||||||
Additional features |
||||||
| Import & Export all data | ||||||
| Search (ability to add criteria) | ||||||
| Installation on-premises or Cloud | Cloud | Both | Both | |||
| Task Board | ||||||
Security and certifications |
||||||
| Soc 2 | ||||||
| ISO 27001 | ||||||
Service and Support |
||||||
| Support | ||||||
Pricing |
||||||
| Price |
|
Zephyr Pricing Starts at $10 Free Trial |
Xray Pricing Starts at $10 / year Start Subscription |
|||
Legend:
1: Functionality is on roadmap to be implemented.
2: Email support to activate for your project.
Test Case Management Tool Comparison – FAQ
What is AccelaTest?
Accelatest Test Runner Overview
Test Case Creation in AccelaTest
AccelaTest is different from many popular tools. We have made all core functionality our competitors charge money for entirely free for small teams. You read that right. We offer the ability to efficiently manage test plans, test case writing, test executions, test results, and reporting (test plans report, test runs report, test case execution report, etc.) at no cost.
What is Xray?
Xray is another comprehensive test case management application built to monitor testing progress and track progress. Xray was built with integration with Jira in mind. It can integrate with Jira Issue types through a Jira add-on.
Xray offers many impressive reports that allow teams to know the exact status of their project:
- Test Plans Report
- Traceability Report
- Overall Requirement Coverage Report
- Historical Requirement Coverage Report
- Test Executions Report
- Test Runs Report
Many agile teams use Xray to boost productivity and ensure the highest quality software is delivered. Users have found great success using this solution.
What is Zephyr?
Zephyr is owned by Smartbear and has been built to heavily integrate with Jira through the use of Jira plug-ins. In addition to Jira, Zephyr also integrates through many other third-party plugins available on the market like Bamboo, Jenkins, and many others.
Similar to Xray, Zephyr offers many quality reports:
- Traceability Report
- Test Execution Burndown Chart
- Test Executions Report
- Top Defects Impacting Testing
Your QA team could join the many other happy users who have found much success with Zephyr.
Zephyr vs. Xray Conclusion
Hopefully, you now have a better understanding of Xray vs. Zephyr, in addition to our Free Test Management tool. As you have seen, Zephyr and Xray have similar features, and AccelaTest will continue adding new features. This makes the cost of test management tools a significant difference. The test case management features of AccelaTest are free while Zephyr and Xray will continue to cost you monthly. Open your free account now or discover all the features offered here.
Zephyr and Xray Resources: