6.4 qTest 5.0 Release Notes - Apr 12th, 2015

Automation Integration:

In this release, we enhance the current integration with TestNG and provide new integrations with two automation frameworks, JUnit and Cucumber for Java.

qTest now features Automation Scheduling which allows you to schedule and kick off your automation tests that reside on your local machine and report the results of these tests back to your qTest server.

1. Automation Host:

An Automation Host is your local server where your automation scripts are deployed and executed. You will need to download our Automation Agent tool from the qTest Resources page and run it on the server. We provide three packages for three different OS types – Windows, Linux, and Macintosh.

The tool will register the host in your qTest. After that, you are able to download and install plugins for the supported frameworks on which you are working.

2. Automation Agents:

Once you have installed your desired plugins, you can create and configure your Automation Agents. An agent integrates a qTest project with one test project deployed in your Automation Host. It supports multiple agents in one host.

Basically, you need to select a qTest project; specify the test project’s directory; and which tool (Command line, Ant or Maven) to deploy and run the scripts. The required information may vary based upon the selected plugins.

The agents will execute your testing scripts based on its configurations.

3. Automatically create Test Cases

Our provided plugins support the ability to scan your script directory and then create automation Test Cases in your qTest project from detected tests.

An automation Test Case in qTest may be associated with either a Java class or a Java method from your test script.

  • If a class has 2 methods; then the corresponding Test Cases will have 2 steps and their descriptions are the same as methods’ names.
  • If you create a Test Case from a method; then it will have one step.

4. Automation Scheduling

Once the Automation Host and Agents are set up, you are ready to schedule your automation Test Runs. You can either schedule a single Test Run or multiple Test Runs in the same Test Suites.

You will have many options in creating an automation schedule:

The Automation Agent polls qTest for new schedules frequently based on a configured interval. As soon as it receives your schedules, it will run the corresponding scripts on your Automation Host. After the execution is done, the test and test step results are reported back to qTest. If there are any failed steps, you can find their error logs attached to the corresponding Test Log.

5. View Agents’ Schedules

In your qTest project, there is a centralized place for you to:

  • View all registered Automation Hosts
  • View each agent’s schedules and their jobs’ status
  • View which Test Runs are included in an schedule
  • You can also delete a planned schedule and its following occurences.

Once created, a scheduled job’s status will change corresponding to its execution result.

  • Planned: the job is scheduled to be executed in the future
  • Running: the agent has received the schedule from qTest and is executing it
  • Completed: the agent has executed the schedule without error
  • Failed: the agent failed to execute the schedule
  • Not Run: the schedule has expired


    • Currently, your user need to be granted with Manage Project Settings permission to view your project's schedules. We will provide the permissions to control rights to view and manipulate schedules soon.


Configuration Settings:


    • This feature is available in qTest Premium edition.

A Configuration is a combination of multiple variables such as CPU, OS type, device model, etc. that may need to be tested against and tracked by your team. qTest now provides a solution to create and manage Configurations at Site level, so they can be reported on uniformly across your qTest projects.

1. Create Configurations in Administration section

In qTest, a Configuration is created from a combination of Variables. You first need to create the possible Variables and define its allowed Allowed Values.

Once you have all of the Variables set up, you can create Configurations by combining specific values of the Variables you have chosen.

2. Choose active Configurations for each Project

Configuration, a new system field, is added in each project’s Field Settings and must be activated to enable the configuration feature in each project. The field will initially include all Configurations created in Site level. You can select which one of them are active in each particular project. Each value may be assigned labels that are specific to that project to improve the ways in which configurations can be leveraged.

3. Create multiple Test Runs

You can create multiple test runs from each test case selected by leveraging a label or selecting multiple configurations directly in the Add Test Runs dialog.


Enhanced Reporting:

1. Automation vs. Manual

A new filter is added to all Test Case and Test Execution reports. You have options to view either Automation Type or Manual Type Test Cases/Runs only, if desired.

2. Snapshot reports

In addition to the two existing analysis reports (Test Execution Analysis and Defect Analysis reports) which provide the accumulative views, this release features the snapshot views of these reports.

      • Test Execution Analysis (Snapshot view)
      • Defect Analysis (Snapshot view)

They report the activities recorded in a day, a week, a month, a quarter or a month instead of continuously aggregated numbers.

3. Saved Filter Settings

The report settings now can be saved and reused afterwards. They are shared across the project team members for viewing.

As the creator, you can modify and delete your saved filters. You can modify other members’ filters and save them as yours.


JIRA Widget:

You can view this widget right from your JIRA projects. It provides the high level reports of integrated data between your JIRA project and qTest. There are four blocks of summarized reports, each of which has its own drill down report.

      • Unresolved Defects by Test Run Status
      • Resolved Defects by Test Run Status
      • Synced Requirements without Test Case
      • Top 10 Synced Requirements with Defects submitted from qTest

You can export the drill down reports to Excel files for further reporting purposes.


      • If you are using a JIRA Server instance, please upgrade your qTest Connector plugin to the latest version.
      • The feature is currently unavailable for JIRA version 4.x. We are releasing it in a near future.
      • For JIRA Cloud instances, you will need to install our new plugin (click here for instructions).


Enhancements for Integration with external systems (JIRA, Rally and VersionOne)

1. Automatically organize imported Requirements from JIRA

You can organize imported requirements from JIRA automatically based on specific criteria in 1 or 2 JIRA fields. When selected, qTest will organize requirements by these fields’ values into child folders below the "Imported from JIRA" folder.

2. Has Association with Defects in external defect trackers

When you query for Requirements with criterion “Has association with defects”, Requirements associated with defects tracked in your defect tracker (JIRA, Rally or VersionOne) will be included in the result set.

3. New criteria to query imported Requirements from the external system by their external ID and object types.

4. Search function

Imported Requirements from external system can be searched using the following syntax:

external:”free text”

5. Display external ID on Requirement tree

6. Requirement Traceability Matrix report

In the exported file, a new column, Associated External ID, is added. If a requirement is imported from JIRA, its JIRA ID will be included in this column.


Other enhancements

1. Test Case Traceability Matrix report

A new column, Automation, is added to the exported Excel file. It indicates if this is an automation Test Case or a manual one.

2. Requirements and Test Execution Results report

New columns are added to the exported Excel file to show the number of associated Defects and their ID.

3. Requirement/Test Case Import

New columns are added to the exported Excel file to show the number of associated Defects and their ID.

4. Security Settings

An email will be sent to inform a user if his qTest password is expired in the next x days.