- From the release of qTest version 5.0.0, we enhanced the integration with TestNG and provide new integrations with two automation frameworks, JUnit and Cucumber for Java.
- qTest features Automation Scheduling which allows you to schedule and kick off your automation tests that reside on your local machine and report the results of these tests back to your qTest server.
- Please click here for instructions.
A.Introduction to qTest Automation Integration
qTest automation integration is available for Enterprise and higher packages. It includes a set of features and the API & SDK packages enable qTest user to submit automation execution results along with manual testing activities outputs.
- Create automation test cases and define qTest test suite using these test cases.
- Import automation test suite (TestNG XML) as qTest test suite.
- Export the XML from qTest test suite for automation execution.
- Commit the automation execution result to qTest API services or SDK packages.
Project admin has to activate the automation settings to enable the above features, links to download Instructions and Integration packages are available on qTest Resource page.
qTest will not store the automation scripts or schedule the automation execution.
Automation Deployment Model
qTest supports TestNG (Selenium) by default (XML format and a JAR package to automate the integration process), we are working on additional automation frameworks that will be available in upcoming releases. Please contact our Sales to have your automation framework / tool supported.
We also provide APIs, Java & .NET SDK for automation engineer to build custom integration with qTest.
Integration Process Summary
- Admin has to turn on Project Automation Settings and map the Automation Result Values to qTest execution result value eg: TestNG: Passed, Failed, Skipped to qTest: Passed, Failed, Incomplete, Blocked. This step is necessary for your automation execution to post consistent result with the manual test execution.
- If you already have Automation Execution Results that need to be stored toqTest. You can manually create / import these automation scripts and update Automation Execution Results manually using Run/Quick Run as follows:
a.Create Automation Test Case > Create Automation Test Suite > Create Automation Test Run > Quick Run to mark execution status.
b.Import Automation Test Suite XML to Test Suite in Test Execution > Quick Run to mark execution status.
- Tosetup the direct integration withqTest:
a.If you are using TestNG: Download the TestNG Integration Package and follow the instruction (refer to the instruction Settings on Automation Project" below)
b.If you are using a Java & .NET based automation framework: Download the SDK Packages and refer to qTest SDK specification document to build the integration.
c.If you are using other frameworks: Building the integration via API refer to qTest API specification document.
Defining Automation Test Cases - Test Suites with qTest
You can create/import and export automation test case with qTest, below is an example of how qTest will create the automation test case using the imported XML.
- A test case is defined by element, it can be class(es), include or exclude methods of a class or a package.
- By default qTest will use to be qTest test case name, if this value is missing qTest will show the class name as test case name, users has the option to change this value during the import process.
- If the user manually creates the automation test case, the automation value should be entered exactly by using the content within the XML tag. qTest will use this string to identify the automation test case and to export XML test suite for execution later.
- If the user exports the automation test suite, qTest will include the content of each test case and the associate qTest Test Run's ID so the API or SDK has the information to update the result back to qTest properly.
<?xml version="1.0" encoding="iso-8859-1"?>
<suite name="TS qTest Suite">
<test name="Sample Case 1">
<package name="com.testng.examples.suite.*" />
<test name="Sample Case 2">
<class name="com.testng.examples.suite.TestPlan" />
<exclude name="testLinkAssociatedTestCase" />
<test name="Sample Case 3">
<class name="com.testng.examples.suite.TestDesign" />
<include name="testCreateTestCycle" />
<include name="testCreateTestSuite" />
<test name="Sample Case 4">
<parameter name="testRunId" value="474821"/>
<include name="testCreateDefects" />
B.Instruction to set up Automation Integration with qTest
Automation Settings on qTest
Important: You need the qTest Premium licenses to enable Automation Integration feature on qTest.
- Go to Project Settings and select Automation Settings
- Turn on the Activation Status in Automation Settings page
- Map Test Status you define in your automation script to qTest Execution Status
- Click Save
Define Automation Test Cases in qTest's Test Design
- Go to qTest's Test Design menu
- Click Add New Automation Test Case icon on the toolbar
- Define Automation XMLor copy from your automation script
- Click Save to finish creating automation Test Case
You can edit automation Test Case by updating its Automation XML in its Properties.
Define Automation Test Runs into qTest's Test Execution
- You can add Automation Test Runs in Test Execution as adding manual Test Runs.
- Select a Test Suite to contain Automation Test Runs
- Click Add Test Runs button on toolbar of either navigation tree or Test Runs Grid
- Select Automation Test Cases in the Add Test Runs pop-up
- Click Add button and Save Test Suite
- Alternatively, you can import Automation Test Runs from an XML file.
- Select a Test Suite to contain Automation Test Runs
- Click Import Automation icon on the top toolbar
- Upload an Automation script from your local folder and click Next.
Refer to our Sample XML file for the best mapping structure.
- Check the Automation Mapping pop-up and update your mapped XML script if needed
- Click Import button
- If the imported Automation XML does not exist in qTest's Test Case, system will automatically add new Automation Test Cases and simultaneously add associated Automation Test Runs.
- If the imported Automation XML already exists in qTest's Test Case, system will only add Automation Test Runs.
- Currently, you can update Automation XML in individual Test Cases ONLY.
- Export Automation XML
- Go to Test Execution and click to open a Test Suite
- Select Test Runs you want to import in the Test Runs Grid table
- Click Export Automation button the Test Run Grid toolbar
- The Automation Test will be exported into XML file
- qTest will automatically generate a new parameter "testRunId" in the exported XML file to identify each Test Run. Upon the unique "testRunId", qTest will update Results to executed Test Run correctly.
- You can replace executed script by the exported XML file and run Automation Test
Settings on Automation Project
qTest Automation integratation fully supports scripts that implemented by Selenium and TestNG framework.
- Go to Download qTest Resources page.
- Download properties file and TestNG Integration package from the qTest Resource page.
- Un-ziptheqtest-testng-agent.zip file and move these jar files below to your project’s libs folder.
- Modify qtest.properties (if necessary)
#define protocol on your qTest site (using uppercase string)
#if qTest OnPremise is setup without a ssl certificate, use HTTP protocol client.protocol=HTTPS
#your qTest site domain
#token is generated as soon as logging successfully in qTest site(refer to Hint)
# the project ID in qTest site(refer step 1.6)
- In the API &SDKsectioninqTest Resources page, get the tokenoflogin user to update in qtest.properties file.
- Go to Project's Test Plan menu and gettheprojectId from the URL
2. qTest Automation Integration with different Testing Frameworks
You can either build up qTest Automation Integration using either TESTNG or other frameworks
A. Configure qTest automation integration in a project built in the Selenium, TestNG framework, using Maven, ANT, Java, Eclipse Editor.
a) Make sure your project developed in the Selenium, TestNG framework has been run correctly. (e.g.: your project located in D:\SampleProject )
b) Copy qtest.properties into your root folder project (e.g.: D:\SampleProject\qtest.properties)
c) Copy all jar files from the qtest-testng-agent.zip file and put into your LIBS folder (e.g.: D:\SampleProject\resources\libs)
- Build qTest Automation Integration using MAVEN
- Add some dependencies and plugin into your pom.xml file
- Build your project by using command line: mvn -DskipTests=false compile package or CI tools (e.g.: Jenkins)
Important: You can use either of below Test Case listeners
- Real time listerner (i.e. com.qas.qtest.agent.testng.listener.TestCaseRealTimeListerner): to return result of each Test Run right after finishing its execution.
- Normal listener (i.e. com.qas.qtest.agent.testng.listener.TestCaseListener): to return results of all Test Runs after finishing the entire Test Suite execution.
- Build qTest Automation Integration using ANT
- Modify your ANT build.xml
- Build your project by using command line: ant –f build.xml
- Run your project by Java command line
- Make sure your project integrate with qTest Automation has been complied into a JAR file.
- Set up an environment that can run TestNG framework. It would contain your script, your all libraries (include jar files to be compiled from your project) and testng.xml file.
- Run command below to execute your test case in Console by java command line
- Run your project using TestNG framework by Eclipse editor
- Make sure qTest library has been imported into your project first
- Right click on your project and select Properties
- Select Java Build Path -> click Libraries tab -> click Add JARS button -> select qTest libraries from your local folder to import.
- Verify qTest libraries has been imported into your project properly in Referenced Libraries.
- Add listener element into your TestNG XML file
- Compile your project and execute your script.
B.Configure qTest automation integration using other frameworks, such as .NET and APIs
- Go to Download qTest Resources page
- Copy qTest SDK into your library
- Implement new class to integrate between your project with qTest Automation through API document (http://support.qasymphony.com/entries/46439048-qTest-API-v3-Specification)