How to set Jenkins Job status based on Test Case Pass % ?

We all love to run our tests from Jenkins ! Its quite easy to run your test script from the Jenkins command line execution. What is “not so easy” is setting the status of the Job depending on what percentage of your tests have passed.

For example: There can be a type of Automation test which needs to be 100% passed in order for the Jenkins job to pass. In other cases, that threshold may be 80%.What we intend to do is to figure out a testing framework agnostic way of doing this.

Following is my shot at solving this problem:-

  1. Add a Jenkins Hidden Parameter to your Job through the Hidden Parameter Plugin:-

  1.  Run your tests from the command line. After the execution finishes, just make sure you print the Test Run specific information on the console(you might need to write some code here if your testing framework does not provide this on the console), something like – 98% Tests Passed. For the sake of simplicity, I will just print the message directly:-

 

Note the “exit /b 0” command, this will make sure your batch command does not decide the fate of your Jenkins job. It simply means “whatever happens in the command, always return a status code of 0.

  1. Using the PostBuild Groovy Plugin, analyse the results from the console and set the job status. The Groovy code for this is:-

import hudson.model.*
def matcher = manager.getLogMatcher("(.*)% Tests Passed\$")

if (matcher?.matches()) {
 testinfo=matcher.group(1) + "% Tests Passed"
 
 manager.build.addAction(
 new ParametersAction([
 new StringParameterValue("TEST_INFO", testinfo)
 ]))
 
 manager.addShortText(testinfo) 

 def intValue = matcher.group(1) as int
 if(intValue>95){
 manager.listener.logger.println("More than 95% test method passed.Build is marked as pass")
 manager.buildSuccess()
 manager.addBadge("green.gif",testinfo)
 }
 else if(intValue>90 && intValue<=95){
 manager.listener.logger.println("Between 90 to 95% test method passed.Build is marked as unstable")
 manager.buildUnstable()
 manager.addBadge("yellow.gif",testinfo)
 }
 else{
 manager.listener.logger.println("Less than 90% test method passed.Build is marked as fail")
 manager.buildFailure()
 manager.addBadge("red.gif",testinfo)
 }
}
else{ 
 manager.listener.logger.println("Overall Build is marked as fail")
 manager.buildFailure()
 manager.addBadge("red.gif",testinfo)
}

A better way of doing this can be to put the above code in a .groovy file and mention it in the job like this:-

 

  1. Going one step ahead, we can also pass the variable TEST_INFO’s value to the Email Subject just to make our subject more informative:-

 

Happy Automating!

Harshit Kohli

Tagged with: , , , , , , ,
Posted in Awesum Stuff

Jenkins Training Programme

Jenkins is THE first tool that comes in mind when you talk about Continuous Integration and Deployment and there is no reason why it shouldn’t ! It is by far the most used CI/CD tool in the industry and some points that justify this fact are:-

  1. It is Open Source and Cross Platform.
  2. Its Easy to use and Setup.
  3. It can be scaled up through the Master-Slave Architecture.
  4. Thousands of plugins that almost have all your requirements sorted.
  5. Great Community support – Jenkins is present everywhere from Twitter to Quora!

Jenkins is a great addition to your arsenal for speeding up your project delivery process. If you want to master the craft of CI/CD, take up our Jenkins Training Programme for corporates as well as individuals !

For more details, reach out to kohli_harshit@hotmail.com.

Tagged with: , , , , , ,
Posted in Trainings

Docker Training Programme

Containerization has been rocking the IT world from some time now with Docker leading the way. Docker basically answers questions like – “Its working on Machine 1 but not working on Machine 2”. This magic happens because with Docker, everything is a Container(read self contained and isolated virtual environment). Some of the benefits of using Docker are:-

  1. Fast – Spinning up a new environment for Development/Testing is a matter of minutes!
  2. Isolated – We can have multiple isolated environments hosted on a single docker engine(even 10-15 containers on a machine work like a charm)
  3. Continuous Deployment and Testing – Its a boon to the DevOps world because it addresses some basic problems like Environment consistency  and dependency management.
  4. Cloud Support – All major cloud computing providers now have docker support so that containers can now be deployed directly on the cloud.
  5. Version Control – Just like GitHub, you have a DockerHub where you can maintain your images.

With our Docker Training programme, Developers/Testers/DevOps professionals can quickly grab Docker concepts and start implementing containers in their day to day routine. Individuals can learn online using Skype at their own pace and Corporate Trainings are undertaken in batches.

For more details, reach out to kohli_harshit@hotmail.com.

 

Tagged with: , , , , , ,
Posted in Trainings

TestNG – Using @DataProvider and @Factory

TestNG gives you two awesome annotations to implement Data Driven tests. Lets look at them one by one:-

  1. @DataProvider – This annotation can be used to denote that the method will be used for supplying data to a test method. The condition that needs to be met here is that the method marked as @DataProvider must return a 2D Object array (Object[][]) where each Object[] will be used as the input parameter to an iteration of the test method which uses the data provider. Please note that there is no direct way of linking an Excel sheet or CSV to a Data Provider. In the following example, I will be using Apache POI to read an excel to provide the data to my Test Method:-

@DataProvider(name="SearchData")
public Object[][] GetSearchData() {
//Returning 2D array of Test Data
Object[][] arrayObject = readFromExcel("C:/InputData.xls","Sheet1");
return arrayObject;
}

public String[][] readFromExcel(String fileName, String sheetName) {
String[][] arrayExcelData = null;
try {

//Read the data from the Excel Sheet
FileInputStream fs = new FileInputStream(fileName);
Workbook wb = Workbook.getWorkbook(fs);
Sheet sh = wb.getSheet(sheetName);

int totalNoOfCols = sh.getColumns();
int totalNoOfRows = sh.getRows();
arrayExcelData = new String[totalNoOfRows-1][totalNoOfCols];

for (int i= 1 ; i < totalNoOfRows; i++) {

for (int j=0; j < totalNoOfCols; j++) {
arrayExcelData[i-1][j] = sh.getCell(j, i).getContents();
}

}
} catch (Exception e) {
e.printStackTrace();
}
return arrayExcelData;
}

//Using the Data Provider in the Test Method
@Test(dataProvider = "SearchData")
public void SearchData(String searchTerm)
{
//use searchTerm in the test case. The Test method will run as many times as the
//rows in the excel(with searchTerm having different value everytime)
}

2. @Factory – This is another way of achieving data driven testing using testng.
This feature allows all methods in a class to be run on multiple sets of data. Contrary
to the @DataProvider annotation, this can be used to instantiate a class multiple times
rather than just a method. The factory method should return an Object[]. This can be an
array of Method calls or class objects. We can run the Factory class directly. Here’s a
sample code that shows this:-


public class FactoryTest
{
private string searchValue;

public FactoryTest(string searchValue) {
this.searchValue= searchValue;
}

@Test
public void testMethodOne() {
 //do something with searchValue
}

@Test
public void testMethodTwo() {
 //do something else with searchValue
}
}

public class FactoryClass
{
@Factory
public Object[] factoryMethod() {
//Return an array of Objects of Factory Test Class, each with a different search value
return new Object[] { new FactoryTest("SearchTerm1"), new FactoryTest("SearchTerm1") };
}
}

In a nutshell, @DataProvider gives you the power to run a test method with different sets of data and @Factory gives you the power to run all methods inside a test class with different sets of data. Though you can also run test methods with @Factory also, it depends on your use case as to which approach fits it better. Please comment in case the difference between the two is not clear 🙂

Happy Automating!
Harshit Kohli

Tagged with: , , , , , ,
Posted in Awesum Stuff

How to connect to Mobile Center Server

In the previous post, I explained the process to install Mobile Center Server. For Mobile Center Introduction and its Architecture, please click on the respective links.

Since now Mobile Center Server is installed, we should be able to connect to the Server.

To check if we are able to connect to the server, please navigate to Mobile Center Server URL that I asked to note in the previous post.

mc_homepage.png

Now, login with default username (admin@default.com) and the password that you provided while installing Mobile Center Server. It will show 3 tabs:

  • Apps
  • Devices
  • Administration

moc_homepage_afterLogin.png

If you are not getting the login page for Mobile Center, then there can be 2 issues:

  • The Mobile Center Server is not installed properly
  • Either of the 2 services that got installed with Mobile Center Server are not started (snapshot below)

mc_services.png

 

Please follow next post in this tutorial series to configure Connector machine (either separate machine or the machine on which Mobile Center Server is installed)

Tagged with: , , , , , ,
Posted in Mobile

How to install and configure Mobile Center Server

In the previous posts, I introduced Mobile Center along with its architecture.

To use Mobile Center, we need to do at least following steps:

  • Install Mobile Center Server
  • Install Mobile Center Connector (only if we want to use separate machine as a Connector)
  • Add Device (Android/IPhone or Emulator)

In this post, we will be looking at the process of installing Mobile Center Server. Now, Mobile Center Server can be installed either on Linux or Windows machine. Windows 7 64-bit is supported. For additional requirements (on both Linux and Windows), please refer System Requirements.

Prerequisites: Install USB drivers for Android Device Manufacturer

Please follow the following steps to install Mobile Center Server:

  • Download Mobile Center Server software from here. You need to click on Download Now button and then register on the website on get the download link
  • Downloaded file will be an ISO file. Extract the contents.
  • Log in to your machine as administrator.
  • Go to Server folder in extracted contents.
  • Choose Run as Administrator after right clicking on install_server-windows-x64-2.01.00.00-352.exe
  • The installation wizard opens. Click Next button until Device requirements Screen

intro_mobile

  • In the Device Configuration screen, select Device Platform for which drivers are installed on the machine

Device_Config.PNG

  • In the Server configuration screen:
    • Enter the IP address or the FQDN of the Mobile Center server machine.
    • Accept or edit the default port 8080.
    • Select SSL only to work with a secure connection using HTTPS
    • Accept or edit the default internal port, 8081
    • Provide a password for the default administrator user, admin@default.com, for accessing the Mobile Center Lab Management console.

Server_Config.PNG

  • In the PostgreSQL configuration screen:
    • Enter a password for the database’s built-in superuser, postgres, created when the PostgreSQL server is installed.
    • Accept or edit the default username for the database administrator, hpmcadmin, and provide a password

postgre.png

  • Installation Summary will be shown after clicking on Next button

ins_summary.PNG

  • Please note Mobile Center Server URL in the above snapshot as we will be using this to connect to it (shown in the next post)
  • When the installation is complete, the wizard displays a message that the server was successfully installed, and provides an option, Start the service.
  • Click Next to start the Mobile Center service.

Please follow next tutorial to connect to Mobile Center Server and access Lab Console

Tagged with: , , , , , , , , , ,
Posted in Mobile

Mobile Center Architecture

Those who are new to Mobile Center, please refer to my previous post about Mobile Center Introduction. We will be covering the high level overview and its architecture in this post.

Broadly, Mobile Center has 2 components:

  1. Mobile Center Server
  2. Mobile Center Connector

Mobile Center Server provides:

  • connection between testing tools (UFT, LeanFT etc) and Mobile Devices
  • User Interface to record and run tests on Real Devices
  • Lab Console to manage devices and users
  • PostgreSQL database storing reusable data, such as Metadata of uploaded apps and connectors, certificates, user information, and so forth

Mobile Center Connector allows communication between Mobile Devices and Mobile Center Server. It is installed automatically with Mobile Center Server, however, it can also be installed as standalone component on different machines.

Following figure shows the Mobile Center components and depicts the relationship between them:

Following connector deployment scenarios are supported:

  1. Devices can be connected to a machine on which Mobile Center Server is installed (as Connector is automatically installed with Server)
  2. Connector can be installed as standalone component on several machines in distributed locations

In the next post, we will be looking into the installation of Mobile Center Server component.

Tagged with: , , , , , , , , , ,
Posted in Mobile
Blog Stats
  • 162,350 hits
Categories

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 792 other followers

%d bloggers like this: