TestNG Report Generation in Selenium WebDriver

TestNG Report: In our earlier post regarding TestNG, we saw many powerful features of TestNG. By using all those features, you can create an Automation Framework. However, as we discussed the TestNG framework, we have mentioned that multiple factors play an important role in creating a robust framework.

One is the Report, which informs you which test case is passed and which failed and helps you discover the potential bugs. So, you should clearly know how you will generate the reports using the Selenium WebDriver.

Before starting anything about Report, let’s understand:

Why do we need reporting?

We try to perform some application operations when we execute our test cases using any automation tool or Selenium WebDriver. As automation testers, we must test the application by executing the test cases and reporting the status to higher management or the development team if we find any bugs. That’s why the Report is one of the essential factors of software testing.

Benefits Of Report

• The report lets you easily determine how many test cases got passed, failed, or skipped.
• Seeing the report, you will know the project’s status.

So For Report generation, using different types of reports as per their organization is suitable, like:

But in this post, we will learn how to generate a TestNG Report.

TestNG Report

Selenium WebDriver does not have a built-in feature for generating reports. But if we use plugins like TestNG and JUnit, we can add this functionality to Selenium WebDriver and get the report.

Once we execute our script using TestNG, TestNg generates a test output folder in the project’s root directory. Which has two types of reports on that:

• Detailed Report (index.html)
• Summary Report (emailable-report.html)

Detailed Report:

This report has all the information like errors, Test Groups, Execution Time, Step by Step Logs, and The TestNG XML File. This report is in the output folder; the file name is index.html.

Summary report:

This report does not give all the information; you can say it is a trimmed version of a detailed description, and it has information like the number of tests that are “passed/Failed/Skipped.”

This report is also available inside the output folder and is a shareable report you can share with your stakeholders. You can find this report in the name of emailable-report.html.

Steps to Generate Reports Using TestNG:

• Select The Proper Listener Interface.
• Create the Listener Class
• Create The Test Class
• Create the TestNg.XMl File and Run For Generate the Report.

When we discussed the listeners, we mentioned two interfaces are available; we can create the Test Report using those.

• Commonly Used Interface ITestListener
• Another Method is using the IReporter Interface.

This post will discuss the widely used interface, the ITestListener interface. You can refer to the listener class below, the test class, and the Testng.xml file.

Listener Class:

package com.softwaretestingo.testng.listeners;
import org.testng.ITestContext;
import org.testng.ITestListener;
import org.testng.ITestResult;
public class ListenerImplementedClass implements ITestListener 
{
	public void onTestStart(ITestResult result) 
	{
		System.out.println("New Test Started: " + result.getName());
	}

	public void onTestSuccess(ITestResult result) 
	{
		System.out.println("Test Successfully Finished: " + result.getName());
	}

	public void onTestFailure(ITestResult result) 
	{
		System.out.println("Test Failed: " + result.getName());
	}

	public void onTestSkipped(ITestResult result) 
	{
		System.out.println("Test Skipped: " + result.getName());
	}

	public void onTestFailedButWithinSuccessPercentage(ITestResult result) 
	{
		System.out.println("Test Failed but within success percentage: " + result.getName());
	}

	public void onStart(ITestContext context) 
	{
		System.out.println("This is onStart method: " + context.getOutputDirectory());
	}

	public void onFinish(ITestContext context) 
	{
		System.out.println("This is onFinish method: " + context.getPassedTests());
		System.out.println("This is onFinish method: " + context.getFailedTests());
	}
}

package com.softwaretestingo.testng.listeners;
import org.testng.ITestContext;
import org.testng.ITestListener;
import org.testng.ITestResult;
public class ListenerImplementedClass implements ITestListener 
{
	public void onTestStart(ITestResult result) 
	{
		System.out.println("New Test Started: " + result.getName());
	}

	public void onTestSuccess(ITestResult result) 
	{
		System.out.println("Test Successfully Finished: " + result.getName());
	}

	public void onTestFailure(ITestResult result) 
	{
		System.out.println("Test Failed: " + result.getName());
	}

	public void onTestSkipped(ITestResult result) 
	{
		System.out.println("Test Skipped: " + result.getName());
	}

	public void onTestFailedButWithinSuccessPercentage(ITestResult result) 
	{
		System.out.println("Test Failed but within success percentage: " + result.getName());
	}

	public void onStart(ITestContext context) 
	{
		System.out.println("This is onStart method: " + context.getOutputDirectory());
	}

	public void onFinish(ITestContext context) 
	{
		System.out.println("This is onFinish method: " + context.getPassedTests());
		System.out.println("This is onFinish method: " + context.getFailedTests());
	}
}

Test Class:

package com.softwaretestingo.testng.listeners;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.Assert;
import org.testng.SkipException;
import org.testng.annotations.Listeners;
import org.testng.annotations.Test;
//@Listeners(com.softwaretestingo.testng.listeners.ListenerImplementedClass.class)
public class TestNGListenersTest 
{
	@Test //Passing Test
	public void sampleTest1() throws InterruptedException
	{
		WebDriver driver = new ChromeDriver();
		driver.get("https://opencart.softwaretestingo.com/");
		driver.manage().window().maximize();
		driver.findElement(By.linkText("My Account")).click();
		Thread.sleep(2000);
		driver.findElement(By.linkText("Login")).click();
		Thread.sleep(2000);
		driver.findElement(By.xpath("(//button)[4]")).click();
		Thread.sleep(2000);
		driver.quit();
	}
	@Test //Failing Test
	public void sampleTest2() throws InterruptedException
	{
		System.out.println("Forcely Failed Test Method");
		Assert.assertTrue(false);
	}
	private int i = 0;
	//Test Failing But Within Success Percentage
	@Test(successPercentage = 60, invocationCount = 5) 
	public void sampleTest3() 
	{
		i++;
		System.out.println("Test Failed But Within Success Percentage Test Method, invocation count: " + i);
		if (i == 1 || i == 2)
		{
			System.out.println("sampleTest3 Failed");
			Assert.assertEquals(i, 6);
		}
	}
	//Skipping Test
	@Test 
	public void sampleTest4()
	{
		throw new SkipException("Forcely skipping the sampleTest4");
	}
}

package com.softwaretestingo.testng.listeners;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.Assert;
import org.testng.SkipException;
import org.testng.annotations.Listeners;
import org.testng.annotations.Test;
//@Listeners(com.softwaretestingo.testng.listeners.ListenerImplementedClass.class)
public class TestNGListenersTest 
{
	@Test //Passing Test
	public void sampleTest1() throws InterruptedException
	{
		WebDriver driver = new ChromeDriver();
		driver.get("https://opencart.softwaretestingo.com/");
		driver.manage().window().maximize();
		driver.findElement(By.linkText("My Account")).click();
		Thread.sleep(2000);
		driver.findElement(By.linkText("Login")).click();
		Thread.sleep(2000);
		driver.findElement(By.xpath("(//button)[4]")).click();
		Thread.sleep(2000);
		driver.quit();
	}
	@Test //Failing Test
	public void sampleTest2() throws InterruptedException
	{
		System.out.println("Forcely Failed Test Method");
		Assert.assertTrue(false);
	}
	private int i = 0;
	//Test Failing But Within Success Percentage
	@Test(successPercentage = 60, invocationCount = 5) 
	public void sampleTest3() 
	{
		i++;
		System.out.println("Test Failed But Within Success Percentage Test Method, invocation count: " + i);
		if (i == 1 || i == 2)
		{
			System.out.println("sampleTest3 Failed");
			Assert.assertEquals(i, 6);
		}
	}
	//Skipping Test
	@Test 
	public void sampleTest4()
	{
		throw new SkipException("Forcely skipping the sampleTest4");
	}
}

TestNG.XML File

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite">
	<listeners>
		<listener class-name="com.softwaretestingo.testng.listeners.ListenerImplementedClass"></listener>
	</listeners>
	<test thread-count="5" name="Test">
		<classes>
			<class
				name="com.softwaretestingo.testng.listeners.TestNGListenersTest" />
		</classes>
	</test> <!--
	Test -->
</suite> <!--
Suite -->

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite">
	<listeners>
		<listener class-name="com.softwaretestingo.testng.listeners.ListenerImplementedClass"></listener>
	</listeners>
	<test thread-count="5" name="Test">
		<classes>
			<class
				name="com.softwaretestingo.testng.listeners.TestNGListenersTest" />
		</classes>
	</test> <!--
	Test -->
</suite> <!--
Suite -->

Output:

This is onStart method: D:\Workspace\Automation\SeleniumPractice\test-output\Suite
New Test Started: sampleTest1
Nov 16, 2023 10:07:48 AM org.openqa.selenium.devtools.CdpVersionFinder findNearestMatch
WARNING: Unable to find an exact match for CDP version 119, so returning the closest version found: 117
Test Successfully Finished: sampleTest1
New Test Started: sampleTest2
Forcely Failed Test Method
Test Failed: sampleTest2
New Test Started: sampleTest3
Test Failed But Within Success Percentage Test Method, invocation count: 1
sampleTest3 Failed
Test Failed but within success percentage: sampleTest3
New Test Started: sampleTest3
Test Failed But Within Success Percentage Test Method, invocation count: 2
sampleTest3 Failed
Test Failed but within success percentage: sampleTest3
New Test Started: sampleTest3
Test Failed But Within Success Percentage Test Method, invocation count: 3
Test Successfully Finished: sampleTest3
New Test Started: sampleTest3
Test Failed But Within Success Percentage Test Method, invocation count: 4
Test Successfully Finished: sampleTest3
New Test Started: sampleTest3
Test Failed But Within Success Percentage Test Method, invocation count: 5
Test Successfully Finished: sampleTest3
New Test Started: sampleTest4
Test Skipped: sampleTest4
This is onFinish method: [ResultMap map=[[TestResult name=sampleTest1 status=SUCCESS method=TestNGListenersTest.sampleTest1()[pri:0, instance:com.softwaretestingo.testng.listeners.TestNGListenersTest@19fb8826] output={null}], [TestResult name=sampleTest3 status=SUCCESS method=TestNGListenersTest.sampleTest3()[pri:0, instance:com.softwaretestingo.testng.listeners.TestNGListenersTest@19fb8826] output={null}], [TestResult name=sampleTest3 status=SUCCESS method=TestNGListenersTest.sampleTest3()[pri:0, instance:com.softwaretestingo.testng.listeners.TestNGListenersTest@19fb8826] output={null}], [TestResult name=sampleTest3 status=SUCCESS method=TestNGListenersTest.sampleTest3()[pri:0, instance:com.softwaretestingo.testng.listeners.TestNGListenersTest@19fb8826] output={null}]]]
This is onFinish method: [ResultMap map=[[TestResult name=sampleTest2 status=FAILURE method=TestNGListenersTest.sampleTest2()[pri:0, instance:com.softwaretestingo.testng.listeners.TestNGListenersTest@19fb8826] output={null}]]]

===============================================
Suite
Total tests run: 8, Passes: 4, Failures: 3, Skips: 1
===============================================

This is onStart method: D:\Workspace\Automation\SeleniumPractice\test-output\Suite
New Test Started: sampleTest1
Nov 16, 2023 10:07:48 AM org.openqa.selenium.devtools.CdpVersionFinder findNearestMatch
WARNING: Unable to find an exact match for CDP version 119, so returning the closest version found: 117
Test Successfully Finished: sampleTest1
New Test Started: sampleTest2
Forcely Failed Test Method
Test Failed: sampleTest2
New Test Started: sampleTest3
Test Failed But Within Success Percentage Test Method, invocation count: 1
sampleTest3 Failed
Test Failed but within success percentage: sampleTest3
New Test Started: sampleTest3
Test Failed But Within Success Percentage Test Method, invocation count: 2
sampleTest3 Failed
Test Failed but within success percentage: sampleTest3
New Test Started: sampleTest3
Test Failed But Within Success Percentage Test Method, invocation count: 3
Test Successfully Finished: sampleTest3
New Test Started: sampleTest3
Test Failed But Within Success Percentage Test Method, invocation count: 4
Test Successfully Finished: sampleTest3
New Test Started: sampleTest3
Test Failed But Within Success Percentage Test Method, invocation count: 5
Test Successfully Finished: sampleTest3
New Test Started: sampleTest4
Test Skipped: sampleTest4
This is onFinish method: [ResultMap map=[[TestResult name=sampleTest1 status=SUCCESS method=TestNGListenersTest.sampleTest1()[pri:0, instance:com.softwaretestingo.testng.listeners.TestNGListenersTest@19fb8826] output={null}], [TestResult name=sampleTest3 status=SUCCESS method=TestNGListenersTest.sampleTest3()[pri:0, instance:com.softwaretestingo.testng.listeners.TestNGListenersTest@19fb8826] output={null}], [TestResult name=sampleTest3 status=SUCCESS method=TestNGListenersTest.sampleTest3()[pri:0, instance:com.softwaretestingo.testng.listeners.TestNGListenersTest@19fb8826] output={null}], [TestResult name=sampleTest3 status=SUCCESS method=TestNGListenersTest.sampleTest3()[pri:0, instance:com.softwaretestingo.testng.listeners.TestNGListenersTest@19fb8826] output={null}]]]
This is onFinish method: [ResultMap map=[[TestResult name=sampleTest2 status=FAILURE method=TestNGListenersTest.sampleTest2()[pri:0, instance:com.softwaretestingo.testng.listeners.TestNGListenersTest@19fb8826] output={null}]]]

===============================================
Suite
Total tests run: 8, Passes: 4, Failures: 3, Skips: 1
===============================================