alwaysRun Attribute In TestNG

alwaysRun Attribute In TestNG: While discussing different annotation TestNG, we have mentioned many attributes there. Out of them, alwaysRun is one of them.

In this post, we will discuss in detail the TestNG attribute & how to use it with @test Annotation.

As per the TestNg official site statement

For before methods (beforeSuite, beforeTest, beforeTestClass, and beforeTestMethod, but not beforeGroups): If set to true, this configuration method will run regardless of what groups it belongs to.

For after methods (afterSuite, afterClass): If set to true, this configuration method will be run even if one or more methods invoked previously failed or were skipped.

The Above statement is applicable for these annotations @BeforeSuite, @AfterSuite, @BeforeTest, @AfterTest, @BeforeGroups, @AfterGroups, @BeforeClass, @AfterClass, @BeforeMethod & @AfterMethod.

And for @Test Annotation: If the alwaysRun attribute is true, this test method will always be run, even if it depends on a failed method.

Now let us try to understand by following with some simple programs:

package com.softwaretestingo.testng.attributes;
import org.testng.annotations.Test;
public class AlwaysRunAttributeEx1 
{
	@Test
	public void method1()
	{
		System.out.println("Method 1");
	}

	@Test
	public void method2()
	{
		System.out.println("Method 2");
	}
}

Output:

Method 1
Method 2

When we run the above class, we can find that both methods were executed in the output.

Now, Let’s change things a little bit differently and try to fail the first method explicitly and check whether, after failing the first method, execution stops from that point or still executing the second method:

package com.softwaretestingo.testng.attributes;
import org.testng.Assert;
import org.testng.annotations.Test;
public class AlwaysRunAttributeEx2 
{
	@Test
	public void method1()
	{
		System.out.println("Method 1");
		// Failing test explicitly
		Assert.fail();
	}

	@Test
	public void method2()
	{
		System.out.println("Method 2");
	}
}

Output:

Method 1
Method 2
===============================================
    Default test
    Tests run: 2, Failures: 1, Skips: 0
===============================================


===============================================
Default suite
Total tests run: 2, Passes: 1, Failures: 1, Skips: 0
===============================================

Here, we learned from the output that when one method failed, it did not impact the execution of the other method of the testNG class. Still, if you look the same thing in Java when we get any error, exception, or statement fails, then the execution is stopped from that point.

Now let us go one more step ahead and try to build dependency between the methods of a TestNG class, try to fail one method explicitly, and check the output.

package com.softwaretestingo.testng.attributes;
import org.testng.Assert;
import org.testng.annotations.Test;
public class AlwaysRunAttributeEx3 
{
	@Test
	public void method1() 
	{
		System.out.println("Method 1");
		// Failing test explicitly
		Assert.fail();
	}

	// This method will run only if method1 is passed. If method1 is failed or
	// skipped, method2 will not run.
	@Test(dependsOnMethods = "method1")
	public void method2() 
	{
		System.out.println("Method 2");
	}

	// Since method1 will fail, so method2 will be skipped. Since method2 is skipped, method3 will also be skipped as
	// method3 is dependent on method2
	@Test(dependsOnMethods = "method2")
	public void method3() 
	{
		System.out.println("Method 3");
	}
}

In the above program, you can see that we have made a dependency between the methods, like the second method being dependent upon the execution results of the first method and the third method being dependent upon the execution results of the second method.

Method 1
===============================================
    Default test
    Tests run: 3, Failures: 1, Skips: 2
===============================================


===============================================
Default suite
Total tests run: 3, Passes: 0, Failures: 1, Skips: 2
===============================================

As you can see, the output when the first method failed, the other two methods got skipped. The second method depends on the first method, but as the primary method failed, the second method got skipped, and the same thing happens with the third method.

Suppose there is a scenario where other test methods should be executed irrespective of the result of the dependent method. Such things are called soft dependency.

In that case, the “alwaysRun” attribute comes into the picture, and by setting the attribute value to true, we can run other test methods when the dependent methods fail or are skipped.

Let’s Get to know how we can achieve this by using the “alwaysRun” attribute:

package com.softwaretestingo.testng.attributes;
import org.testng.Assert;
import org.testng.annotations.Test;
public class AlwaysRunAttributeEx4 
{
	@Test
	public void method1() 
	{
		System.out.println("Method 1");
		// Failing test explicitly
		Assert.fail();
	}

	@Test(dependsOnMethods = "method1", alwaysRun = true)
	public void method2() 
	{
		System.out.println("Method 2");
	}

	@Test(dependsOnMethods = "method2", alwaysRun = true)
	public void method3() 
	{
		System.out.println("Method 3");
	}
}

Now, if we run the above TestNG class, you will find that the first method failed, but other methods were successfully executed.

Method 1
Method 2
Method 3
===============================================
    Default test
    Tests run: 3, Failures: 1, Skips: 0
===============================================


===============================================
Default suite
Total tests run: 3, Passes: 2, Failures: 1, Skips: 0
===============================================

How to Use alwaysRun Attribute With Configuration Annotation?

Above, we have seen how we can use the alwaysRun method with the @test annotation, and now we will learn how to use it with configuration annotations like @BeforeXXXX or @AfterXXXX annotations.

Let’s take a simple program with the configuration annotations and explicitly failed the @beforetest annotation and check what we got in the output:

package com.softwaretestingo.testng.attributes;
import org.testng.Assert;
import org.testng.annotations.AfterClass;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.AfterSuite;
import org.testng.annotations.AfterTest;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.BeforeSuite;
import org.testng.annotations.BeforeTest;
import org.testng.annotations.Test;
public class AlwaysRunAttributeEx5 
{
	@BeforeSuite
	public void beforeSuiteMethod()
	{
		System.out.println("beforeSuite");
		// We fail this method explicitly
		Assert.fail();
	}

	@BeforeTest
	public void beforeTestMethod()
	{
		System.out.println("beforeTest");
	}

	@BeforeClass
	public void beforeClassMethod()
	{
		System.out.println("beforeClass");
	}

	@BeforeMethod
	public void beforeMethodMethod()
	{
		System.out.println("beforeMethod");

	}

	@Test
	public void testMethod()
	{
		System.out.println("test");
	}
	@AfterSuite
	public void afterSuiteMethod()
	{
		System.out.println("afterSuite");
	}

	@AfterTest
	public void afterTestMethod()
	{
		System.out.println("afterTest");
	}

	@AfterClass
	public void afterClassMethod()
	{
		System.out.println("afterClass");
	}

	@AfterMethod
	public void afterMethodMethod()
	{
		System.out.println("afterMethod");
	}
}

Output:

beforeSuite
===============================================
    Default test
    Tests run: 1, Failures: 0, Skips: 1
    Configuration Failures: 1, Skips: 6
===============================================


===============================================
Default suite
Total tests run: 1, Passes: 0, Failures: 0, Skips: 1
Configuration Failures: 1, Skips: 7
===============================================

You can see in the output that the @beforesuite annotation failed, and it skips other configuration methods and test methods, too.

In the TestNG report, you will get test and configuration execution separately.

But as we all know, @before methods are mainly used for configuration purposes, so some configuration is already completed before the failure. Still, because of the failure, other configuration methods got skipped, but before completing the execution, we should release the resource.

So to achieve that, we must run at least one @after configuration method; in that case, we can use the alwaysRun attribute with the @after configuration so that the after configuration method will execute and help us to release the blocked resources.

Sample Program:

package com.softwaretestingo.testng.attributes;
import org.testng.Assert;
import org.testng.annotations.AfterClass;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.AfterSuite;
import org.testng.annotations.AfterTest;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.BeforeSuite;
import org.testng.annotations.BeforeTest;
import org.testng.annotations.Test;
public class AlwaysRunAttributeEx6 
{
	@BeforeSuite
	public void beforeSuiteMethod()
	{
		System.out.println("beforeSuite");
		// We fail this method explicitly
		Assert.fail();
	}

	@BeforeTest
	public void beforeTestMethod()
	{
		System.out.println("beforeTest");
	}

	@BeforeClass
	public void beforeClassMethod()
	{
		System.out.println("beforeClass");
	}

	@BeforeMethod
	public void beforeMethodMethod()
	{
		System.out.println("beforeMethod");

	}

	@Test
	public void testMethod()
	{
		System.out.println("test");
	}
	@AfterSuite(alwaysRun=true)
	public void afterSuiteMethod()
	{
		System.out.println("afterSuite");
	}

	@AfterTest
	public void afterTestMethod()
	{
		System.out.println("afterTest");
	}

	@AfterClass
	public void afterClassMethod()
	{
		System.out.println("afterClass");
	}

	@AfterMethod
	public void afterMethodMethod()
	{
		System.out.println("afterMethod");
	}
}

Output:

beforeSuite
===============================================
    Default test
    Tests run: 1, Failures: 0, Skips: 1
    Configuration Failures: 1, Skips: 6
===============================================

afterSuite

===============================================
Default suite
Total tests run: 1, Passes: 0, Failures: 0, Skips: 1
Configuration Failures: 1, Skips: 6
===============================================

By following the same way, you can use the alwaysRun attribute to true for other configuration methods.