Table of content

• Run Failed TestCases in TestNG
• Retry Analyzer in TestNG
• Steps to Run Failed TestCases in TestNG

You may have more than 200 test cases on your regression test suite; sometimes, a few test cases may fail. Below reasons may cause these failures.

• 1. Element not present
• 2. Element is not visible
• 3. Element not clickable
• 4. Browser slowness/responsiveness

But we cannot run the total test suite again because we have failures, before coming to the conclusion we have to be sure about the failures are not flaky failures.

Every time tests fail in a suite, TestNG creates a file called testng-failed.xml in the test-output directory.

the testng-failed.xml file contains all the information to rerun only these methods that failed, allowing you to quickly reproduce the failures without having to run the entirety of your tests.

Program with failures

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.testng.annotations.Test;
public class TestNGClass
{
	WebDriver driver=new FirefoxDriver();
	@Test
	public void openSeleniumMentor()
	{
		driver.get("https://chercher.tech");
	}
	@Test
	public void openGoogle()
	{
		//fails
		driver.get("www.google.com");
	}
	@Test
	public void openGmail()
	{
		//fails
		driver.get("gmail.com");
	}
}

Dropdown Option selected or Not in selenium

What is a retry analyzer ?
When a test fails, the retry() method gets called, and the retry() method will result in either true or false.

If we have a test method that is annotated with retryAnalyzer along with @Test, so based on the result from the retry() method @Test method decides whether it should try or not.

If retry() results in true then @Test method will try to re-execute the test method. if results false then test method will not be re-executed

retry() receives a parameter for TestResult, this will contain whether the test is passed or not and test method names, based on this increment the counter in RetryFailedTest.

• Create a class called RetryFailedTest which should implement IRetryAnalyzer
• Initialize a two variables maxTries=5, and currentTry=0
• Override the method called retry() and write condition where it should enter to if block only if maxTries > currentTry
• Inside if block returns true; after if block return false.

public class TestJenkins {
	@Test(retryAnalyzer = RetryFailedTest.class)
	public static void main() {
		fail();
	}
}

import org.testng.IRetryAnalyzer;
import org.testng.ITestResult;
public class RetryFailedTest implements IRetryAnalyzer{
	int currentTry = 0;
	int maxTries = 5;
	@Override
	public boolean retry(ITestResult result) {
		if(currentTry < maxTries)
		{
			System.out.println(result.getName()+ " test will be retired");
			// increase the currentTry
			currentTry++;
			return true;
		}
		System.out.println(result.getName()+ " test will not be retired");
		return false;
	}
}

In the above result, You might see the last test is failed, but the remaining steps are marked as skipped.

1. After execution on the "Result of Running Class" you see the details like PASSED and Failed, Skipped, and time taken to execute.


2.Refresh the project and open the test-output folder
3. Now open the index, it might look like below with pass and failed details


4. Now you could see the testng-failed.xml file in the test-output folder


5. Now modify the program like below

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.testng.annotations.Test;
public class TestNGClass
{
	WebDriver driver=new FirefoxDriver();
	@Test
	public void openSeleniumMentor()
	{
		driver.get("https://chercher.tech");
	}
	@Test
	public void openGoogle()
	{
		//fails
		driver.get("https://www.google.com");
	}
	@Test
	public void openGmail()
	{
		//fails
		driver.get("gmail.com");
	}
}

5. Now go to eclipse and right-click on the testng-failed.xml


6.Select Run As->TestNG Test


7. Now it starts executing only those failed script, and it skips the passed scenario. If you wonder why the look into the "testng-failed.xml"" it contains only the failed tests


8. Now check the "Result from Running Class."


Take Screenshots of failed tests selenium

Recommended Readings

• Data Provider in TestNG with Selenium Webdriver | @DataProvider
• Run TestNG From CMD Selenium | java -jar org.testng.TestNg
• testng.xml in TestNG with Selenium| testng.xml
• WindowsUtils & Zip class in Selenium
• TestNG Listeners with Selenium
• Assertions in TestNG with Selenium
• Class level Annotations in TestNG Selenium
• @Test Attributes in TestNG with Selenium

I am Pavankumar, Having 8.5 years of experience currently working in Video/Live Analytics project.