Run Failed TestCases in TestNG

You may have more than 200 test cases on your regression test suite; sometimes, a few test cases may fail. Below reasons may cause these failures.

  • 1. Element not present
  • 2. Element is not visible
  • 3. Element not clickable
  • 4. Browser slowness/responsiveness

But we cannot run the total test suite again because we have failures, before coming to the conclusion we have to be sure about the failures are not flaky failures.

Every time tests fail in a suite, TestNG creates a file called testng-failed.xml in the test-output directory.

the testng-failed.xml file contains all the information to rerun only these methods that failed, allowing you to quickly reproduce the failures without having to run the entirety of your tests.

Program with failures

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.testng.annotations.Test;

public class TestNGClass
{
	WebDriver driver=new FirefoxDriver();
	@Test
	public void openSeleniumMentor()
	{
		driver.get("https://chercher.tech");
	}

	@Test
	public void openGoogle()
	{
		//fails
		driver.get("www.google.com");
	}
	@Test
	public void openGmail()
	{
		//fails
		driver.get("gmail.com");
	}
}

Dropdown Option selected or Not in selenium

Retry Analyzer in TestNG

What is a retry analyzer ?
When a test fails, the retry() method gets called, and the retry() method will result in either true or false.

If we have a test method that is annotated with retryAnalyzer along with @Test, so based on the result from the retry() method @Test method decides whether it should try or not.

If retry() results in true then @Test method will try to re-execute the test method. if results false then test method will not be re-executed

retry() receives a parameter for TestResult, this will contain whether the test is passed or not and test method names, based on this increment the counter in RetryFailedTest.

  • Create a class called RetryFailedTest which should implement IRetryAnalyzer
  • Initialize a two variables maxTries=5, and currentTry=0
  • Override the method called retry() and write condition where it should enter to if block only if maxTries > currentTry
  • Inside if block returns true; after if block return false.
Once the return statement occurs in a method, then control moves out of the method.
public class TestJenkins {
	@Test(retryAnalyzer = RetryFailedTest.class)
	public static void main() {
		fail();
	}
}
import org.testng.IRetryAnalyzer;
import org.testng.ITestResult;

public class RetryFailedTest implements IRetryAnalyzer{
	int currentTry = 0;
	int maxTries = 5;
	@Override
	public boolean retry(ITestResult result) {
		if(currentTry < maxTries)
		{
			System.out.println(result.getName()+ " test will be retired");
			// increase the currentTry
			currentTry++;
			return true;
		}
		System.out.println(result.getName()+ " test will not be retired");
		return false;
	}
}

retry-analyzer-testng

In the above result, You might see the last test is failed, but the remaining steps are marked as skipped.

Steps to Run Failed TestCases in TestNG

1. After execution on the "Result of Running Class" you see the details like PASSED and Failed, Skipped, and time taken to execute.

failed%20selenium%20webdriver

2.Refresh the project and open the test-output folder
3. Now open the index, it might look like below with pass and failed details

index%20selenium%20webdriver

4. Now you could see the testng-failed.xml file in the test-output folder

failed%20exe%20selenium%20webdriver

5. Now modify the program like below

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.testng.annotations.Test;

public class TestNGClass
{
	WebDriver driver=new FirefoxDriver();
	@Test
	public void openSeleniumMentor()
	{
		driver.get("https://chercher.tech");
	}

	@Test
	public void openGoogle()
	{
		//fails
		driver.get("https://www.google.com");
	}
	@Test
	public void openGmail()
	{
		//fails
		driver.get("gmail.com");
	}
}


5. Now go to eclipse and right-click on the testng-failed.xml

right%20failed%20xml%20selenium%20webdriver

6.Select Run As->TestNG Test

failed%20xml%20exe%20selenium%20webdriver

7. Now it starts executing only those failed script, and it skips the passed scenario. If you wonder why the look into the "testng-failed.xml"" it contains only the failed tests

console%20passed%20selenium%20webdriver

8. Now check the "Result from Running Class."

result%20a%20selenium%20webdriver

Take Screenshots of failed tests selenium

Comment / Suggestion Section
Point our Mistakes and Post Your Suggestions