Rerunning failed tests in TestNG
To automatically rerun failed TestNG tests, you typically use a retry analyzer in combination with a listener (or via the retryAnalyzer attribute on @Test). On failure, TestNG will re-invoke the test up to a configured number of times before finally marking it as failed. Here’s how to set it up:
1. Implement an IRetryAnalyzer
Create a class that implements TestNG’s IRetryAnalyzer interface. It controls how many times to retry and when to stop retrying:
package com.example.listeners;
import org.testng.IRetryAnalyzer;
import org.testng.ITestResult;
public class RetryAnalyzer implements IRetryAnalyzer {
private int retryCount = 0;
private static final int maxRetryCount = 2; // retry up to 2 times
@Override
public boolean retry(ITestResult result) {
if (retryCount < maxRetryCount) {
retryCount++;
System.out.println("[Retry] Retrying " + result.getMethod().getMethodName()
+ " (attempt " + (retryCount + 1) + ")");
return true;
}
return false;
}
}- maxRetryCount controls how many retries beyond the initial run.
- Each time a test fails, TestNG calls retry(…); returning true tells TestNG to rerun the test.
2. Attach the Analyzer to Your Tests
There are two main ways to wire your RetryAnalyzer into TestNG:
A. Directly on the @Test
import org.testng.annotations.Test;
import com.example.listeners.RetryAnalyzer;
public class SomeTest {
@Test(retryAnalyzer = RetryAnalyzer.class)
public void flakyTest() {
// your test code that might intermittently fail
}
}Any test annotated this way will be retried up to your maxRetryCount on failure.
B. Globally via an IAnnotationTransformer
If you don’t want to sprinkle retryAnalyzer on every @Test, write an annotation transformer listener that assigns it automatically:
package com.example.listeners;
import java.lang.reflect.Constructor;
import java.lang.reflect.Method;
import org.testng.IAnnotationTransformer;
import org.testng.annotations.ITestAnnotation;
public class RetryTransformer implements IAnnotationTransformer {
@Override
public void transform(ITestAnnotation annotation,
Class testClass,
Constructor testConstructor,
Method testMethod) {
// only add retry to @Test methods (skip data providers, config methods, etc.)
if (annotation.getRetryAnalyzer() == null) {
annotation.setRetryAnalyzer(RetryAnalyzer.class);
}
}
}Then register your transformer in testng.xml:
<suite name="SuiteWithRetry">
<listeners>
<listener class-name="com.example.listeners.RetryTransformer"/>
</listeners>
<test name="AllTests">
<classes>
<class name="com.example.tests.SomeTest"/>
<!-- other test classes... -->
</classes>
</test>
</suite>Now every @Test in your suite will automatically use the retry logic.
3. Reviewing Retried Results
- When a retry succeeds, TestNG reports the test as PASSED but logs retry attempts.
- If all attempts fail, the test is marked FAILED in the final report.
For more advanced reporting, you can also implement an ITestListener to capture the retry counts or even take screenshots on each retry attempt.
4. Alternative: Maven Surefire Plugin
If you’re running tests via Maven, the Surefire plugin offers its own rerun support:
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M5</version>
<configuration>
<retryCount>2</retryCount> <!-- rerun failing tests up to 2 times -->
</configuration>
</plugin>This is a simpler, configuration-only approach—but only works when you invoke TestNG via Maven Surefire.
Summary
1. IRetryAnalyzer controls retry logic in code.
2. Attach it per-test (@Test(retryAnalyzer=…)) or globally with an IAnnotationTransformer.
3. Optionally, use Maven Surefire’s retryCount for a plugin-based rerun.
With these patterns, your suite can automatically recover from transient failures and yield more stable overall results.
