Absolutely! Let’s continue building your automation framework by exploring TestNG Grouping, ExtentReports Integration, and Handling Failed Test Executions. This comprehensive guide will help you organize your tests efficiently, generate detailed reports, capture screenshots on failures, and rerun only the failed tests seamlessly.

Table of Contents

1. Introduction to TestNG Grouping

TestNG Grouping allows you to categorize your test methods into different groups, enabling selective execution based on specific criteria. This is particularly useful for organizing tests into categories like Sanity, Regression, Master, etc., allowing for flexible and efficient test runs.

Benefits of Grouping:

  • Selective Execution: Run only specific groups of tests without executing the entire test suite.
  • Organized Structure: Maintain a clear structure by categorizing tests based on functionality or priority.
  • Parallel Execution: Combine grouping with parallel execution for optimized test runs.

2. Implementing TestNG Groups in Your Test Cases

To start grouping your tests, you need to annotate your test methods with the @Test annotation’s groups attribute.

2.1. Adding Groups to Test Methods

  1. Open Your Test Classes:
    • Navigate to your test classes, e.g., LoginTest.java, AccountRegistrationTest.java, etc.
  2. Annotate Test Methods with Groups:
    • Use the groups attribute in the @Test annotation to assign groups to each test method.
    package com.opencart.tests;

import org.testng.Assert;
import org.testng.annotations.Test;

/**
 * Test Class for Account Registration.
 */
public class AccountRegistrationTest extends BaseTest {

    @Test(groups = {"regression", "master"})
    public void tc001_accountRegistrationTest() {
        // Test implementation
    }
}

    Explanation:

    • Groups Assigned:
      • "regression": Indicates that this test is part of the regression suite.
      • "master": Indicates that this test is part of the master suite, which includes all tests.
  3. Assigning Multiple Groups:
    • A single test method can belong to multiple groups.
    @Test(groups = {"sanity", "master"})
public void tc002_verifyLogin() {
    // Test implementation
}

    Explanation: This test belongs to both "sanity" and "master" groups.

  4. Creating a Data-Driven Test Group:
    • For data-driven tests, you might want to assign them to a separate group like "dataDriven".
    @Test(dataProvider = "accountRegistrationData", dataProviderClass = DataProviderUtil.class, groups = {"dataDriven", "master"})
public void tc003_accountRegistrationDataDrivenTest(String firstName, String lastName, String email, String telephone, String password) {
    // Test implementation
}

    Explanation: This test is part of both "dataDriven" and "master" groups.

2.2. Updating the Base Class with Group Annotations

To ensure that setup and teardown methods are associated with specific groups, annotate them accordingly.

package com.opencart.base;

import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeClass;

public class BaseTest {
    protected static WebDriver driver;

    @BeforeClass(groups = {"sanity", "regression", "master", "dataDriven"})
    public void setUp() {
        // Setup implementation
    }

    @AfterClass(groups = {"sanity", "regression", "master", "dataDriven"})
    public void tearDown() {
        // Teardown implementation
    }
}

Explanation:

  • The setUp and tearDown methods are associated with all relevant groups to ensure they are executed during group-specific test runs.

3. Configuring TestNG XML for Group Execution

TestNG XML files define how tests are executed. By configuring groups in the XML, you can specify which groups to include or exclude during a test run.

3.1. Creating a Grouping TestNG XML File

  1. Create a New TestNG XML File:
    • Right-click on your project > New > File.
    • Name it grouping.xml and place it in your project root or a designated folder.
  2. Define Groups in the XML File: 
    <?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="GroupingSuite">

    
    <listeners>
        <listener class-name="com.opencart.utilities.ExtentReportManager"/>
    </listeners>

    
    <test name="GroupingTests">
        <groups>
            <run>
                
                <include name="sanity"/>
                <include name="regression"/>
                <include name="master"/>
                <include name="dataDriven"/>
            </run>
        </groups>

        
        <classes>
            <class name="com.opencart.tests.LoginTest"/>
            <class name="com.opencart.tests.AccountRegistrationTest"/>
            
        </classes>
    </test>

</suite>

    Explanation:

    • Listeners: Integrate ExtentReportManager to handle report generation.
    • Groups: Specify which groups to include during this test run.
    • Classes: List all test classes that contain grouped tests.

3.2. Running Specific Groups

To run only specific groups of tests, modify the <include> tags in your testng.xml file.

  • Sanity Group: 
    <include name="sanity"/>

    To run only the “sanity” group.

  • Regression Group: 
    <include name="regression"/>

    To run only the “regression” group.

  • Master Group: 
    <include name="master"/>

    To run all tests under the “master” group.

  • Multiple Groups: 
    <include name="sanity"/>
<include name="regression"/>

    To run both “sanity” and “regression” groups.

3.3. Excluding Groups

Sometimes, you might want to include certain groups but exclude others.

<groups>
    <run>
        <include name="sanity"/>
        <exclude name="regression"/>
    </run>
</groups>

Explanation:

  • This configuration will run tests in the “sanity” group but exclude those in the “regression” group.

4. Setting Up ExtentReports with Enhanced Features

ExtentReports is a popular reporting library that provides visually appealing and detailed reports. We’ll enhance its setup to include dynamic report naming, category information, and more.

4.1. Creating ExtentReportManager Utility Class

  1. Navigate to Utilities Package:
    • Go to src/test/java/com/opencart/utilities.
  2. Create ExtentReportManager.java:
    • Right-click on utilities > New > Class.
    • Name the class ExtentReportManager and click Finish.
  3. Implement ExtentReportManager Class: 
    package com.opencart.utilities;

import com.aventstack.extentreports.ExtentReports;
import com.aventstack.extentreports.ExtentTest;
import com.aventstack.extentreports.reporter.ExtentSparkReporter;
import com.aventstack.extentreports.reporter.configuration.Theme;
import org.testng.ITestContext;
import org.testng.ITestListener;
import org.testng.ITestResult;

import java.io.File;
import java.io.IOException;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.List;

/**
 * Listener class for ExtentReports integration.
 */
public class ExtentReportManager implements ITestListener {
    private static ExtentReports extent;
    private static ExtentTest test;

    /**
     * Initializes the ExtentReports instance with dynamic report naming.
     *
     * @param context TestNG test context.
     */
    @Override
    public void onStart(ITestContext context) {
        // Generate timestamp for unique report name
        String timeStamp = new SimpleDateFormat("yyyy.MM.dd.HH.mm.ss").format(new Date());
        String reportName = "TestAutomationReport_" + timeStamp + ".html";

        // Define report file location
        String reportPath = System.getProperty("user.dir") + "/reports/" + reportName;

        // Initialize ExtentSparkReporter
        ExtentSparkReporter spark = new ExtentSparkReporter(reportPath);
        spark.config().setDocumentTitle("Automation Test Report");
        spark.config().setReportName("Functional Testing");
        spark.config().setTheme(Theme.STANDARD);

        // Initialize ExtentReports
        extent = new ExtentReports();
        extent.attachReporter(spark);

        // Add system info
        extent.setSystemInfo("Tester Name", System.getProperty("user.name"));
        extent.setSystemInfo("Environment", "QA");
        extent.setSystemInfo("Operating System", System.getProperty("os.name"));
    }

    /**
     * Creates a new test entry in the report upon test start.
     *
     * @param result The test result.
     */
    @Override
    public void onTestStart(ITestResult result) {
        // Get test class and method name
        String className = result.getTestClass().getName();
        String methodName = result.getMethod().getMethodName();

        // Create new test in the report
        test = extent.createTest(className + " :: " + methodName);

        // Get groups and add as categories
        List groups = List.of(result.getMethod().getGroups());
        for (String group : groups) {
            test.assignCategory(group);
        }
    }

    /**
     * Logs test success in the report.
     *
     * @param result The test result.
     */
    @Override
    public void onTestSuccess(ITestResult result) {
        test.pass("Test Passed");
    }

    /**
     * Logs test failure in the report and attaches screenshots.
     *
     * @param result The test result.
     */
    @Override
    public void onTestFailure(ITestResult result) {
        try {
            // Capture screenshot
            String screenshotPath = new BaseTest().captureScreen(result.getMethod().getMethodName());

            // Attach screenshot to the report
            test.fail(result.getThrowable());
            test.addScreenCaptureFromPath(screenshotPath);
        } catch (IOException e) {
            e.printStackTrace();
            test.fail("Failed to attach screenshot");
        }
    }

    /**
     * Logs test skip in the report.
     *
     * @param result The test result.
     */
    @Override
    public void onTestSkipped(ITestResult result) {
        test.skip(result.getThrowable());
    }

    /**
     * Flushes the ExtentReports instance at the end of the suite.
     *
     * @param context The test context.
     */
    @Override
    public void onFinish(ITestContext context) {
        extent.flush();
    }

    // Other overridden methods can remain empty if not used
    @Override
    public void onTestFailedButWithinSuccessPercentage(ITestResult result) {}

    @Override
    public void onTestFailedWithTimeout(ITestResult result) {}
}

    Explanation:

    • onStart:
      • Initializes ExtentReports with a dynamic report name based on the current timestamp.
      • Configures report properties like title, name, and theme.
      • Adds system information dynamically (e.g., tester name, environment, OS).
    • onTestStart:
      • Creates a new test entry in the report using the class and method name.
      • Assigns categories based on the groups the test belongs to.
    • onTestSuccess:
      • Logs the test as passed.
    • onTestFailure:
      • Captures a screenshot upon test failure.
      • Attaches the screenshot and logs the exception details.
    • onTestSkipped:
      • Logs the test as skipped along with the reason.
    • onFinish:
      • Flushes the report to ensure all logs are written.
  4. Integrating ExtentReports with TestNG XML:To enable ExtentReports as a listener, you need to specify it in your TestNG XML files.

    Example:

    <listeners>
    <listener class-name="com.opencart.utilities.ExtentReportManager"/>
</listeners>

    Explanation: The <listeners> section integrates the ExtentReportManager as a TestNG listener, ensuring that reports are generated based on test execution events.

5. Capturing Screenshots on Test Failures

Capturing screenshots upon test failures is crucial for debugging and provides visual evidence of failures.

5.1. Implementing Screenshot Capture in BaseTest

  1. Open BaseTest.java:
    • Navigate to src/test/java/com/opencart/base/BaseTest.java.
  2. Add Screenshot Capture Method: 
    package com.opencart.base;

import org.apache.commons.io.FileUtils;
import org.openqa.selenium.OutputType;
import org.openqa.selenium.TakesScreenshot;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeClass;

import java.io.File;
import java.io.IOException;
import java.text.SimpleDateFormat;
import java.util.Date;

public class BaseTest {
    protected static WebDriver driver;

    @BeforeClass(groups = {"sanity", "regression", "master", "dataDriven"})
    public void setUp() {
        // Initialize WebDriver (e.g., ChromeDriver)
        System.setProperty("webdriver.chrome.driver", "path_to_chromedriver");
        driver = new ChromeDriver();
        driver.manage().window().maximize();
        driver.get("http://localhost/opencart");
    }

    @AfterClass(groups = {"sanity", "regression", "master", "dataDriven"})
    public void tearDown() {
        if (driver != null) {
            driver.quit();
        }
    }

    /**
     * Captures a screenshot of the current browser window.
     *
     * @param testMethodName The name of the test method.
     * @return The path to the captured screenshot.
     */
    public String captureScreen(String testMethodName) {
        // Generate timestamp
        String timeStamp = new SimpleDateFormat("yyyy.MM.dd.HH.mm.ss").format(new Date());

        // Define screenshot file name and path
        String screenshotName = testMethodName + "_" + timeStamp + ".png";
        String screenshotPath = System.getProperty("user.dir") + "/screenshots/" + screenshotName;

        // Capture screenshot
        File src = ((TakesScreenshot) driver).getScreenshotAs(OutputType.FILE);
        File dest = new File(screenshotPath);

        try {
            FileUtils.copyFile(src, dest);
        } catch (IOException e) {
            e.printStackTrace();
        }

        return screenshotPath;
    }
}

    Explanation:

    • captureScreen Method:
      • Generates a timestamp to ensure unique screenshot filenames.
      • Captures the screenshot using Selenium’s TakesScreenshot interface.
      • Saves the screenshot in the screenshots folder with the test method name and timestamp.
      • Returns the path to the captured screenshot for report attachment.
  3. Creating Screenshots Directory:
    • Ensure that the screenshots directory exists in your project root.
    • Create Screenshots Folder:
      • Navigate to your project root.
      • Right-click > New > Folder.
      • Name it screenshots.

6. Integrating ExtentReports Listener with TestNG

To fully integrate ExtentReports, ensure that your TestNG XML files are configured to use the listener, and your test classes are properly set up.

6.1. Updating TestNG XML with Listener

As shown in Section 3.3, ensure that your TestNG XML includes the ExtentReportManager listener.

<listeners>
    <listener class-name="com.opencart.utilities.ExtentReportManager"/>
</listeners>

6.2. Ensuring WebDriver is Static

To prevent multiple instances of WebDriver, make the driver variable static in the BaseTest class.

protected static WebDriver driver;

Explanation: Making driver static ensures that the same WebDriver instance is shared across different classes and listeners, preventing conflicts.

7. Executing Only Failed Test Cases

TestNG provides a mechanism to rerun only the failed test cases from the previous run, which is highly beneficial for efficient test executions and debugging.

7.1. Understanding TestNG Failed XML

After a test run, TestNG generates a testng-failed.xml file containing only the failed tests. You can rerun this XML to execute only the failed tests.

7.2. Executing Failed Tests

  1. Locate testng-failed.xml:
    • After a test run, navigate to the test-output folder.
    • Find the testng-failed.xml file.
  2. Run testng-failed.xml:
    • Right-click on testng-failed.xml > Run As > TestNG Suite.
    • This will execute only the tests that failed in the previous run.

7.3. Automating Failed Test Execution

For convenience, you can create a separate TestNG XML file dedicated to running failed tests.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="FailedTestsSuite">

    <listeners>
        <listener class-name="com.opencart.utilities.ExtentReportManager"/>
    </listeners>

    <test name="FailedTests">
        <classes>
            <class name="com.opencart.tests.FailedTestClass"/>
        </classes>
    </test>

</suite>

Note:

  • TestNG’s testng-failed.xml automatically includes only failed tests. Manually creating a separate XML for failed tests is optional and depends on your project’s requirements.

8. Best Practices

Implementing TestNG Grouping and ExtentReports effectively requires adhering to best practices to ensure maintainability and scalability.

  1. Consistent Group Naming:
    • Use clear and descriptive group names like "sanity", "regression", "master", "dataDriven".
    • Avoid ambiguous or overlapping group names.
  2. Maintain Single XML Configuration for Grouping:
    • Utilize a single grouping.xml file to manage all group executions.
    • Avoid creating multiple XML files for different groups to reduce complexity.
  3. Dynamic Report Naming:
    • Incorporate timestamps in report names to maintain history and avoid overwriting previous reports.
  4. Centralized Configuration:
    • Manage common configurations (like report paths, WebDriver setups) in utility classes or configuration files.
  5. Error Handling:
    • Implement robust error handling in listeners to gracefully manage exceptions during report generation or screenshot capture.
  6. Avoid Hardcoding Sensitive Information:
    • Refrain from hardcoding credentials or sensitive data. Use environment variables or secure storage mechanisms.
  7. Regularly Review and Update Dependencies:
    • Keep your project dependencies up-to-date to leverage the latest features and security patches.
  8. Integrate with Continuous Integration (CI) Tools:
    • Incorporate your TestNG suites into CI pipelines (like Jenkins) for automated test executions on code changes.
  9. Document Your Framework:
    • Maintain clear documentation of your test framework setup, configurations, and conventions to aid team collaboration.
  10. Optimize Test Execution:
    • Balance the number of groups and test methods to optimize parallel executions without overloading system resources.

9. Conclusion

By implementing TestNG Grouping and integrating ExtentReports, you’ve significantly enhanced your automation framework’s efficiency, organization, and reporting capabilities. Grouping allows for selective test executions, making your testing process more flexible and targeted. ExtentReports provides comprehensive and visually appealing reports, aiding in quick analysis and debugging.

Key Takeaways:

  • TestNG Grouping: Categorize tests into meaningful groups for selective execution.
  • ExtentReports Integration: Generate dynamic and detailed reports with screenshots on failures.
  • Handling Failed Tests: Efficiently rerun only failed tests to save time and resources.
  • Best Practices: Adhere to best practices for maintainable and scalable test frameworks.

Next Steps:

  1. Implement Additional Features:
    • Explore integrating Selenium Grid for distributed test executions across multiple environments.
    • Incorporate Data-Driven Testing using Apache POI for running tests with multiple data sets.
  2. Enhance Reporting:
    • Customize ExtentReports further by adding charts, graphs, or integrating with other reporting tools.
    • Automate report archiving and retention policies.
  3. Continuous Integration (CI):
    • Set up CI pipelines to automate test executions on code commits or schedule regular test runs.
  4. Advanced TestNG Features:
    • Utilize TestNG’s advanced features like dependencies, priorities, and annotations for more complex test scenarios.
  5. Security Enhancements:
    • Secure sensitive data within your test framework and reports.
    • Implement access controls for report visibility within your team.

Happy Testing! 🚀


Last Post


Next Post