Calculating Overlapping Code Coverage of Java-Based Services Using Automation Suites

on

 

Calculating Overlapping Code Coverage of Java-Based Services Using Automation Suites

Introduction

Code coverage is a critical metric in software testing, providing insights into how much of your code is exercised by automated tests. High code coverage ensures that the code is well-tested, reducing bugs and increasing software reliability. This blog provides a detailed step-by-step guide to calculating overlapping code coverage for Java-based services using automation suites and JaCoCo. We will also discuss how testers can help developers improve code quality and maintainability through detailed analysis and example code.

Prerequisites

  • Java Development Kit (JDK) installed
  • Maven installed
  • JaCoCo JARs downloaded from the JaCoCo website
  • An automation test suite ready for execution

Step-by-Step Guide

Step 1: Checkout Developer's Repository on Your Local Machine

First, clone the developer's repository to your local machine:


git clone <repository_url>
cd <repository_directory>

Step 2: Compile Code and Create JAR of Developer's Repository

Navigate to the repository directory and compile the code to generate a JAR file:


mvn clean install

This command will create a JAR file in the target directory, typically named devRepo.jar.

Step 3: Download JaCoCo Agent Runtime JAR

Download the JaCoCo agent runtime JAR from the JaCoCo website and place it in a known location, e.g., libs/org.jacoco.agent.jar.

Step 4: Start the Java Server with JaCoCo Agent

Start the Java server with the JaCoCo agent by running the following command:


java -javaagent:libs/org.jacoco.agent.jar=destfile=jacoco.exec -Dsun.misc.URLClassPath.disableJarChecking=true -Dspring.profiles.active=stage -jar target/devRepo.jar

This command starts the server with the JaCoCo agent enabled, which will collect coverage data and store it in jacoco.exec.

Step 5: Wait for the Spring Service to Start

Allow the Spring service to start on a port, which typically takes about 30 seconds.

Step 6: Run Your Automation Suite

Now, run your automation suite. Since the development repository is running on your local machine, the Spring service will be available on localhost:8080 (or the port specified in the configuration). Point the base URL of your automation suite to localhost:8080 and execute the full suite.


// Example of configuring the base URL in a Java automation framework
RestAssured.baseURI = "http://localhost:8080";

Run your test suite using your preferred method, for example:


mvn test

Step 7: Observe the Growth of jacoco.exec

As your test suite runs, the size of jacoco.exec will increase, indicating that coverage data is being collected.

Step 8: Generate the Coverage Report

Generate the coverage report using the JaCoCo CLI:


java -jar jacococli.jar report jacoco.exec --classfiles target/classes --html report/ --sourcefiles src/main/java

This command will produce an HTML report in the report directory. Open report/index.html in your browser to view the coverage report.

Understanding Code Coverage Metrics

JaCoCo provides several important code coverage metrics, which are visually represented in the generated report:

Line Coverage

Line coverage measures the percentage of code lines that have been executed by the tests. Each line of code is either fully covered (green), partially covered (yellow), or not covered at all (red).

Example:


public int add(int a, int b) {
    return a + b; // This line should be covered by a test case.
}

Test case:


@Test
public void testAdd() {
    int result = calculator.add(2, 3);
    assertEquals(5, result);
}

Branch Coverage

Branch coverage measures the percentage of control flow branches (e.g., if, else, switch statements) that have been executed by the tests. It helps ensure that all possible paths through the code are tested.

Example:


public String getGrade(int score) {
    if (score >= 90) {
        return "A";
    } else if (score >= 80) {
        return "B";
    } else {
        return "C";
    }
}

Test cases:


@Test
public void testGetGradeA() {
    String grade = student.getGrade(95);
    assertEquals("A", grade);
}

@Test
public void testGetGradeB() {
    String grade = student.getGrade(85);
    assertEquals("B", grade);
}

@Test
public void testGetGradeC() {
    String grade = student.getGrade(75);
    assertEquals("C", grade);
}

Cyclomatic Complexity

Cyclomatic complexity measures the complexity of the code by counting the number of independent paths through the code. It helps identify complex code that might require more thorough testing.

Example:


public String getFeedback(int score) {
    if (score >= 90) {
        return "Excellent";
    } else if (score >= 80) {
        return "Good";
    } else if (score >= 70) {
        return "Average";
    } else {
        return "Poor";
    }
}

Test cases to cover all paths:


@Test
public void testGetFeedbackExcellent() {
    String feedback = student.getFeedback(95);
    assertEquals("Excellent", feedback);
}

@Test
public void testGetFeedbackGood() {
    String feedback = student.getFeedback(85);
    assertEquals("Good", feedback);
}

@Test
public void testGetFeedbackAverage() {
    String feedback = student.getFeedback(75);
    assertEquals("Average", feedback);
}

@Test
public void testGetFeedbackPoor() {
    String feedback = student.getFeedback(65);
    assertEquals("Poor", feedback);
}

Benefits of Code Coverage Reports

  1. Identifying Missing Test Cases:

    • The report helps identify which use cases are not covered by your automation tests. By analyzing the report, you can add test cases to cover untested branches or conditions.

  2. Cleaning Up Unused Code:

    • The report can highlight unused code, allowing you to collaborate with developers to clean up unnecessary code, improving code quality and maintainability.

How to Read the Report

The JaCoCo report provides visual indicators to help analyze code coverage:

  • Red Diamond: No branches have been exercised.
  • Yellow Diamond: Some branches have not been exercised.
  • Green Diamond: All branches have been exercised.

The report offers three crucial metrics:

  1. Line Coverage: The percentage of Java bytecode instructions executed by the tests.
  2. Branch Coverage: The percentage of branches (if/else, switch statements) executed.
  3. Cyclomatic Complexity: The complexity of the code based on the number of paths required to cover all possible execution paths.

Example Analysis

Suppose our report shows 94% instruction coverage and 100% branch coverage. This high coverage score is excellent, but our goal is to achieve 100% coverage. The report indicates 38 instructions not covered, referring to bytecode instructions rather than Java code lines. By examining the report, we can add tests to cover the remaining instructions.

Conclusion

Calculating and analyzing code coverage using JaCoCo and automation suites is a powerful practice for software testers. It not only boosts confidence in the code but also aids in identifying missing test cases and cleaning up unused code, contributing to overall code quality and maintainability. By following this detailed guide, you can effectively measure and improve the coverage of your Java-based services.

Happy testing!

Comments

Post a Comment