🅴🆆🅰🆁🆃

When running pull request validation, raw artifacts are not enough. Developers need immediate feedback.

TRX files and Cobertura XML reports are useful, but they require downloading artifacts. That slows down review cycles. What we wanted instead was:

• Tests run on every PR
• Coverage generated automatically
• A clear quality summary visible directly in the GitHub job summary
• A hard coverage threshold that fails the build when violated

This article explains exactly how we achieved that in a fully deterministic .NET 10 pipeline.

---

The Goal

We wanted the pull request pipeline to:

1. Restore and build the solution
2. Run tests
3. Generate Cobertura coverage
4. Produce a Markdown coverage summary
5. Parse TRX + coverage for a consolidated quality report
6. Append everything to the GitHub job summary
7. Enforce a coverage threshold

The key mechanism that makes this possible is:

$GITHUB_STEP_SUMMARY

See the official GitHub documentation on adding a job summary for details.

Any content appended to this file appears in the job summary UI.

No marketplace actions. No PR comments. Just native GitHub functionality.

---

The Full Workflow

Below is the complete workflow used in our repository.

name: Quality Validation

on:
  pull_request:
    paths-ignore:
      - '**/*.md'
  push:
    branches: [ main ]
    paths-ignore:
      - '**/*.md'

permissions:
  contents: read

concurrency:
  group: quality-${{ github.ref }}
  cancel-in-progress: true

env:
  DOTNET_NOLOGO: true
  DOTNET_CLI_TELEMETRY_OPTOUT: true
  NUGET_XMLDOC_MODE: skip

jobs:
  format-test-coverage:
    runs-on: ubuntu-latest
    timeout-minutes: 30

This job:

• Validates formatting
• Builds in Release mode
• Executes tests
• Generates coverage
• Produces Markdown summaries
• Enforces a coverage threshold
• Uploads artifacts

---

Step 1 – Restore, Format, Build

- name: Restore
  run: dotnet restore SolutionName.sln

- name: Validate formatting
  run: >
    dotnet format SolutionName.sln
    --verify-no-changes
    --no-restore
    --exclude ./sample/Simple ./sample/MinimalApi

- name: Build
  run: dotnet build SolutionName.sln -c Release --no-restore

Formatting failures fail fast before tests are executed.

---

Step 2 – Run Tests (With Coverage Enabled)

- name: Test (with coverage)
  run: >
    dotnet test SolutionName.sln
    -c Release
    --no-build
    --logger "trx;LogFilePrefix=test-results"

Coverage files are produced by Coverlet integration inside the test projects.

The important output files are:

**/TestResults/**/*.trx
**/TestResults/**/coverage.cobertura.xml

Cobertura format integrates cleanly with ReportGenerator and is easy to parse manually.

---

Step 3 – Install ReportGenerator

- name: Install ReportGenerator
  if: always()
  run: dotnet tool install -g dotnet-reportgenerator-globaltool

We explicitly use if: always() so coverage generation still runs even if tests fail.

---

Step 4 – Generate Coverage Reports

- name: Generate coverage report
  if: always()
  shell: bash
  run: |
    reportgenerator \
      -reports:"**/TestResults/**/coverage.cobertura.xml" \
      -targetdir:"CoverageReport" \
      -reporttypes:"Html;MarkdownSummary"

This produces:

CoverageReport/
  ├── index.html
  ├── Summary.md
  └── ...

Summary.md is directly consumable by GitHub’s job summary UI.

---

Step 5 – Append ReportGenerator Summary

- name: Append ReportGenerator summary to job summary
  if: always()
  shell: bash
  run: |
    if [ -f CoverageReport/Summary.md ]; then
      echo "" >> "$GITHUB_STEP_SUMMARY"
      echo "## Detailed Coverage Report (ReportGenerator)" >> "$GITHUB_STEP_SUMMARY"
      cat CoverageReport/Summary.md >> "$GITHUB_STEP_SUMMARY"
    fi

Note the use of >>. Using > would overwrite previous content.

---

Step 6 – Enforce Coverage Threshold + Consolidated Quality Report

This is where the pipeline becomes more interesting.

We parse:

• All TRX files (for test statistics and failure details)
• All Cobertura XML files (for line + branch coverage)
• Per-module coverage
• Failed test details (with stack traces)
• Apply a coverage threshold (80%)

The entire report is generated via an inline Python script:

- name: Enforce coverage threshold + write summary
  if: always()
  shell: bash
  run: |
    set +e

    python3 <<'PY' > quality-summary.md
    import glob
    import subprocess
    import urllib.parse
    import xml.etree.ElementTree as ET

    def pct(covered, total):
    return (covered / total * 100.0) if total else 0.0

    def badge(label, message, ok):
    color = "brightgreen" if ok else "red"
    le = urllib.parse.quote(label)
    me = urllib.parse.quote(message)
    return f'![{label}: {message}](https://img.shields.io/badge/{le}-{me}-{color})'

    coverage_threshold = 80.0
    max_failed_tests_to_print = 25

    total = passed = failed = skipped = 0
    failed_tests = []  # (testName, trxFile, message, stack)

    trx_files = glob.glob('**/TestResults/**/*.trx', recursive=True)
    for trx_file in trx_files:
    root = ET.parse(trx_file).getroot()

    counters = root.find('.//{*}ResultSummary/{*}Counters')
    if counters is not None:
        total += int(counters.attrib.get('total', 0))
        passed += int(counters.attrib.get('passed', 0))
        failed += int(counters.attrib.get('failed', 0))
        skipped += int(counters.attrib.get('notExecuted', 0))

    # Collect failed test details
    for r in root.findall('.//{*}Results/{*}UnitTestResult'):
        if (r.attrib.get('outcome') or '').lower() != 'failed':
        continue

        test_name = r.attrib.get('testName', 'Unknown test')
        # ErrorInfo is typically under Output/ErrorInfo
        msg_el = r.find('.//{*}Output/{*}ErrorInfo/{*}Message')
        stk_el = r.find('.//{*}Output/{*}ErrorInfo/{*}StackTrace')
        message = (msg_el.text or '').strip() if msg_el is not None else ''
        stack = (stk_el.text or '').strip() if stk_el is not None else ''
        failed_tests.append((test_name, trx_file, message, stack))

    # Coverage parsing (Cobertura)
    line_valid = line_covered = branch_valid = branch_covered = 0
    modules = {}
    cov_files = glob.glob('**/TestResults/**/coverage.cobertura.xml', recursive=True)
    if not cov_files:
    print('## Quality report\n')
    print('❌ No coverage.cobertura.xml files found under **/TestResults.')
    raise SystemExit(1)

    for cov_file in cov_files:
    root = ET.parse(cov_file).getroot()
    line_valid += int(float(root.attrib.get('lines-valid', 0)))
    line_covered += int(float(root.attrib.get('lines-covered', 0)))
    branch_valid += int(float(root.attrib.get('branches-valid', 0)))
    branch_covered += int(float(root.attrib.get('branches-covered', 0)))

    for package in root.findall('.//packages/package'):
        name = package.attrib.get('name', 'Unknown')
        module_line_rate = float(package.attrib.get('line-rate', 0.0))
        module_branch_rate = float(package.attrib.get('branch-rate', 0.0))
        modules.setdefault(name, {'line_rate_sum': 0.0, 'branch_rate_sum': 0.0, 'count': 0})
        modules[name]['line_rate_sum'] += module_line_rate
        modules[name]['branch_rate_sum'] += module_branch_rate
        modules[name]['count'] += 1

    line_total_pct = pct(line_covered, line_valid)
    branch_total_pct = pct(branch_covered, branch_valid)

    tests_ok = failed == 0
    coverage_ok = line_total_pct >= coverage_threshold

    tests_badge = badge('tests', f'{passed}/{total} passed', tests_ok)
    coverage_badge = badge('coverage', f'{line_total_pct:.2f}%', coverage_ok)

    generated_at = subprocess.check_output(['date', '-u'], text=True).strip()

    print('## Quality report\n')
    print(f'{tests_badge} {coverage_badge}\n')
    print('- ✅ Format: OK')
    print(f'- {"✅" if tests_ok else "❌"} Tests: {passed}/{total} passed, {failed} failed, {skipped} skipped')
    print(f'- {"✅" if coverage_ok else "❌"} Coverage: {line_total_pct:.2f}% line (threshold: {coverage_threshold:.0f}%)\n')

    # Tests detail
    print('### Tests')
    print('| Metric | Count |')
    print('|---|---:|')
    print(f'| Total | {total} |')
    print(f'| Passed | {passed} |')
    print(f'| Failed | {failed} |')
    print(f'| Skipped | {skipped} |')

    if failed_tests:
    print('\n### Failed tests')
    to_show = failed_tests[:max_failed_tests_to_print]
    for (name, trx_file, message, stack) in to_show:
        print(f'\n#### {name}')
        print(f'- TRX: `{trx_file}`')
        if message:
        print(f'- Message: `{message.replace("`","\\`")}`')
        if stack:
        print('<details><summary>Stack trace</summary>\n')
        print('```')
        print(stack)
        print('```')
        print('\n</details>')
    if len(failed_tests) > max_failed_tests_to_print:
        print(f'\n> Showing first {max_failed_tests_to_print} failures (out of {len(failed_tests)}).')

    # Coverage detail
    print('\n### Coverage')
    print('| Module | Line | Branch |')
    print('|---|---:|---:|')
    print(f'| Total | {line_total_pct:.2f}% | {branch_total_pct:.2f}% |')

    for module_name in sorted(modules):
    module = modules[module_name]
    avg_line = (module['line_rate_sum'] / module['count']) * 100.0 if module['count'] else 0.0
    avg_branch = (module['branch_rate_sum'] / module['count']) * 100.0 if module['count'] else 0.0
    print(f'| {module_name} | {avg_line:.2f}% | {avg_branch:.2f}% |')

    print(f'\n> Generated at {generated_at}')

    if not tests_ok:
    raise SystemExit(1)
    if not coverage_ok:
    raise SystemExit(2)
    PY

    py_exit=$?

    # Always publish summary, even when failing
    cat quality-summary.md >> "$GITHUB_STEP_SUMMARY"

    # Fail the job after summary is published
    exit $py_exit

What This Script Does

1. Aggregates test counts across all TRX files

2. Extracts detailed failure messages + stack traces

3. Aggregates coverage across multiple test projects

4. Calculates:

   • Total line coverage
   • Total branch coverage

5. Builds shields.io badges

6. Fails if:

   • Any tests failed
   • Line coverage < 80%

Because the summary is appended before exiting with failure, developers always see diagnostics.

---

What the Pull Request UI Shows

When the job completes, the summary includes:

• Test badge (passed/total)
• Coverage badge
• Coverage threshold status
• Per-module breakdown
• Failed test details
• Collapsible stack traces
• Timestamp

Reviewers immediately see whether:

• Tests are green
• Coverage regressed
• Which tests failed
• In which TRX file
• With what stack trace

No artifact download required.

---

Artifact Upload Still Matters

We still upload everything for deeper inspection:

- name: Upload test results (TRX)
  if: always()
  uses: actions/upload-artifact@v5
  with:
    name: test-results
    path: |
      **/TestResults/**/*.trx
    if-no-files-found: error
    retention-days: 2

- name: Upload raw coverage (cobertura)
  if: always()
  uses: actions/upload-artifact@v5
  with:
    name: code-coverage
    path: |
      **/TestResults/**/coverage.cobertura.xml
    if-no-files-found: error
    retention-days: 2

- name: Upload coverage report
  if: always()
  uses: actions/upload-artifact@v5
  with:
    name: coverage-report
    path: CoverageReport
    if-no-files-found: error
    retention-days: 2

The summary provides fast signal. Artifacts provide full traceability.

---

Why This Approach

• We want deterministic pipelines
• We avoid external dependencies as much as possible
• We control formatting completely
• We already use ReportGenerator
• We want unified test + coverage reporting

Everything runs with:

• dotnet CLI
• ReportGenerator
• A small inline Python script
• Native GitHub functionality

No hidden abstractions.

---

Why This Is Superior to Only Uploading Artifacts

Artifacts are passive. Job summaries are active.

This approach:

• Shortens PR review cycles
• Makes regressions visible immediately
• Enforces quality gates automatically
• Keeps CI self-contained
• Avoids comment spam in PR threads

The signal is embedded directly in the job output.

---

Conclusion

Adding coverage and test diagnostics to the GitHub job summary requires:

1. Generate Cobertura coverage during dotnet test
2. Convert it with ReportGenerator
3. Parse TRX + Cobertura for consolidated metrics
4. Append Markdown to $GITHUB_STEP_SUMMARY
5. Enforce thresholds explicitly

No complex integrations. No third-party actions. Just explicit, deterministic CI.

This small change materially improves developer feedback and PR quality with minimal pipeline complexity.

Happy Coding.