Skip to content

Demo - Consolidate pages [DOCS-12576] #32662

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
+348 −373
Closed

Demo - Consolidate pages [DOCS-12576] #32662

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 0 additions & 5 deletions config/_default/menus/main.en.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5380,11 +5380,6 @@ menu:
parent: code_coverage
identifier: code_coverage_data_collected
weight: 2
- name: Upload Reports
url: code_coverage/upload/
parent: code_coverage
identifier: code_coverage_upload
weight: 3
- name: PR Gates
url: pr_gates/
pre: ci
Expand Down
348 changes: 348 additions & 0 deletions content/en/code_coverage/setup.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,346 @@

Navigate to [PR Gates rule creation][5] and configure a rule to gate on total or patch coverage.

## Upload code coverage reports

Check warning on line 95 in content/en/code_coverage/setup.md

View workflow job for this annotation

GitHub Actions / vale

Datadog.headings 

'Upload code coverage reports' should use sentence-style capitalization.

Update your CI pipeline to upload code coverage report files to Datadog. This involves installing and running the `datadog-ci` CLI in your CI environment.

See [Data Collected][14] for details on what data is collected during code coverage report upload.

### Supported coverage report formats

Datadog supports the following coverage data formats—expand for examples:

{{% collapse-content title="LCOV" level="h4" expanded=false id="lcov" %}}
{{< code-block lang="text" >}}
TN:
SF:src/example.c
FN:3,add
FNDA:5,add
FNF:1
FNH:1
DA:3,5
DA:4,5
DA:5,5
DA:8,0
DA:9,0
LF:5
LH:3
BRDA:4,0,0,5
BRDA:4,0,1,0
BRF:2
BRH:1
end_of_record
{{< /code-block >}}
{{% /collapse-content %}}

{{% collapse-content title="Cobertura XML" level="h4" expanded=false id="cobertura-xml" %}}
{{< code-block lang="xml" >}}
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE coverage SYSTEM "http://cobertura.sourceforge.net/xml/coverage-04.dtd">
<coverage lines-valid="5" lines-covered="3" line-rate="0.6" branches-valid="2" branches-covered="1" branch-rate="0.5" timestamp="1690658886" version="1.9">
<sources>
<source>src</source>
</sources>
<packages>
<package name="example" line-rate="0.6" branch-rate="0.5">
<classes>
<class name="Example" filename="example/Example.java" line-rate="0.6" branch-rate="0.5">
<methods>
<method name="add" signature="(II)I" line-rate="1.0" branch-rate="1.0">
<lines>
<line number="3" hits="5"/>
<line number="4" hits="5" branch="true" condition-coverage="50% (1/2)"/>
<line number="5" hits="5"/>
</lines>
</method>
</methods>
<lines>
<line number="3" hits="5"/>
<line number="4" hits="5" branch="true" condition-coverage="50% (1/2)"/>
<line number="5" hits="5"/>
<line number="8" hits="0"/>
<line number="9" hits="0"/>
</lines>
</class>
</classes>
</package>
</packages>
</coverage>
{{< /code-block >}}
{{% /collapse-content %}}

{{% collapse-content title="Jacoco XML" level="h4" expanded=false id="jacoco-xml" %}}
{{< code-block lang="xml" >}}
<?xml version="1.0" encoding="UTF-8"?>
<report name="Example">
<sessioninfo id="SessionId" start="1690658886000" dump="1690658887000"/>
<package name="example">
<sourcefile name="Example.java">
<line nr="3" mi="0" ci="5"/>
<line nr="4" mi="0" ci="5" mb="1" cb="1"/>
<line nr="5" mi="0" ci="5"/>
<line nr="8" mi="1" ci="0"/>
<line nr="9" mi="1" ci="0"/>
</sourcefile>
</package>
</report>
{{< /code-block >}}
{{% /collapse-content %}}

{{% collapse-content title="Clover XML" level="h4" expanded=false id="clover-xml" %}}
{{< code-block lang="xml" >}}
<coverage generated="1661852015">
<project timestamp="1661852015">
<file name="/var/www/html/src/App/Console/CronjobRunnerCommand.php">
<class name="App\Console\CronjobRunnerCommand" namespace="global">
<metrics complexity="3" methods="3" coveredmethods="0" conditionals="0" coveredconditionals="0" statements="4" coveredstatements="0" elements="7" coveredelements="0"/>
</class>
<line num="18" type="method" name="__construct" visibility="public" complexity="1" crap="2" count="0"/>
<line num="20" type="stmt" count="1"/>
<line num="27" type="stmt" count="0"/>
<line num="30" type="method" name="execute" visibility="protected" complexity="1" crap="2" count="0"/>
<line num="32" type="stmt" count="0"/>
<metrics loc="35" ncloc="35" classes="1" methods="3" coveredmethods="0" conditionals="0" coveredconditionals="0" statements="4" coveredstatements="0" elements="7" coveredelements="0"/>
</file>
<file name="/var/www/html/src/App/Console/CronjobRunnerCommand2.php">
<line num="42" type="stmt" count="1"/>
</file>
</project>
</coverage>
{{< /code-block >}}
{{% /collapse-content %}}

{{% collapse-content title="OpenCover XML" level="h4" expanded=false id="opencover-xml" %}}
{{< code-block lang="xml" >}}
<?xml version="1.0" encoding="utf-8"?>
<CoverageSession xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<Modules>
<Module hash="ABC123">
<ModulePath>Example.dll</ModulePath>
<Files>
<File uid="1" fullPath="src\example\Example.cs" />
</Files>
<Classes>
<Class>
<Methods>
<Method visited="true" cyclomaticComplexity="1" sequenceCoverage="100">
<FileRef uid="1"/>
<SequencePoints>
<SequencePoint vc="5" sl="3" />
<SequencePoint vc="5" sl="4" />
<SequencePoint vc="5" sl="5" />
<SequencePoint vc="0" sl="9" />
</SequencePoints>
<BranchPoints>
<BranchPoint vc="5" sl="4" path="0"/>
<BranchPoint vc="0" sl="4" path="1"/>
</BranchPoints>
</Method>
</Methods>
</Class>
</Classes>
</Module>
</Modules>
</CoverageSession>
{{< /code-block >}}
{{% /collapse-content %}}

{{% collapse-content title="Simplecov JSON" level="h4" expanded=false id="simplecov-json" %}}
{{< code-block lang="json" >}}
{
"meta": {
"simplecov_version": "0.21.2"
},
"coverage": {
"/path/to/file1.rb": {
"lines": [
null,
1,
2,
0,
null,
1,
null,
null,
null,
"ignored",
"ignored",
"ignored",
null
],
"branches": []
},
"/path/to/file2.rb": {
"lines": [1, 1, null, 0, 1],
"branches": []
}
}
}
{{< /code-block >}}
{{% /collapse-content %}}

### Install the datadog-ci CLI

Check warning on line 274 in content/en/code_coverage/setup.md

View workflow job for this annotation

GitHub Actions / vale

Datadog.headings 

'Install the datadog-ci CLI' should use sentence-style capitalization.

Install the [`datadog-ci`][7] CLI globally using `npm`:

{{< code-block lang="shell" >}}
npm install -g @datadog/datadog-ci
{{< /code-block >}}

#### Standalone binary

If installing Node.js in the CI is an issue, standalone binaries are provided with [Datadog CI releases][8]. Only _linux-x64_, _linux-arm64_, _darwin-x64_, _darwin-arm64_ (MacOS) and _win-x64_ (Windows) are supported. To install, run the following from your terminal:

{{< tabs >}}
{{% tab "Linux" %}}
{{< code-block lang="shell" >}}
curl -L --fail "https://github.com/DataDog/datadog-ci/releases/latest/download/datadog-ci_linux-x64" --output "/usr/local/bin/datadog-ci" && chmod +x /usr/local/bin/datadog-ci
{{< /code-block >}}

Then run any command with `datadog-ci`:
{{< code-block lang="shell" >}}
datadog-ci version
{{< /code-block >}}
{{% /tab %}}

{{% tab "MacOS" %}}
{{< code-block lang="shell" >}}
curl -L --fail "https://github.com/DataDog/datadog-ci/releases/latest/download/datadog-ci_darwin-x64" --output "/usr/local/bin/datadog-ci" && chmod +x /usr/local/bin/datadog-ci
{{< /code-block >}}

Then run any command with `datadog-ci`:
{{< code-block lang="shell" >}}
datadog-ci version
{{< /code-block >}}
{{% /tab %}}

{{% tab "Windows" %}}
{{< code-block lang="powershell" >}}
Invoke-WebRequest -Uri "https://github.com/DataDog/datadog-ci/releases/latest/download/datadog-ci_win-x64" -OutFile "datadog-ci.exe"
{{< /code-block >}}

Then run any command with `Start-Process -FilePath "datadog-ci.exe"`:
{{< code-block lang="powershell" >}}
Start-Process -FilePath "./datadog-ci.exe" -ArgumentList version
{{< /code-block >}}
{{% /tab %}}
{{< /tabs >}}

#### Docker image

Alternatively, you can update your CI job to run in a container based on the [Datadog CI Docker image][13].
The image comes with `datadog-ci` preinstalled and ready to use.

### Uploading coverage reports

<div class="alert alert-info">
Datadog automatically aggregates all reports for the same commit on the backend. You don't need to merge coverage reports before uploading them.
</div>

To upload your code coverage reports to Datadog, run the following command. Provide a valid [Datadog API key][9] (`DD_API_KEY`), and one or more file paths to either the coverage report files directly or directories containing them:

{{< tabs >}}
{{% tab "GitHub Actions" %}}
<pre>
<code class="language-yaml" data-lang="yaml">
steps:
- name: Upload coverage reports to Datadog
run: datadog-ci coverage upload .
env:
DD_API_KEY: ${{ secrets.DD_API_KEY }}
DD_SITE: {{< region-param key="dd_site" >}}
</code>
</pre>
{{% /tab %}}
{{% tab "Gitlab" %}}
<pre>
<code class="language-yaml" data-lang="yaml">
test:
stage: test
script:
- ... # run your tests and generate coverage reports
- datadog-ci coverage upload . # make sure to add the DD_API_KEY CI/CD variable
</code>
</pre>
{{% /tab %}}
{{< /tabs >}}

The command recursively searches the specified directories for supported coverage report files, so specifying the current directory (`.`) is usually sufficient.
See the [`datadog-ci` documentation][10] for more details on the `datadog-ci coverage upload` command.

Shortly after the code coverage report upload is finished, Datadog adds a PR comment with code coverage percentage values.
You can also view your coverage data aggregated by pull request in the [Code Coverage page][11] in Datadog, with the ability to examine individual files and lines of code.

Check notice on line 364 in content/en/code_coverage/setup.md

View workflow job for this annotation

GitHub Actions / vale

Datadog.sentencelength 

Suggestion: Try to keep your sentence length to 25 words or fewer.

{{< img src="/code_coverage/pr_details.png" text="Code Coverage PR details page in Datadog" style="width:100%" >}}

## Troubleshooting

### Coverage upload command does not detect coverage report files

The `datadog-ci coverage upload` command automatically detects supported coverage report files in the specified directories using heuristics, such as file names and extensions.

Check warning on line 372 in content/en/code_coverage/setup.md

View workflow job for this annotation

GitHub Actions / vale

Datadog.words 

Use 'filenames' instead of 'file names'.
If your coverage report files do not match expected patterns, the command might not detect them automatically. In this case, specify the report format and provide the file paths as positional arguments. For example:

{{< code-block lang="shell" >}}
datadog-ci coverage upload --format=lcov \
src/coverage-reports/unit-tests/coverage.info \
src/coverage-reports/e2e-tests/coverage.info
{{< /code-block >}}

### Coverage upload fails with "Format could not be detected" error

The `datadog-ci coverage upload` command automatically detects the format of the coverage report files based on their content and file extension.
If the command fails with the following error:
```
Invalid coverage report file [...]: format could not be detected
```
specify the format explicitly using the `--format` option, like this:

{{< code-block lang="shell" >}}
datadog-ci coverage upload --format=cobertura reports/cobertura.xml
{{< /code-block >}}

### Coverage upload outputs "Could not sync git metadata" error

Check warning on line 394 in content/en/code_coverage/setup.md

View workflow job for this annotation

GitHub Actions / vale

Datadog.headings 

'Coverage upload outputs "Could not sync git metadata" error' should use sentence-style capitalization.

Git metadata upload is only required if you can't integrate your CI provider directly with Datadog.
If you are using a [source code provider integration][12], such as Datadog GitHub app or Gitlab integration, you can disable the git metadata upload by passing the `--skip-git-metadata-upload=1` flag to the `datadog-ci coverage upload` command, like this:

Check notice on line 397 in content/en/code_coverage/setup.md

View workflow job for this annotation

GitHub Actions / vale

Datadog.sentencelength 

Suggestion: Try to keep your sentence length to 25 words or fewer.

{{< code-block lang="shell" >}}
datadog-ci coverage upload --skip-git-metadata-upload=1 .
{{< /code-block >}}

### Datadog UI does not show changed files in the PR view

By default, the "Changed files" table only contains executable source code files that are present in the uploaded coverage reports.
Select **Non-executable files** or **All** in the table header to display all files that were changed in the PR, regardless of whether they are executable or not.

Check notice on line 406 in content/en/code_coverage/setup.md

View workflow job for this annotation

GitHub Actions / vale

Datadog.sentencelength 

Suggestion: Try to keep your sentence length to 25 words or fewer.

{{< img src="/code_coverage/non_executable_files.png" text="In Changed files, you have the option to select Non-executable on the table header" style="width:100%" >}}

If a source code file is mistakenly marked as non-executable, it is probably missing from your uploaded coverage reports.
Make sure that you are uploading all of your relevant reports, and double-check your coverage tool configuration to verify that coverage data is collected for all applicable files.

Check notice on line 411 in content/en/code_coverage/setup.md

View workflow job for this annotation

GitHub Actions / vale

Datadog.sentencelength 

Suggestion: Try to keep your sentence length to 25 words or fewer.

Test sources are not considered executable files as they are not part of the production codebase being measured for coverage.

### Datadog UI shows incorrect file paths

Code Coverage relies on the file paths in coverage reports to be either absolute or relative to the repository root.
If the paths in your report are relative to a different directory in your repository, specify the correct base path (relative to the repo root) with the `--base-path` option when running the `datadog-ci coverage upload` command, like this:

Check notice on line 418 in content/en/code_coverage/setup.md

View workflow job for this annotation

GitHub Actions / vale

Datadog.sentencelength 

Suggestion: Try to keep your sentence length to 25 words or fewer.

{{< code-block lang="shell" >}}
datadog-ci coverage upload --base-path=frontend/src .
{{< /code-block >}}

### Discrepancy between Datadog UI and coverage report values

Datadog automatically merges coverage reports for the same commit.
As a result, the coverage percentage displayed in the Datadog UI may differ from the values in your individual coverage reports, especially if those reports contain overlapping or duplicate source code file entries.

Check notice on line 427 in content/en/code_coverage/setup.md

View workflow job for this annotation

GitHub Actions / vale

Datadog.sentencelength 

Suggestion: Try to keep your sentence length to 25 words or fewer.

If you use an external tool (such as [ReportGenerator](https://reportgenerator.io/)) to merge coverage reports before uploading to Datadog,

Check notice on line 429 in content/en/code_coverage/setup.md

View workflow job for this annotation

GitHub Actions / vale

Datadog.sentencelength 

Suggestion: Try to keep your sentence length to 25 words or fewer.
ensure your merged reports do not contain duplicate source code file entries.
Datadog deduplicates overlapping files across reports, which can result in differences between your original coverage values and the merged values displayed in the Datadog UI.



## Further reading

{{< partial name="whats-next/whats-next.html" >}}
Expand All @@ -102,3 +442,11 @@
[4]: https://app.datadoghq.com/organization-settings/roles
[5]: https://app.datadoghq.com/ci/pr-gates/rule/create
[6]: /code_coverage/upload/
[7]: https://www.npmjs.com/package/@datadog/datadog-ci
[8]: https://github.com/DataDog/datadog-ci/releases
[9]: https://app.datadoghq.com/organization-settings/api-keys
[10]: https://github.com/DataDog/datadog-ci/blob/master/packages/datadog-ci/src/commands/coverage/README.md
[11]: https://app.datadoghq.com/ci/code-coverage
[12]: /code_coverage/setup/#integrate-with-source-code-provider
[13]: https://hub.docker.com/r/datadog/ci
[14]: /code_coverage/data_collected/#code-coverage-report-upload
Loading
Loading