Skip to content
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
202 changes: 138 additions & 64 deletions content/en/tracing/trace_collection/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,112 +28,179 @@
---

## Overview
Application {{< tooltip glossary="instrumentation" >}} with Datadog APM involves two steps:

To get started with Datadog APM, you need to follow these key steps:
1. **SDK setup**: Adding a Datadog SDK to your application.
2. **Span creation**: Capturing observability data as {{< tooltip glossary="span" >}}s.

1. Install and configure the Datadog Agent.
2. Instrument your application.
Spans are automatically generated by default as soon as the SDK is loaded. For most users, this automatic span creation provides sufficient visibility. If you need more control, you can optionally add custom spans.

<div class="alert alert-info"><strong>Simplify your setup!</strong> Install the Agent and instrument your application in one step with <a href="https://docs.datadoghq.com/tracing/trace_collection/single-step-apm/">Single Step Instrumentation</a>.</div>

Instrumenting your application allows observability data to be sent to the Agent, which then passes data to the Datadog backend to display in the UI.
**Note**: These steps assume you have a [Datadog Agent][5] installed and configured to receive traces.

{{< img src="tracing/visualization/troubleshooting_pipeline.png" alt="The APM pipeline">}}

## Choose your path

<div class="alert alert-info">
<strong>Prefer vendor-neutral instrumentation?</strong> See the <a href="/opentelemetry/">OpenTelemetry documentation</a> for using OpenTelemetry with Datadog.
</div>

### No code changes

## Instrumentation types
**Best for**: Teams that want tracing without modifying application code.

There are two main approaches to {{< tooltip glossary="instrument" >}} your application:
Use [Single Step Instrumentation][1] to automatically set up Datadog SDKs. If you need custom spans, you can also use [Dynamic Instrumentation][4] to add them from the Datadog UI without changing code.

**Automatic instrumentation** creates {{< tooltip glossary="span" >}}s for your application with minimal manual steps, capturing essential observability data across common libraries and languages with minimal configuration.
{{< whatsnext desc=" " >}}
{{< nextlink href="/tracing/trace_collection/single-step-apm/" >}}Get started with Single Step Instrumentation{{< /nextlink >}}
{{< nextlink href="/tracing/trace_collection/dynamic_instrumentation/" >}}Add custom spans with Dynamic Instrumentation{{< /nextlink >}}
{{< /whatsnext >}}

**Custom instrumentation** captures observability data from in-house code or complex functions that aren't captured by automatic instrumentation, providing deeper visibility and context into spans when you need fine-grained control.
### Code-based setup and customization

The following table compares the different instrumentation methods available.
**Best for**: Teams that want full control over SDK configuration and span creation.

<div class="alert alert-info">
If you prefer vendor-neutral instrumentation, see the <a href="https://docs.datadoghq.com/tracing/trace_collection/custom_instrumentation/otel_instrumentation/">OpenTelemetry documentation</a> for using OpenTelemetry API support in Datadog libraries.
</div>
Use [Datadog SDKs][2] to manually configure instrumentation. If you need custom spans, you can also add [code-based custom instrumentation][3].

{{< whatsnext desc=" " >}}
{{< nextlink href="/tracing/trace_collection/dd_libraries/" >}}Get started with Datadog SDKs{{< /nextlink >}}
{{< nextlink href="/tracing/trace_collection/custom_instrumentation/" >}}Add custom spans with code-based instrumentation{{< /nextlink >}}
{{< /whatsnext >}}

## How instrumentation works

### SDK setup

Check warning on line 72 in content/en/tracing/trace_collection/_index.md

View workflow job for this annotation

GitHub Actions / vale

Datadog.headings

'SDK setup' should use sentence-style capitalization.

The first step is adding a Datadog SDK to your application. You have two options:

| Method | Description | Code changes? | Best for |
|--------|-------------|--------------|----------|
| [**Single Step Instrumentation (SSI)**][1] | Automatically installs and loads SDKs at runtime with a single command. | No | SREs and platform teams who want tracing across services without developer involvement. |
| [**Manual SDK setup**][2] | Manually install and configure SDKs in your application. | Yes | Development teams who need granular control over SDK configuration. |

### Span creation

After the SDK is loaded, spans are automatically created through auto-instrumentation for supported frameworks and libraries. This automatic span creation captures essential observability data with no additional work required.

If you need visibility into custom code paths or want to enrich traces with application-specific data, you can add custom spans:

| Method | Description | Code changes? | Best for |
|--------|-------------|--------------|----------|
| [**Code-based custom instrumentation**][3] | Add spans programmatically using Datadog tracing APIs. | Yes | Teams that need custom logic, specialized spans, or visibility into custom code paths. |
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As part of the different methods of Span creation, we should add Auto-instrumentation as the top row.

Method: Auto-Instrumentation
Description: Datadog automatically captures spans from your services once SDKs are initialised.
Code Changes?: No
Best for: Teams looking for fast, standardized tracing coverage across supported frameworks with minimal setup.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This table is specifically for custom span creation, so I think we should leave out auto-instrumentation? We can include that in the tables below but I think it would muddy the waters here.

| [**Dynamic Instrumentation**][4] | Add spans at runtime from the Datadog UI without code changes. | No | Teams that want to add spans without redeploying or modifying source code. |

## Decision guide

Use this guide to determine which combination of methods fits your needs:

| Goal | SDK setup | Span creation |
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Span creation column should mention auto-instrumentation for span creation instead of Automatic (built-in)

|------|-----------|---------------|
| Get started with minimal effort | [Single Step Instrumentation][1] | Auto-instrumentation |
| Manually control SDK setup but use auto-instrumentation to capture spans | [Datadog SDKs][2] | Auto-instrumentation |
| Automatically load SDKs and customize spans without code changes | [Single Step Instrumentation][1] | [Dynamic Instrumentation][4] |
| Automatically load SDKs and customize spans in code | [Single Step Instrumentation][1] | [Code-based custom instrumentation][3] |
| Manually configure SDKs and customize spans without code changes | [Datadog SDKs][2] | [Dynamic Instrumentation][4] |
| Have full control over both SDK setup and span creation | [Datadog SDKs][2] | [Code-based custom instrumentation][3] |

## Detailed comparison

### SDK setup options

Check warning on line 107 in content/en/tracing/trace_collection/_index.md

View workflow job for this annotation

GitHub Actions / vale

Datadog.headings

'SDK setup options' should use sentence-style capitalization.

<table style="width:100%; border-collapse:collapse; border:2px solid #999;">
<!-- Header group -->
<tr style="background-color:#d4d4e8;">
<th style="border:1px solid #ccc; background-color:#d4d4e8;"></th>
<th colspan="2" style="border:1px solid #ccc; border-right:2px solid #999; text-align:center; background-color:#d4d4e8; font-weight:bold; text-transform:uppercase; font-size:1.15em; padding:12px 8px;">
Automatic Instrumentation
</th>
<th colspan="2" style="border:1px solid #ccc; text-align:center; background-color:#d4d4e8; font-weight:bold; text-transform:uppercase; font-size:1.15em; padding:12px 8px;">
Custom Instrumentation
</th>
</tr>
<tr style="background-color:#f2f2f2;">
<th style="border:1px solid #ccc; text-transform:uppercase; font-weight:bold;"></th>
<th style="border:1px solid #ccc; text-transform:uppercase; font-weight:bold;"><a href="/tracing/trace_collection/automatic_instrumentation/single-step-apm/">Single Step Instrumentation</a></th>
<th style="border:1px solid #ccc; border-right:2px solid #999; text-transform:uppercase; font-weight:bold;"><a href="/tracing/trace_collection/automatic_instrumentation/dd_libraries/">Manually managed SDKs</a></th>
<th style="border:1px solid #ccc; text-transform:uppercase; font-weight:bold;"><a href="/tracing/trace_collection/custom_instrumentation/">Code-based Custom Instrumentation</a></th>
<th style="border:1px solid #ccc; text-transform:uppercase; font-weight:bold;"><a href="/tracing/dynamic_instrumentation/">Dynamic Instrumentation</a><br>(UI-based Custom Instrumentation)</th>
<th style="border:1px solid #ccc;"></th>
<th style="border:1px solid #ccc; font-weight:bold;"><a href="/tracing/trace_collection/single-step-apm/">Single Step Instrumentation</a></th>
<th style="border:1px solid #ccc; font-weight:bold;"><a href="/tracing/trace_collection/dd_libraries/">Datadog SDKs</a></th>
</tr>

<!-- Body rows -->
<tr>
<td style="border:1px solid #ccc; font-weight:bold;">Description</td>
<td style="border:1px solid #ccc;">With a single command, Datadog automatically loads language SDKs to your application processes. You can also control which processes to instrument.</td>
<td style="border:1px solid #ccc; border-right:2px solid #999;">Add Datadog language SDKs to your applications. The SDK handles instrumentation automatically.</td>
<td style="border:1px solid #ccc;">Add explicit tracing API calls or span logic in your application code.</td>
<td style="border:1px solid #ccc;">Add instrumentation rules in the Datadog UI. Rules are applied dynamically at runtime and do not require code changes.</td>
<td style="border:1px solid #ccc; font-weight:bold;">How it works</td>
<td style="border:1px solid #ccc;">Datadog automatically loads SDKs into your application processes with a single command.</td>
<td style="border:1px solid #ccc;">You install and configure SDKs directly in your application code or build process.</td>
</tr>
<tr>
<td style="border:1px solid #ccc; font-weight:bold;">Code changes?</td>
<td style="border:1px solid #ccc;">No</td>
<td style="border:1px solid #ccc; border-right:2px solid #999;">No</td>
<td style="border:1px solid #ccc;">Yes</td>
<td style="border:1px solid #ccc;">No</td>
</tr>
<tr>
<td style="border:1px solid #ccc; font-weight:bold;">Environment config changes?</td>
<td style="border:1px solid #ccc;">No</td>
<td style="border:1px solid #ccc; border-right:2px solid #999;">Yes</td>
<td style="border:1px solid #ccc;">Yes</td>
<td style="border:1px solid #ccc;">No</td>
<td style="border:1px solid #ccc; font-weight:bold;">Environment configuration</td>
<td style="border:1px solid #ccc;">Minimal</td>
<td style="border:1px solid #ccc;">Required</td>
</tr>
<tr>
<td style="border:1px solid #ccc; font-weight:bold;">Setup complexity</td>
<td style="border:1px solid #ccc;">Low</td>
<td style="border:1px solid #ccc; border-right:2px solid #999;">Medium</td>
<td style="border:1px solid #ccc;">High</td>
<td style="border:1px solid #ccc;">Low</td>
<td style="border:1px solid #ccc;">Medium</td>
</tr>
<tr>
<td style="border:1px solid #ccc; font-weight:bold;">Configuration control</td>
<td style="border:1px solid #ccc;">Standard defaults with optional overrides</td>
<td style="border:1px solid #ccc;">Full control through environment variables and code</td>
</tr>
<tr>
<td style="border:1px solid #ccc; font-weight:bold;">Best for</td>
<td style="border:1px solid #ccc;">SRE, admins, or central teams who want tracing across services without developer involvement.</td>
<td style="border:1px solid #ccc; border-right:2px solid #999;">App dev teams who want to instrument applications individually with granular control over configuration through environment variables.</td>
<td style="border:1px solid #ccc;">Teams needing custom logic, specialized spans, or visibility into custom code paths.</td>
<td style="border:1px solid #ccc;">Teams wanting to add spans, logs, or metrics to specific code locations at runtime without redeploying or modifying source code. Configuration is managed through the Datadog UI.</td>
<td style="border:1px solid #ccc;">SREs, platform teams, and organizations wanting fast, consistent instrumentation across services.</td>
<td style="border:1px solid #ccc;">Development teams needing granular control over SDK behavior and configuration.</td>
</tr>
</table>

### Span customization options

<table style="width:100%; border-collapse:collapse; border:2px solid #999;">
<tr style="background-color:#f2f2f2;">
<th style="border:1px solid #ccc;"></th>
<th style="border:1px solid #ccc; font-weight:bold;"><a href="/tracing/trace_collection/custom_instrumentation/">Code-based custom instrumentation</a></th>
<th style="border:1px solid #ccc; font-weight:bold;"><a href="/tracing/trace_collection/dynamic_instrumentation/">Dynamic Instrumentation</a></th>
</tr>
<tr>
<td style="border:1px solid #ccc; font-weight:bold;">How it works</td>
<td style="border:1px solid #ccc;">Add explicit tracing API calls in your application code.</td>
<td style="border:1px solid #ccc;">Configure instrumentation rules in the Datadog UI; rules are applied at runtime.</td>
</tr>
<tr>
<td style="border:1px solid #ccc; font-weight:bold;">Code changes?</td>
<td style="border:1px solid #ccc;">Yes</td>
<td style="border:1px solid #ccc;">No</td>
</tr>
<tr>
<td style="border:1px solid #ccc; font-weight:bold;">Deployment required</td>
<td style="border:1px solid #ccc;">Yes (to add or modify spans)</td>
<td style="border:1px solid #ccc;">No</td>
</tr>
<tr>
<td style="border:1px solid #ccc; font-weight:bold;">Setup complexity</td>
<td style="border:1px solid #ccc;">Medium to high</td>
<td style="border:1px solid #ccc;">Low</td>
</tr>
<tr>
<td style="border:1px solid #ccc; font-weight:bold;">Use cases</td>
<td colspan="2" style="border:1px solid #ccc; border-right:2px solid #999;">
<ul style="margin:0; padding-left:20px; font-size:inherit;">
<li style="font-size:inherit;">Capturing essential observability data across common libraries and languages with minimal configuration.</li>
<li style="font-size:inherit;">Enabling real-time monitoring with pre-configured settings for immediate insights into application performance.</li>
<li style="font-size:inherit;">Simplifying the observability setup for projects where custom instrumentation is not required.</li>
<td style="border:1px solid #ccc;">
<ul style="margin:0; padding-left:20px;">
<li>Instrumenting custom business logic</li>
<li>Adding application-specific metadata to spans</li>
<li>Removing unwanted spans from traces</li>
<li>Complex span manipulation</li>
</ul>
</td>
<td colspan="2" style="border:1px solid #ccc;">
<ul style="margin:0; padding-left:20px; font-size:inherit;">
<li style="font-size:inherit;">Collecting observability data from custom code with unique or complex business logic.</li>
<li style="font-size:inherit;">Providing deeper visibility and context into spans, including adding span tags.</li>
<li style="font-size:inherit;">Precisely monitoring specific sequences of operations or user interactions that require fine-grained control.</li>
<li style="font-size:inherit;">Removing unwanted spans from traces.</li>
<td style="border:1px solid #ccc;">
<ul style="margin:0; padding-left:20px;">
<li>Adding spans to production without redeploying</li>
<li>Debugging specific code paths</li>
<li>Adding logs or metrics at runtime</li>
<li>Fast instrumentation changes</li>
</ul>
</td>
</tr>
<tr>
<td style="border:1px solid #ccc; font-weight:bold;">Best for</td>
<td style="border:1px solid #ccc;">Teams with complex instrumentation needs who want spans defined in code.</td>
<td style="border:1px solid #ccc;">Teams wanting to add or modify spans without code changes or redeployments.</td>
</tr>
</table>


## APM setup tutorials

The following tutorials guide you through setting up distributed tracing for a sample application on various infrastructure scenarios, with both automatic and custom instrumentation, using the Datadog tracing libraries:
The following tutorials guide you through setting up distributed tracing for a sample application on various infrastructure scenarios, with both automatic and custom instrumentation:

{{< whatsnext desc="Choose your language and environment:" >}}
{{< nextlink href="tracing/guide/tutorial-enable-python-host" >}}<img src="/images/integrations_logos/python-avatar.png" /> <img src="/images/tracing/guide/tutorials/tutorial-host-icon.png" /> Enabling Tracing on a Python Application on the Same Host as Datadog Agent{{< /nextlink >}}
Expand All @@ -143,7 +210,7 @@
{{< nextlink href="tracing/guide/tutorial-enable-java-containers" >}}<img src="/images/integrations_logos/java-avatar.png" /> <img src="/images/tracing/guide/tutorials/tutorial-container-icon.png" /> Enabling Tracing on a Java Application and Datadog Agent in Containers{{< /nextlink >}}
{{< nextlink href="tracing/guide/tutorial-enable-java-container-agent-host" >}}<img src="/images/integrations_logos/java-avatar.png" /> <img src="/images/tracing/guide/tutorials/tutorial-container-icon.png" /> <img src="/images/tracing/guide/tutorials/tutorial-host-icon.png" /> Enabling Tracing for a Java Application in a Container and an Agent on a Host{{< /nextlink >}}
{{< nextlink href="tracing/guide/tutorial-enable-java-gke" >}}<img src="/images/integrations_logos/java-avatar.png" /> <img src="/images/tracing/guide/tutorials/tutorial-gke-icon.png" /> Enabling Tracing for a Java Application on GKE{{< /nextlink >}}
{{< nextlink href="tracing/guide/tutorial-enable-java-aws-eks" >}}<img src="/images/integrations_logos/java-avatar.png" /> <img src="/images/tracing/guide/tutorials/tutorial-eks-icon.png" /> Enabling Tracing for a Java Application on AWS EKS{{< /nextlink >}}
{{< nextlink href="tracing/guide/tutorial-enable-java-aws-eks" >}}<img src="/images/integrations_logos/java-avatar.png" /> <img src="/images/tracing/guide/tutorials/tutorial-eks-icon.png" /> Enabling Tracing for a Java Application on Amazon EKS{{< /nextlink >}}
{{< nextlink href="tracing/guide/tutorial-enable-java-aws-ecs-ec2" >}}<img src="/images/integrations_logos/java-avatar.png" /> <img src="/images/tracing/guide/tutorials/tutorial-ec2-icon.png" /> Enabling Tracing for a Java Application in Amazon ECS with EC2{{< /nextlink >}}
{{< nextlink href="tracing/guide/tutorial-enable-java-aws-ecs-fargate" >}}<img src="/images/integrations_logos/java-avatar.png" /> <img src="/images/tracing/guide/tutorials/tutorial-fargate-icon.png" /> Enabling Tracing for a Java Application in Amazon ECS with Fargate{{< /nextlink >}}
{{< nextlink href="tracing/guide/tutorial-enable-java-admission-controller" >}}<img src="/images/integrations_logos/java-avatar.png" /> Enabling Tracing for a Java Application with the Admission Controller{{< /nextlink >}}
Expand All @@ -153,7 +220,14 @@
{{< nextlink href="tracing/guide/tutorial-enable-go-aws-ecs-fargate" >}}<img src="/images/integrations_logos/golang-avatar.png" /> <img src="/images/tracing/guide/tutorials/tutorial-fargate-icon.png" /> Enabling Tracing for a Go Application in Amazon ECS with Fargate{{< /nextlink >}}

{{< /whatsnext >}}

## Further reading

{{< partial name="whats-next/whats-next.html" >}}

[1]: /tracing/trace_collection/single-step-apm/
[2]: /tracing/trace_collection/dd_libraries/
[3]: /tracing/trace_collection/custom_instrumentation/
[4]: /tracing/trace_collection/dynamic_instrumentation/
[5]: /agent/

Loading