Skip to content

Commit 143ab85

Browse files
authored
Docs: Tail Sampling Caveat: Spans starting after root ended, e.g. background tasks (#1033)
1 parent 09db802 commit 143ab85

File tree

1 file changed

+60
-0
lines changed

1 file changed

+60
-0
lines changed

docs/how-to-guides/sampling.md

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -202,6 +202,66 @@ If you start a trace with the Logfire SDK with tail sampling, and then propagate
202202
spans generated by the SDK may be discarded, while the spans generated by the other process may be included, leading to
203203
an incomplete trace.
204204

205+
### Spans starting after root ended, e.g. background tasks
206+
207+
When the root span of a trace ends, if the trace doesn't meet the tail sampling criteria, all spans in the trace are
208+
discarded. If you start a new span in that trace (i.e. as a descendant of the root span) after the root span has ended,
209+
the new span will always be included anyway, and its parent will be missing. This is because the tail sampling mechanism
210+
only keeps track of active traces to save memory. This is similar to the distributed tracing case above.
211+
212+
Here's an example with a FastAPI background task which starts after the root span corresponding to the request has
213+
ended:
214+
215+
```python
216+
import uvicorn
217+
from fastapi import BackgroundTasks, FastAPI
218+
219+
import logfire
220+
221+
app = FastAPI()
222+
223+
logfire.configure(
224+
sampling=logfire.SamplingOptions.level_or_duration(
225+
duration_threshold=0.1,
226+
),
227+
)
228+
logfire.instrument_fastapi(app)
229+
230+
231+
async def background_task():
232+
# This will be included even if the root span was excluded.
233+
logfire.info('background')
234+
235+
236+
@app.get('/')
237+
async def index(background_tasks: BackgroundTasks):
238+
# Uncomment to prevent request span from being sampled out.
239+
# await asyncio.sleep(0.2)
240+
241+
background_tasks.add_task(background_task)
242+
return {}
243+
244+
245+
uvicorn.run(app)
246+
```
247+
248+
A workaround is to explicitly put the new spans in their own trace using [
249+
`attach_context`][logfire.propagate.attach_context]:
250+
251+
```python
252+
from logfire.propagate import attach_context
253+
254+
255+
async def background_task():
256+
# `attach_context({})` forgets existing context
257+
# so that spans within start a new trace.
258+
with attach_context({}):
259+
with logfire.span('new trace'):
260+
await asyncio.sleep(0.2)
261+
logfire.info('background')
262+
263+
```
264+
205265
## Custom head sampling
206266

207267
If you need more control than random sampling, you can pass an [OpenTelemetry

0 commit comments

Comments
 (0)