Add Dynamic Sampling Context + Baggage Spec

[smeubank]

In Develop Docs

Write down baggage propagation “spec” to define clear rules of how baggage is created and passed along downstream.

clarify confusion around naming:

• trace context vs baggage
• "propagation mechanism"
• content - what is in it?
  • call it dynamic sampling context
  • meaning: the key values included in the baggage header

The following list is probably not complete but it should include (writing this down after the meeting on Daily from 18th):

• Check if incoming request has baggage header and parse it to see if there’s sentry- entries in it
• If not, create the baggage content (i.e. the sentry- keys) and propagate it with the outgoing requests
  • include the specific contect types (release, environment, user, transaction, usergroup)
• If yes, leave it untouched (don’t add/change stuff) and send it along with the outgoing request
• In any of those two cases, “foreign” baggage entries must be passed along with the next outgoing requests (if they exist)

reference existing docs

#425

#409

[smeubank]

Problem:
As customer adopt DS with in product Sentry server side traces and transaction sample rate rules, then they should change the sample rate in their applications. In the situations where there is distributed tracing, there could be conflicting sample rates on different projects, leading to issues where we cannot provide DS

where should it be sent?

• in dynamic sample context (by baggage)
• in the envelope header
• meta data (today in JS, but will become unnecessary)

Does what we sent in the envelope header need to be same as http header (baggage)?

• where does the public key need to be?

[antonpirker]

A list of PRs that @Lms24 sent me with all the baggage functionality in JS. Maybe this helps writing the documentation?

1. Baggage data structures and operations

   • feat(utils): Introduce Baggage API sentry-javascript#5066

2. Baggage data in Envelope Header

   • feat(core): Send Baggage in Envelope Header sentry-javascript#5104
   • ref: Rename baggage env header to trace sentry-javascript#5128
   • (Mutability of Baggage only handled in 5.)

3. Baggage passing on to outgoing requests (without data population)

   • feat(tracing): Add empty baggage header propagation to outgoing requests sentry-javascript#5133

4. Release and Environment propagation

   • feat(tracing): Propagate environment and release values in baggage Http headers sentry-javascript#5193

5. Mutability of Baggage:

   • ref(tracing): Check and set mutability of baggage sentry-javascript#5205
   • ref(tracing): Sync baggage data in Http and envelope headers sentry-javascript#5218

[smeubank]

getsentry/sentry-javascript#5290 (comment)

for reference of what concerns to address see this comment. Notes from SDK DS sync. Transactions have platform specific complexities. Maybe a table where devs can include descriptions and examples of such can be included

[smeubank]

confirming list of relevant values:

Trace-aware head sampling based on trace state information from the frontend SDK (requires SDK support):

• Transaction name
• Release
• Environment
• User ID
• User Segment
• No condition - uniform sampling rate

@ale-cota can you clarify what "No condition" means?

[Lms24]

@smeubank to add on the list posted above:

@lforst and I talked about names/keys of values to be propagated via baggage and sent via an envelope header. He'll include a detailed list once his PR is ready but this is what we came up with:

Baggage keys:

• sentry-traceid
• sentry-publickey
• sentry-samplerate
• sentry-environment
• sentry-release
• sentry-transaction
• sentry-userid
• sentry-usersegment

Items in trace envelope header
Envelope header items for trace header are described in Develop docs as well as implemented in Relay.
Additionally, in Relay a field must be added for the sample rate, which we propose to call

• sample_rate (for consistency with other fields)

[AbhiPrasad]

As part of these docs, I would like to take some time to establish the terms we refer to these different concepts.

Traces, transactions, and spans have all been long defined in our product docs.

In event payload (so both errors and transactions), there exists the contexts interface, one of which is the Trace Contexts (a contexts keyed by trace). I believe this should continue to be referred to as such.

We then have the key-value bag needed to make dynamic sampling decisions. Previously this was called Trace Contexts also, but I propose it be called Dynamic Sampling Context (thanks @lobsterkatie for the suggestion). This dynamic sampling context is used in two ways:

1. It is sent to Sentry through the envelope header, keyed by trace.

2. It is propogated between services (and so between different Sentry SDKs running in those services) so that dynamic sampling can occur on entire traces. Currently the mechanism for propogation we have chosen for the SDKs is leveraging the baggage HTTP header, based on the w3c baggage spec.

[mitsuhiko]

I like the idea of calling this dynamic sampling context, so that gets a buy-in from me. Annoyingly on the relay side the TraceContext and the dynamic sampling context are in fact synonyms right now I believe. I think that in the protocol it's in fact currently the case that the trace item in the envelope headers have exactly the same format as the trace context in the protocol but someone from relay would need to double check that.

[jjbayer]

@mitsuhiko In relay, these are two different structs with the same name. As far as I can see, trace_id is the only overlap:
https://github.com/getsentry/relay/blob/d217da6fdf3392f0476a0b019223e267c7c90df0/relay-sampling/src/lib.rs#L687
https://github.com/getsentry/relay/blob/d217da6fdf3392f0476a0b019223e267c7c90df0/relay-general/src/protocol/contexts.rs#L475

[lforst]

I just opened #613 to get this specced. It would be cool if we could get some early feedback.