Skip to content

Latest commit

 

History

History
201 lines (138 loc) · 8.06 KB

File metadata and controls

201 lines (138 loc) · 8.06 KB

Observing a Workflow with W&B Weave

This guide provides a step-by-step process to enable observability in a NeMo Agent Toolkit workflow using Weights and Biases (W&B) Weave for tracing using just a few lines of code in your workflow configuration file.

Weave Tracing Dashboard

Prerequisites

An account on Weights & Biases is required to use Weave.

You can create an account on Weights & Biases by clicking on the "Sign Up" button in the top right corner of the website.

Under the "Account" section, you can find your API key. Click on the "Show" button to reveal the API key. Take note of this API key as you will need it to run the workflow.

export WANDB_API_KEY=<your_api_key>

Step 1: Install the Weave plugin

To install the Weave plugin, run one of the following commands, depending on whether you installed the NeMo Agent Toolkit from source or from a package.

::::{tab-set} :sync-group: install-tool

:::{tab-item} source :selected: :sync: source

uv pip install -e ".[weave]"

:::

:::{tab-item} package :sync: package

uv pip install "nvidia-nat[weave]"

:::

::::

Step 2: Install the Workflow

Pick an example from the list of available workflows. In this guide, we will be using the simple_calculator example.

uv pip install -e examples/observability/simple_calculator_observability

Step 3: Modify Workflow Configuration

Update your workflow configuration file to include the weave telemetry settings. For example, examples/observability/simple_calculator_observability/configs/config-weave.yml has the following weave settings:

general:
  telemetry:
    tracing:
      weave:
        _type: weave
        project: "nat-demo"

This setup enables logging trace data to W&B weave. The weave integration only requires the project parameter to be set.

Parameter Description Example
project The name of your W&B Weave project "nat-demo"
entity (deprecated) Your W&B username or team name "your-wandb-username-or-teamname"

Step 4: Run Your Workflow

Install simple_calculator example using the instructions in the examples/observability/simple_calculator_observability/README.md guide. Run the workflow using config-weave.yml configuration file:

nat run --config_file examples/observability/simple_calculator_observability/configs/config-weave.yml --input "Is the product of 2 * 4 greater than the current hour of the day?"

If it is your first time running the workflow, you will be prompted to login to W&B Weave.

Step 5: View Traces Data in Weave Dashboard

As the workflow runs, you will find a Weave URL (starting with a 🍩 emoji). Click on the URL to access your logged trace timeline.

Note how the integration captures not only the nat intermediate steps but also the underlying framework. This is because Weave has integrations with many of your favorite frameworks.

Step 6: Redacting Sensitive Data

When tracing LLM workflows, you may be processing sensitive information like personal identifiers, credit card numbers, or API keys. NeMo Agent Toolkit Weave integration supports automatic redaction of Personally Identifiable Information (PII) and sensitive keys from your traces.

Prerequisites

To enable PII redaction, you need presidio-analyzer and presidio-anonymizer installed. Installing the weave plugin will install these packages for you.

Enabling PII Redaction

Update your workflow configuration to enable PII redaction:

general:
  telemetry:
    tracing:
      weave:
        _type: weave
        project: "nat-demo"
        redact_pii: true                    # Enable PII redaction
        redact_pii_fields:                  # Optional: specify which entity types to redact
          - EMAIL_ADDRESS
          - PHONE_NUMBER
          - CREDIT_CARD
          - US_SSN
          - PERSON
        redact_keys:                        # Optional: specify additional keys to redact
          - custom_secret
          - api_key
          - auth_token

Redaction Options

The Weave integration supports the following redaction options:

Parameter Description Required
redact_pii Enable PII redaction (true/false) No (default: false)
redact_pii_fields List of PII entity types to redact No (default: all supported entities)
redact_keys List of additional keys to redact beyond the defaults No

When redact_pii is enabled, common PII entities like email addresses, phone numbers, credit cards, and more are automatically redacted from your traces before they are sent to Weave. The redact_pii_fields parameter allows you to customize which entity types to redact.

See the Microsoft Presidio documentation for a full list of supported entity types.

Additionally, the redact_keys parameter allows you to specify custom keys that should be redacted beyond the default sensitive keys (api_key, auth_headers, authorization).

User Feedback Integration

When using Weave telemetry with the FastAPI front end, you can enable a /feedback endpoint that allows users to provide thumbs-up and thumbs-down feedback on agent responses. This feedback is linked to specific traces in your Weave project for analysis.

Enabling the Feedback Endpoint

To enable the feedback endpoint, configure your workflow to use the WeaveFastAPIPluginWorker:

general:
  front_end:
    _type: fastapi
    runner_class: nat.plugins.weave.fastapi_plugin_worker.WeaveFastAPIPluginWorker
  telemetry:
    tracing:
      weave:
        _type: weave
        project: "nat-demo"

The WeaveFastAPIPluginWorker registers the /feedback endpoint when Weave telemetry is configured. For more details on the feedback API, see the API Server Endpoints documentation.

User Attribution

To associate traces and feedback with a specific user, set one or more of the following fields on the NeMo Agent Toolkit context metadata from within your authentication callback: trace_user_name, trace_user_email, or trace_user_id. These trace_-prefixed fields are an explicit opt-in, so general identity fields set on the metadata for other purposes will not affect Weave attribution.

Any fields that are set will be written to the Weave call summary. When a user submits feedback through the /feedback endpoint, the first available value is used in priority order (trace_user_nametrace_user_emailtrace_user_id), falling back to anonymous.

The following example shows how to set these fields from within an authentication callback:

from nat.runtime.context import Context

def my_auth_callback(request):
    user_info = authenticate(request)  # returns user data from your identity provider
    context = Context.get()
    context.metadata.trace_user_name = user_info.get("name")
    context.metadata.trace_user_email = user_info.get("email")

Resources

  • Learn more about tracing here.
  • Learn more about how to navigate the logged traces here.
  • Learn more about PII redaction here.