End-to-End Observability in Go with OTel and SigNoz

• End-to-End Observability in Go with OTel and SigNoz
  • Prerequisites
  • Setup Instructions
    • 1. Clone the repository
    • 2. Install Go Dependencies
    • 3. Prepare the OpenTelemetry Collector Config File
    • 4. Run the OpenTelemetry Collector
    • 5. Run the Go Service
      • Test the order endpoint:
      • Test the inventory endpoint:
  • Architecture Overview
  • Troubleshooting
    • 1. Traces, Metrics, or Logs are Not Appearing in SigNoz
      • Is the OpenTelemetry Collector Running?
      • Is the Collector Configured Correctly?
      • Is the Go App Connecting to the Collector?
    • 2. The Go Service Fails to Start
      • Did you run go mod tidy?
  • Next Steps

This project demonstrates how to capture traces, metrics, and logs from HTTP-based microservices in Go, send them to an OpenTelemetry Collector running locally, and visualize them in SigNoz Cloud.

For a deeper dive, visit this blog post

---

Prerequisites

• Go 1.23+
• SigNoz Cloud account (or self-hosted SigNoz)
• SigNoz Ingestion Key
• OpenTelemetry Collector

Note:
This project was built and tested on Windows, with the OpenTelemetry Collector running locally on Windows.
You can also run the Collector on Linux or macOS, or use a Docker container instead of the native binary. The configuration and telemetry flow remain the same across platforms.

---

Setup Instructions

1. Clone the repository

git clone https://github.com/ShrutiC-git/go-otel-e2e.git
cd go-otel-e2e

2. Install Go Dependencies

go mod tidy

This will download all dependencies required to run the application.

3. Prepare the OpenTelemetry Collector Config File

1. Copy the provided config.yaml into the same folder as the OpenTelemetry Collector binary.
2. Open config.yaml in a text editor.
3. Update the following fields to point to your SigNoz Cloud account:

Field	Description	Example / Notes
endpoint	OTLP endpoint for sending telemetry data	ingest.<YOUR_SIGNOZ_REGION>.signoz.cloud:443
signoz-ingestion-key	Your SigNoz ingestion key	<YOUR_SIGNOZ_INGESTION_KEY>

4. Save the file.

Once configured, the Collector will export traces, metrics, and logs from your Go services to SigNoz Cloud.

4. Run the OpenTelemetry Collector

1. Open PowerShell or Command Prompt.
2. Navigate to the folder containing otelcol-contrib.exe.
3. Run the Collector with the config file:

.\otelcol-contrib.exe --config config.yaml

If your config file is in a different path:

.\otelcol-contrib.exe --config <path-to-config.yaml>

You should see Health Check is ready in the logs, indicating the Collector is running.

Note: otelcol-contrib is the name of the OpenTelemetry Collector executable. This specific version includes additional exporters and receivers needed for this project.

5. Run the Go Service

Start the Go application:

go run main.go

The service will start on port 8080 and expose two sample endpoints:

• POST http://localhost:8080/createOrder
• GET http://localhost:8080/checkInventory

Test the order endpoint:

curl -X POST http://localhost:8080/createOrder

Test the inventory endpoint:

curl http://localhost:8080/checkInventory

Once the OpenTelemetry Collector is running, all requests to these endpoints will be recorded, and the traces, metrics, and logs will be visible in SigNoz Cloud.

6. (Optional) Generate Traffic (Bash)

To light up traces/metrics in SigNoz, run a tiny bash loop:

# 60 requests over ~60s (low and steady)
for i in {1..60}; do
  curl -s -o /dev/null -w "%{http_code}\n" -X POST http://localhost:8080/createOrder
  curl -s -o /dev/null -w "%{http_code}\n"        http://localhost:8080/checkInventory
  sleep 1
done

---

Architecture Overview

+------------------+       +----------------------+       +----------------+
|  Go Microservice | ----> | OTel Collector (local)| ---> |   SigNoz Cloud |
| (with OTel SDK)  |       |  Receives traces via |       | (Backend + UI) |
|                  |       |  OTLP (http/grpc)    |       |                |
+------------------+       +----------------------+       +----------------+

---

Troubleshooting

If you're having issues, here are a few common problems and their solutions.

1. Traces, Metrics, or Logs are Not Appearing in SigNoz

This is usually caused by a connectivity or configuration issue.

Is the OpenTelemetry Collector Running?

Confirm that the Collector binary is running in your terminal and you see the Health Check is ready message. If not, double-check that you've navigated to the correct directory and run the command from Step 4 correctly.

Alternatively, you can also check Windows Service to check the OTel Collector is running.

Is the Collector Configured Correctly?

Verify that your config.yaml file contains the correct endpoint and signoz-ingestion-key for your SigNoz Cloud account. A simple typo in the ingestion key can prevent data from being ingested. Check Windows Application Logs if issues persist.

Is the Go App Connecting to the Collector?

The Go service is configured to send telemetry to localhost:4318. Make sure your local OpenTelemetry Collector is running and listening on this address and port.

Check for any firewall rules that might be blocking the connection between your Go application and the Collector.

2. The Go Service Fails to Start

If the application won't start, it's often a dependency issue.

Did you run go mod tidy?

Ensure you have run go mod tidy after cloning the repository to download all necessary dependencies. Check your terminal output for any errors during this process.

---

Next Steps

• Explore the traces in the SigNoz UI under the Traces tab.
• Check out the Metrics and Logs dashboards.