Skip to content

Feat/usage metering#4

Open
tenbytecloud wants to merge 67 commits intoflexprice_v1.0.47from
feat/usage-metering
Open

Feat/usage metering#4
tenbytecloud wants to merge 67 commits intoflexprice_v1.0.47from
feat/usage-metering

Conversation

@tenbytecloud
Copy link
Copy Markdown

@tenbytecloud tenbytecloud commented Feb 1, 2026

Fixes #

📝 Description

Please provide a clear and concise description of what this PR does.


🔨 Changes Made

  • Feature 1
  • Bug Fix
  • Refactor
  • Documentation Update

✅ Checklist

  • My code follows the project's code style.
  • I have added tests where applicable.
  • All new and existing tests pass.
  • I have updated documentation (if needed).

📷 Screenshots / Demo (if applicable)

Add screenshots or demo links here.


Disclaimer: This comment was generated by Greptile AI CTO

Greptile Overview

Greptile Summary

This PR delivers a comprehensive billing enhancement for Tenbyte's usage-metering requirements: SSLCommerz payment gateway integration for Bangladesh market, real-time overage billing with $5 threshold buckets, and unified webhook-driven upgrade flow across all payment methods.

Key Changes:

  • SSLCommerz Integration: Full payment gateway with IPN webhook handling, MD5 validation, high-risk transaction detection. Clean separation follows existing Stripe/Razorpay patterns.
  • Overage Billing: Bucket-based invoicing ($5 thresholds), idempotency-protected, allows unlimited negative wallet balances for post-paid usage. 766 lines of tests (21 test cases) demonstrate thorough validation.
  • Unified Upgrade Flow: All payment methods (Stripe auto-charge, SSLCommerz, links) now execute upgrades via wallet.transaction.created webhook, eliminating inconsistent behavior.
  • Security Fixes: CORS origin whitelisting (addresses P1 from previous review), credentials only with specific origins, removed debug logs leaking endpoints.
  • Balance Tracking: Added balance_before/balance_after columns for audit trail (migration defaults to 0 for historical data).

Implementation Quality:
Solid production-ready code. The team kept it lean—no over-engineering, clear separation of concerns, proper error handling without failing event processing. Documentation (TENBYTE_CHANGES.md) is excellent for future maintenance.

Issues Found:

  • Webhook spam risk in overage billing (line 209-212): sends duplicate wallet.balance.negative webhooks when balance stays negative across multiple bucket catches-up.

Confidence Score: 4/5

  • Safe to merge with one minor webhook logic fix recommended
  • Strong implementation with comprehensive test coverage (766 lines), proper error handling, security fixes applied. One webhook spam issue found but won't cause data corruption—just user experience degradation under specific scenarios. Team demonstrated discipline keeping changes focused and production-ready.
  • internal/service/overage_billing.go (webhook logic at line 209-212)

Important Files Changed

Filename Overview
internal/service/overage_billing.go New overage billing service with bucket-based threshold invoicing ($5 buckets). Handles race conditions with idempotency, includes negative balance webhook, comprehensive test coverage (21 tests).
internal/integration/sslcommerz/webhook/handler.go SSLCommerz IPN webhook handler. Validates via API, handles high-risk transactions, uses original payment amount (USD) not IPN amount (BDT). Never returns errors to ensure 200 OK.
internal/service/wallet.go Added balance_before/balance_after tracking for audit trail. Refactored to return transaction object directly. No functional changes to core wallet logic.
internal/service/event_post_processing.go Integrated overage billing after cost calculation. Added MAX aggregation support. Sentry error capture for monitoring. Clean integration point with proper error handling.
internal/rest/middleware/cors.go Security fix: origin whitelisting, proper credentials handling (no credentials with wildcard), Vary header for cache safety. Addresses P1 CORS issue from previous review.
internal/api/v1/webhook.go SSLCommerz webhook endpoint with goroutine processing. Responds immediately with 200 OK, processes in background with panic recovery. Context handling discussed in previous thread.
migrations/postgres/V5__wallet_transaction_balance_fields.up.sql Adds balance_before/balance_after columns with default 0 for existing rows. Clean migration, backward compatible. Historical snapshots only accurate going forward (as noted in previous review).
internal/service/billing.go Unified upgrade flow - all payment methods (Stripe auto-charge, SSLCommerz, payment links) now use webhook to execute upgrade. Ensures consistency across payment methods.

srtenbyte and others added 12 commits January 7, 2026 02:00
- fix cors issue
- fix wallet tx issue
- fix field missing issue
   - Add SSLCommerz client with session API and validation API support
   - Add payment service for creating SSLCommerz payment links
   - Add webhook handler for IPN callbacks
   - Add SSLCommerz configuration in config.go and config.yaml
   - Add SSLCommerz connection metadata types
   - Add SSLCommerz case in connection repository for metadata conversion
   - Add SSLCommerz payment gateway type and secret provider
   - Integrate SSLCommerz in payment processor service
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
  SSLCommerz improvements:
  - Use PaymentID as TranID for unique transaction identification
  - Update webhook handler to lookup payments by PaymentID
  - Remove redundant GatewayPageURL field from response DTO
  - Fix BaseFair -> BaseFare spelling in validation response

  Security enhancements:
  - Add CORS origin whitelisting with configurable allowed_origins
  - Remove store credentials from config (use Connection API instead)
  - Remove seed script in favor of POST /connections endpoint

  Code cleanup:
  - Separate error handling for service errors vs empty results
  - Remove unused SSLCommerz flags from scripts/main.go
  - Add InvoiceCadence field mapping in GetSubscriptionFromEnt()
  - Add balance_before and balance_after fields to wallet transaction:
    - Update ent schema with new decimal fields
    - Update ToEnt() and FromEnt() conversion functions
    - Add V5 migration files (up/down) for new columns
  Add real-time overage billing that detects overage during event processing,
  accumulates cost per subscription, creates invoices when threshold is reached,
  and deducts from wallet (allowing negative balance).

  Key features:
  - OverageBillingService processes events for overage billing
  - Configurable threshold via overage_billing_config setting (default )
  - Invoice created with "Real-time overage charges" description
  - Wallet payment processed automatically (allows negative balance)
  - Webhook notification (wallet.balance.negative) when wallet crosses to negative

  Files added:
  - internal/service/overage_billing.go - main overage billing service
  - internal/service/overage_billing_test.go - comprehensive test suite (21 tests)
  - internal/testutil/mock_processed_event_repo.go - mock for testing

  Files modified:
  - internal/service/event_post_processing.go - hook overage billing after cost calc
  - internal/types/settings.go - add OverageBillingConfig type and setting key
  - internal/types/webhook.go - add WebhookEventWalletBalanceNegative event
  - internal/webhook/dto/wallet.go - add WalletNegativeBalancePayload
  - internal/webhook/payload/wallet.go - add payload builder
  - internal/webhook/payload/factory.go - register new payload builder add SSLCommerz payment gateway integration

   - Add SSLCommerz client with session API and validation API support
   - Add payment service for creating SSLCommerz payment links
   - Add webhook handler for IPN callbacks
   - Add SSLCommerz configuration in config.go and config.yaml
   - Add SSLCommerz connection metadata types
   - Add SSLCommerz case in connection repository for metadata conversion
   - Add SSLCommerz payment gateway type and secret provider
   - Integrate SSLCommerz in payment processor service
  - Remove immediate upgrade execution after Stripe auto-charge success
  - Let wallet.transaction.created webhook handle all upgrades consistently
  - Add idempotent cleanup when subscription is already cancelled
  - Add findActiveSubscriptionByPlan helper to find new subscription for cleanup
  - Fix transaction reason to use MANUAL_BALANCE_DEBIT (valid in FlexPrice)

  This ensures Stripe, SSLCommerz, and payment link upgrades all follow
  the same flow: payment -> webhook -> execute upgrade -> cleanupfix: webhook controller issue
Copy link
Copy Markdown

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

8 files reviewed, 10 comments

Edit Code Review Agent Settings | Greptile

- Fix threshold bucket calculation to use Floor() for explicit math intent
- Add Sentry error capture for overage billing failures (alerting)
- Document MAX aggregation eventual consistency edge case and mitigations

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@tenbytecloud
Copy link
Copy Markdown
Author

@greptileai review

Copy link
Copy Markdown

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

5 files reviewed, 1 comment

Edit Code Review Agent Settings | Greptile

@greptile-apps
Copy link
Copy Markdown

greptile-apps bot commented Feb 2, 2026

Additional Comments (1)

internal/service/event_post_processing.go
CRITICAL BUG: Logic is inverted. BILLING_MODEL_FLAT_FEE should NOT be used for usage-based billing. This returns true for FLAT_FEE and rejects all usage-based models.

All usage events will be skipped because they use usage-based billing models, not FLAT_FEE. This breaks the entire metering system.

	return billingModel != types.BILLING_MODEL_FLAT_FEE
Prompt To Fix With AI
This is a comment left during a code review.
Path: internal/service/event_post_processing.go
Line: 778:778

Comment:
CRITICAL BUG: Logic is inverted. `BILLING_MODEL_FLAT_FEE` should NOT be used for usage-based billing. This returns `true` for FLAT_FEE and rejects all usage-based models.

All usage events will be skipped because they use usage-based billing models, not FLAT_FEE. This breaks the entire metering system.

```suggestion
	return billingModel != types.BILLING_MODEL_FLAT_FEE
```

How can I resolve this? If you propose a fix, please make it concise.

- Remove debug logs from webhook payload builder
- Fix misleading comment in isSupportedBillingModel (FLAT_FEE is correct)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Copy link
Copy Markdown

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

5 files reviewed, 4 comments

Edit Code Review Agent Settings | Greptile

@greptile-apps
Copy link
Copy Markdown

greptile-apps bot commented Feb 2, 2026

Additional Comments (1)

internal/service/event_post_processing.go
[P0] Billing model support check is inverted

isSupportedBillingModel is documented as excluding FLAT_FEE (“not appropriate for usage-based billing”), but currently returns true only for BILLING_MODEL_FLAT_FEE (event_post_processing.go:774-778). This means usage line items with PACKAGE/TIERED billing models will be treated as unsupported and silently skipped at prepareProcessedEvents (via isSupportedAggregationForPostProcessing). That will break metering + overage billing for the models you likely care about.

Recommendation: flip the predicate to allow the actual usage billing models (and explicitly exclude FLAT_FEE). This is a small change with a big functional impact.

Prompt To Fix With AI
This is a comment left during a code review.
Path: internal/service/event_post_processing.go
Line: 774:778

Comment:
[P0] Billing model support check is inverted

`isSupportedBillingModel` is documented as *excluding* `FLAT_FEE` (“not appropriate for usage-based billing”), but currently returns `true` only for `BILLING_MODEL_FLAT_FEE` (`event_post_processing.go:774-778`). This means usage line items with PACKAGE/TIERED billing models will be treated as unsupported and silently skipped at `prepareProcessedEvents` (via `isSupportedAggregationForPostProcessing`). That will break metering + overage billing for the models you likely care about.

Recommendation: flip the predicate to allow the actual usage billing models (and explicitly exclude `FLAT_FEE`). This is a small change with a big functional impact.

How can I resolve this? If you propose a fix, please make it concise.

@tenbytecloud
Copy link
Copy Markdown
Author

@greptileai review again

- P1: Fix CORS to not set credentials with wildcard origin
- P2: Remove debug logs leaking tenant webhook endpoints
- P3: Reuse sentry service instance in hot error path

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Copy link
Copy Markdown

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

5 files reviewed, 2 comments

Edit Code Review Agent Settings | Greptile

@greptile-apps
Copy link
Copy Markdown

greptile-apps bot commented Feb 2, 2026

Additional Comments (1)

internal/config/config.go
Debug fmt.Printf still logging tenant webhook endpoints to stdout. Remove before production - bypasses structured logging and may leak sensitive URLs.

	// Debug logging removed - use structured logger if needed
Prompt To Fix With AI
This is a comment left during a code review.
Path: internal/config/config.go
Line: 322:326

Comment:
Debug `fmt.Printf` still logging tenant webhook endpoints to stdout. Remove before production - bypasses structured logging and may leak sensitive URLs.

```suggestion
	// Debug logging removed - use structured logger if needed
```

How can I resolve this? If you propose a fix, please make it concise.

Document all changes made to FlexPrice for Tenbyte requirements:
- SSLCommerz payment gateway integration
- Real-time overage billing with threshold-based invoicing
- Unified upgrade flow via webhooks
- Schema & database changes
- Security improvements
- Configuration changes

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@tenbytecloud
Copy link
Copy Markdown
Author

@greptileai review all the files within this PR

Copy link
Copy Markdown

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

8 files reviewed, 1 comment

Edit Code Review Agent Settings | Greptile

Shah Ahmed Saad Rupai and others added 3 commits February 3, 2026 12:28
Adds a new CLI command `create-tenbyte-tenant` that creates the tenbyte
tenant with sandbox/production environments and generates an API key.
- Add SSLCommerz case to encryptMetadata() in connection service
- Add migration V6 to sync schema with missing columns:
  - tenants: status, billing_details
  - customers: environment_id, metadata, address fields
Previously, UpdateConnection only handled QuickBooks encrypted data
updates. This change adds support for SSLCommerz, Stripe, Razorpay,
and Nomod encrypted data updates as well.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Shah Ahmed Saad Rupai added 30 commits February 16, 2026 02:41
- Add SigNoz observability stack (zookeeper, clickhouse, schema-migrator, collector)
- Add SSLCommerz flat metadata conversion in connection DTO
- Fix V4 migration with conditional column check
- Add V7 migration for RBAC columns (type, roles)
- Add organization.id and event.name attributes to HTTP event endpoint
- Add SigNoz tracing to Kafka event consumption
- Add helper functions for span attributes (OrganizationID, EventName, etc.)
- Enables filtering/grouping events by organization in SigNoz dashboard
Adds event.property.* attributes to traces so properties like bandwidth_mb,
storage_mb can be viewed and filtered in SigNoz dashboards.
- Systemd service files for API, consumer, worker, temporal, redpanda, frontend
- Nginx configs for api-billing.tenbyte.io, billing.tenbyte.io, monitoring.tenbyte.io
- Install script with PostgreSQL, ClickHouse, Redpanda, Temporal setup
- Redeploy and status scripts for service management
- Backup/restore scripts with nightly cron setup
- SigNoz docker-compose for monitoring alongside bare metal services
- Add DEPLOYMENT.md with full bare metal deployment summary
- Add config.production.yaml.example as template for production configs
- Documents migration from Docker to systemd services
- Includes service configs, troubleshooting, and quick reference commands
- api.config.yaml.example: API server config (port 8080)
- consumer.config.yaml.example: Event consumer config (port 8081)
- worker.config.yaml.example: Temporal worker config (port 8082)

Each service needs its own config directory with different deployment mode.
Consumer config includes note about rate_limit > 0 requirement to prevent panic.
- Add quick access commands (copy-paste ready)
- Add credentials & access section
- Add ASCII architecture diagram
- Add detailed directory structure (/opt/)
- Add deployment scripts documentation (status.sh, redeploy.sh, backup.sh, restore.sh)
- Add troubleshooting section with common issues
- Add SigNoz bare metal deployment plan (using existing ClickHouse)
- Add emergency contacts placeholder
- Add useful commands cheatsheet
- Add change log

This document enables instant project handover.
- Update architecture diagram with correct ports
- FlexPrice API moved to port 8000 (SigNoz uses 8080)
- Add SigNoz monitoring deployment details
- Add ZooKeeper service documentation
- Update services & ports table
- Add SigNoz quick commands and troubleshooting
- Update change log
Add CLAUDE.md project instructions and Tenbyte billing integration docs.
Fix Redpanda OOM crash loop (MemoryMax 2G->3G), add topic retention policies,
update install script with proper Redpanda config, and sync Stripe customer
metadata on existing customers.
Document cancel subscription flow (contact support, no API call),
billing address country lock, plans grid responsive layout,
payment history sorting, and invoice PDF download behavior.
Webhook handler now tries tenant_id/env_id key first, then falls
back to tenant_id only. This allows different webhook endpoints per
environment (e.g. production env → api.tenbyte.io, sandbox env →
api-billing.tenbyte.io).
…dition

Add entity integration mapping check before creating a customer from
Stripe webhook. This prevents duplicates when customer.created webhook
arrives before flexprice_customer_id metadata is set on the Stripe customer.

Also add reset_stripe_customers.sh script to clear Stripe mappings
per-environment when switching Stripe test keys.
- Fix port references (8000 not 8080) across deployment docs
- Update deployment checklist and config examples
- Remove CLAUDE.md from tracking (contains sensitive config)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants