[Part 3|*] Implement Second Part for SQL Backend functions #10291

ziggie1984 · 2025-10-13T19:12:26Z

Depends on #10287

This PR implements most of the Insert and Delete methods we need for the unit tests in the following PR.

ziggie1984 · 2025-10-17T07:54:25Z

/gemini review

gemini-code-assist

Code Review

This pull request implements a significant portion of the insert and delete methods for the new SQL backend for the payments database. The changes are extensive and include new SQL queries, data conversion functions, and the implementation of several methods on the SQLStore.

The use of ExecuteBatchQuery and ExecutePaginatedQuery is great for performance. The new PaymentAndIntent interface is a good abstraction.

I've found a few areas for improvement:

Adherence to the function commenting style guide for exported functions.
Consistent use of context.Context propagation instead of context.TODO().
Opportunities for performance improvements by using batch inserts and optimizing queries to avoid extra fetches.
A potential code smell related to handling of nil slices from the database.

Overall, this is a solid step forward in the SQL backend implementation. Addressing these points will improve the code's maintainability, robustness, and performance.

gemini-code-assist · 2025-10-17T07:56:20Z

payments/db/sql_store.go

+func (s *SQLStore) FetchPayment(paymentHash lntypes.Hash) (*MPPayment, error) {
+	ctx := context.TODO()


This function should accept a context.Context from its caller instead of using context.TODO(). This allows for proper context propagation, which is crucial for handling cancellations and deadlines. This change will require updating the DB interface as well. Several other methods in this file (DeletePayment, DeleteFailedAttempts, InitPayment, RegisterAttempt, SettleAttempt) have the same issue and should also be updated.

Suggested change

func (s *SQLStore) FetchPayment(paymentHash lntypes.Hash) (*MPPayment, error) {

ctx := context.TODO()

func (s *SQLStore) FetchPayment(ctx context.Context, paymentHash lntypes.Hash) (*MPPayment, error) {

gemini-code-assist · 2025-10-17T07:56:20Z

payments/db/sql_store.go

+				// TODO(ziggie): Can we get rid of this?
+				// This has to be in place otherwise the
+				// comparison will not match.
+				if record.Value == nil {
+					record.Value = []byte{}
+				}


The TODO here highlights a potential code smell. This workaround of converting a nil slice to an empty slice might be hiding an issue or could be simplified. It would be beneficial to investigate whether the downstream code that consumes these records can be made to handle nil slices gracefully. This would remove the need for this conversion and make the data handling more consistent.

gemini-code-assist · 2025-10-17T07:56:20Z

payments/db/sql_store.go

+// QueryPayments queries the payments from the database.
+//
+// This is part of the DB interface.
+func (s *SQLStore) QueryPayments(ctx context.Context,


The comment for this exported function does not fully adhere to the repository's style guide.¹ The style guide requires that function comments begin with the function name and provide detailed information for the caller. Please update this comment to start with QueryPayments and add more details about its behavior, parameters, and return values. This applies to other new exported functions in this file as well, such as FetchPayment, DeletePayment, etc.

Style Guide References

Footnotes

Exported functions should have detailed comments that start with the function name, as per the style guide. (link) ↩

gemini-code-assist · 2025-10-17T07:56:20Z

payments/db/sql_store.go

+		// Get the payment ID from the inserted payment
+		// We need to fetch the payment we just inserted to get its ID
+		insertedPayment, err := db.FetchPayment(ctx, paymentHash[:])
+		if err != nil {
+			return fmt.Errorf("failed to fetch inserted payment: %w", err)
+		}


To improve efficiency, this extra FetchPayment call can be avoided. The InsertPayment SQL query can be modified to return the ID of the newly inserted row. You can achieve this by changing the query in sqldb/sqlc/queries/payments.sql from :exec to :one and adding RETURNING id. The Go code would then be simplified to use the returned ID directly.

gemini-code-assist · 2025-10-17T07:56:20Z

payments/db/sql_store.go

+func (s *SQLStore) insertRouteHops(ctx context.Context, db SQLQueries,
+	hops []*route.Hop, attemptID uint64) error {


This function inserts route hops and their associated data one by one in a loop. For routes with many hops, this can result in a large number of database queries, impacting performance. Consider refactoring this to use batch inserts. sqlc supports batch inserts, which would require modifying the SQL queries to accept slices of parameters and then calling them once with all the data for the hops.

In this commit we add all queries which we will need to insert payment related data into the db.

lightninglabs-deploy · 2025-10-24T08:06:40Z

@ziggie1984, remember to re-request review from reviewers when ready

ellemouton · 2025-10-21T10:50:26Z

payments/db/sql_store.go

+	}, func() {
+	})


ellemouton · 2025-10-24T08:18:00Z

payments/db/sql_store.go

+		completePayment, err := s.fetchPaymentWithCompleteData(
+			ctx, db, fetchPayment,
+		)
+		if err != nil {


i dont think we need to fetch the complete payment here?

ellemouton · 2025-10-24T08:18:58Z

payments/db/sql_store.go

+		if failedHtlcsOnly {
+			return s.db.DeleteFailedAttempts(
+				ctx, fetchPayment.Payment.ID,
+			)
+		}


seems like the next commit should be before this one?

ellemouton · 2025-10-24T08:22:51Z

payments/db/sql_store.go

+			return db.DeleteFailedAttempts(
 				ctx, fetchPayment.Payment.ID,
 			)
 		}

+		// Be careful to not use s.db here, because we are in a
+		// transaction, is there a way to make this more secure?
 		return db.DeletePayment(ctx, fetchPayment.Payment.ID)


these changes should be in the prior commit (was confusing to have it here since it seemed like it might be related to the change below but it is not)

ellemouton · 2025-10-24T08:25:05Z

payments/db/sql_store.go

+	}, func() {
+	})


sqldb.NoOpReset

ellemouton · 2025-10-24T08:38:20Z

payments/db/sql_store.go

+		}
+
+		// If payment exists and is failed, delete it first.
+		if existingPayment.Payment.ID != 0 {


i find this part quite hard to read. I suggest using a switch with 3 different cases.
Cause here, you are referencing the returned existingPayment in the case that the error is not nil which is an antipattern.

this will only be !=0 in the case where err == nil - so just put it in that block

so like:

existingPayment, err := db.FetchPayment(ctx, paymentHash[:]) switch { case err == nil: completePayment, err := s.fetchPaymentWithCompleteData( ctx, db, existingPayment, ) if err != nil { return fmt.Errorf("failed to fetch payment "+ "with complete data: %w", err) } // Check if the payment is initializable otherwise // we'll return early. err = completePayment.Status.initializable() if err != nil { return err } // If the initializable check above passes, then the // existing payment has failed. So we delete it and // all of its previous artifacts. We rely on // cascading deletes to clean up the rest. err = db.DeletePayment(ctx, existingPayment.Payment.ID) if err != nil { return fmt.Errorf("failed to delete "+ "payment: %w", err) } case !errors.Is(err, sql.ErrNoRows): // Some other error occurred return fmt.Errorf("failed to check existing "+ "payment: %w", err) // The payment does not yet exist. default: }

ellemouton · 2025-10-24T08:43:58Z

payments/db/sql_store.go

+		// Only set the intent ID if it's not nil.
+		var intentIDParam sql.NullInt64
+		if intentID != nil {
+			intentIDParam = sqldb.SQLInt64(*intentID)
+		}


i think rather then dont use the sqldb.SQLInt64 helper

var intentID int64 if len(paymentCreationInfo.PaymentRequest) > 0 { intentID, err = db.InsertPaymentIntent(ctx, sqlc.InsertPaymentIntentParams{ IntentType: int16( PaymentIntentTypeBolt11, ), IntentPayload: paymentCreationInfo. PaymentRequest, }) if err != nil { return fmt.Errorf("failed to initialize "+ "payment intent: %w", err) } } paymentID, err := db.InsertPayment(ctx, sqlc.InsertPaymentParams{ IntentID: sql.NullInt64{ Int64: intentID, Valid: intentID != 0, },

ellemouton · 2025-10-24T08:45:41Z

payments/db/sql_store.go

+	var mpPayment *MPPayment
+
+	err := s.db.ExecTx(ctx, sqldb.WriteTxOpt(), func(db SQLQueries) error {
+		// 1. First Fetch the payment and check if it is registrable.


nit: started with numbering and then stopped

ellemouton · 2025-10-24T08:47:32Z

payments/db/sql_store.go

+		// Add the attempt to the payment without fetching it from the
+		// DB again.


ellemouton · 2025-10-24T08:51:25Z

payments/db/sql_store.go

+		payment, err := db.FetchPayment(ctx, paymentHash[:])
+		if err != nil && !errors.Is(err, sql.ErrNoRows) {
+			return fmt.Errorf("failed to fetch payment: %w", err)
+		}
+		if errors.Is(err, sql.ErrNoRows) {
+			return ErrPaymentNotInitiated
+		}
+


why dont we check Status.updatable() as is done for KVStore?

ziggie1984 changed the base branch from master to elle-payment-sql-series-new October 13, 2025 19:13

ziggie1984 self-assigned this Oct 13, 2025

ziggie1984 added payments Related to invoices/payments sql labels Oct 13, 2025

ziggie1984 added this to v0.21 Oct 13, 2025

ziggie1984 added this to the v0.21.0 milestone Oct 13, 2025

saubyk moved this to In progress in v0.21 Oct 14, 2025

sqldb: add index and comment to payment tables

18f03ec

ziggie1984 changed the title ~~[Part 3|*] Implement FetchPayment for SQL Backend~~ [Part 3|*] Implement Second Part for SQL Backend functions Oct 15, 2025

ziggie1984 force-pushed the introduce-sql-schema-payments-part-3 branch 2 times, most recently from 80de0dd to c0134ed Compare October 15, 2025 15:12

ziggie1984 marked this pull request as ready for review October 15, 2025 15:13

ziggie1984 mentioned this pull request Oct 15, 2025

[Part 4|*] Add unit tests for the previous introduced SQL Backend methods #10292

Open

ziggie1984 force-pushed the introduce-sql-schema-payments-part-3 branch 3 times, most recently from f0e3302 to 99ddf07 Compare October 17, 2025 07:53

gemini-code-assist bot reviewed Oct 17, 2025

View reviewed changes

ziggie1984 added 12 commits October 17, 2025 10:20

multi: add relevant queries for QueryPayments implemenation

c908b52

paymentsdb: add new internal error

52d466f

paymentsdb: implement QueryPayments for sql backend

353160c

paymentsdb: implement FetchPayment for sql backend

fc4ce67

docs: add release-notes

0849296

sqldb: add queries for deleting a payment and attempts

ed5622c

paymentsdb: implement DeletePayment for sql backend

215490c

paymentsdb: implement DeleteFailedAttempts for sql backend

3a39bdc

sqldb+paymentsdb: add queries to insert all relavant data

1aae195

In this commit we add all queries which we will need to insert payment related data into the db.

paymentsdb: implement InitPayment for sql backend

f6f2ee3

paymentsdb: implement RegisterAttempt for sql backend

53b47ea

paymentsdb: implement SettleAttempt for sql backend

b3f3627

docs: add release-notes

6145682

ziggie1984 force-pushed the introduce-sql-schema-payments-part-3 branch from 99ddf07 to 6145682 Compare October 17, 2025 08:43

ziggie1984 requested review from ellemouton and yyforyongyu October 17, 2025 08:50

ziggie1984 mentioned this pull request Oct 17, 2025

[feature]: Introduce Payment SQL storage #9863

Open

6 tasks

ziggie1984 moved this from In progress to In review in v0.21 Oct 18, 2025

ellemouton reviewed Oct 24, 2025

View reviewed changes

		func (s SQLStore) FetchPayment(paymentHash lntypes.Hash) (MPPayment, error) {
		ctx := context.TODO()

		func (s *SQLStore) insertRouteHops(ctx context.Context, db SQLQueries,
		hops []*route.Hop, attemptID uint64) error {

		// Add the attempt to the payment without fetching it from the
		// DB again.

[Part 3|*] Implement Second Part for SQL Backend functions #10291

Are you sure you want to change the base?

[Part 3|*] Implement Second Part for SQL Backend functions #10291

Uh oh!

Conversation

ziggie1984 commented Oct 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ziggie1984 commented Oct 17, 2025

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist bot Oct 17, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Oct 17, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Oct 17, 2025

Choose a reason for hiding this comment

Style Guide References

Footnotes

Uh oh!

gemini-code-assist bot Oct 17, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Oct 17, 2025

Choose a reason for hiding this comment

Uh oh!

lightninglabs-deploy commented Oct 24, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ziggie1984 commented Oct 13, 2025 •

edited

Loading