Skip to content

Bulk insert of failed receipts using query macro #328

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 5 commits into from
Closed

Bulk insert of failed receipts using query macro #328

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
9315908
Bulk insert of failed receipts
taslimmuhammed Sep 24, 2024
5bcbcd8
query macro
taslimmuhammed Sep 24, 2024
f98aaf3
sqlx bulk insert after running compiling codes
taslimmuhammed Sep 27, 2024
bcaae0d
sqlx bulk insert after running compiling codes
taslimmuhammed Sep 27, 2024
fc29b3a
added error logs
taslimmuhammed Sep 28, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions .sqlx/query-0d4e055d87b4496202041f3c28618688afa91dc0b8eeb8e551b271fc73c1157e.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 0 additions & 6 deletions .sqlx/query-56c3678866ffe0ec2eed7290394d07007990cc244f598b763ec5470515efe019.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

20 changes: 0 additions & 20 deletions .sqlx/query-6c365bc1b0728ec8d9f1239d7bffe7861ff17c6b8de4365a39f28099f5711613.json
View file
Open in desktop

This file was deleted.

4 changes: 2 additions & 2 deletions migrations/20230915230734_tap_ravs.down.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
DROP TABLE IF EXISTS scalar_tap_ravs CASCADE;
DROP TABLE IF EXISTS scalar_tap_rav_requests_failed CASCADE;
-- DROP TABLE IF EXISTS scalar_tap_ravs CASCADE;
-- DROP TABLE IF EXISTS scalar_tap_rav_requests_failed CASCADE;
68 changes: 44 additions & 24 deletions tap-agent/src/agent/sender_allocation.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -685,6 +685,14 @@ impl SenderAllocationState {
&mut self,
receipts: &[ReceiptWithState<Failed>],
) -> Result<()> {
let reciepts_len = receipts.len();
let mut reciepts_signers = Vec::with_capacity(reciepts_len);
let mut encoded_signatures = Vec::with_capacity(reciepts_len);
let mut allocation_ids = Vec::with_capacity(reciepts_len);
let mut timestamps = Vec::with_capacity(reciepts_len);
let mut nounces = Vec::with_capacity(reciepts_len);
let mut values = Vec::with_capacity(reciepts_len);

for received_receipt in receipts.iter() {
let receipt = received_receipt.signed_receipt();
let allocation_id = receipt.message.allocation_id;
Expand All @@ -696,31 +704,43 @@ impl SenderAllocationState {
error!("Failed to recover receipt signer: {}", e);
anyhow!(e)
})?;
sqlx::query!(
r#"
INSERT INTO scalar_tap_receipts_invalid (
signer_address,
signature,
allocation_id,
timestamp_ns,
nonce,
value,
error_log
)
VALUES ($1, $2, $3, $4, $5, $6, $7)
"#,
receipt_signer.encode_hex(),
encoded_signature,
allocation_id.encode_hex(),
BigDecimal::from(receipt.message.timestamp_ns),
BigDecimal::from(receipt.message.nonce),
BigDecimal::from(BigInt::from(receipt.message.value)),
receipt_error
)
.execute(&self.pgpool)
.await
.map_err(|e| anyhow!("Failed to store invalid receipt: {:?}", e))?;
reciepts_signers.push(receipt_signer.encode_hex());
encoded_signatures.push(encoded_signature);
allocation_ids.push(allocation_id.encode_hex());
timestamps.push(BigDecimal::from(receipt.message.timestamp_ns));
nounces.push(BigDecimal::from(receipt.message.nonce));
values.push(BigDecimal::from(BigInt::from(receipt.message.value)));
}
sqlx::query!(
r#"INSERT INTO scalar_tap_receipts_invalid (
signer_address,
signature,
allocation_id,
timestamp_ns,
nonce,
value
Copy link
Contributor

@gusinacio gusinacio Sep 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's missing here error_log

Copy link
Contributor Author

@taslimmuhammed taslimmuhammed Sep 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did you meant this line
debug!(
"Receipt for allocation {} and signer {} failed reason: {}",
allocation_id.encode_hex(),
receipt_signer.encode_hex(),
receipt_error
);

Copy link
Contributor

@gusinacio gusinacio Sep 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We want to store the error logs for all receipts as well.

Take a look at the older version, we were inserting error_log in the old sql.

) SELECT * FROM UNNEST(
$1::CHAR(40)[],
$2::BYTEA[],
$3::CHAR(40)[],
$4::NUMERIC(20)[],
$5::NUMERIC(20)[],
$6::NUMERIC(40)[]
)"#,
&reciepts_signers,
&encoded_signatures,
&allocation_ids,
&timestamps,
&nounces,
&values,
)
.execute(&self.pgpool)
.await
.map_err(|e| {
error!("Failed to store receipt: {}", e);
anyhow!(e)
})?;

let fees = receipts
.iter()
.map(|receipt| receipt.signed_receipt().message.value)
Expand Down
Loading