Skip to content

Commit 331b458

Browse files
benesjanbenesjan
authored
refactor!: dropping pagination from getLogsByTags (#19161)
In this PR I drop pagination from `getLogsByTags` endpoint because pagination here doesn't really make sense because we get more logs per tag only if there are multiple devices sending logs from a given sender to a recipient. This is the final of 3 PRs in which I clean up the `getLogsByTags` endpoint: 1. First PR - including block timestamp in return value, 2. This is the second PR - I type the tag arg of the function to be `SiloedTag`, 3. in the last PR I will drop pagination from this endpoint. # Potential DoS vector Log tag is unconstrained and hence can be arbitrarily chosen by the sender. This introduces a potential DoS vector because the node currently loads all the logs into memory and returns them. Hence it could be feasible to re-use the same tag thousands of time and then spam a node by request for this tag. I think the solution here is to simply define the maximum number of logs a node is willing to store per tag in the db. Given that there is no legitimate use case for log reuse I think making it even as low as 5 would be fine. Note that this PR didn't introduce this DoS vector because even though I dropped the pagination here it didn't seem to be enforced before (if `limitPerTag` arg was undefined we just returned it all). For this reason I think we can merge it and tackle the issue in a followup PR.
2 parents 318e3f3 + 7afc547 commit 331b458

File tree

8 files changed

+52
-130
lines changed

8 files changed

+52
-130
lines changed

‎yarn-project/archiver/src/archiver/archiver.ts‎

Lines changed: 8 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1415,16 +1415,12 @@ export class Archiver
14151415
return this.store.getSettledTxReceipt(txHash);
14161416
}
14171417

1418-
getPrivateLogsByTags(tags: SiloedTag[], logsPerTag?: number): Promise<TxScopedL2Log[][]> {
1419-
return this.store.getPrivateLogsByTags(tags, logsPerTag);
1418+
getPrivateLogsByTags(tags: SiloedTag[]): Promise<TxScopedL2Log[][]> {
1419+
return this.store.getPrivateLogsByTags(tags);
14201420
}
14211421

1422-
getPublicLogsByTagsFromContract(
1423-
contractAddress: AztecAddress,
1424-
tags: Tag[],
1425-
logsPerTag?: number,
1426-
): Promise<TxScopedL2Log[][]> {
1427-
return this.store.getPublicLogsByTagsFromContract(contractAddress, tags, logsPerTag);
1422+
getPublicLogsByTagsFromContract(contractAddress: AztecAddress, tags: Tag[]): Promise<TxScopedL2Log[][]> {
1423+
return this.store.getPublicLogsByTagsFromContract(contractAddress, tags);
14281424
}
14291425

14301426
/**
@@ -2082,15 +2078,11 @@ export class ArchiverStoreHelper
20822078
getL1ToL2MessageIndex(l1ToL2Message: Fr): Promise<bigint | undefined> {
20832079
return this.store.getL1ToL2MessageIndex(l1ToL2Message);
20842080
}
2085-
getPrivateLogsByTags(tags: SiloedTag[], logsPerTag?: number): Promise<TxScopedL2Log[][]> {
2086-
return this.store.getPrivateLogsByTags(tags, logsPerTag);
2081+
getPrivateLogsByTags(tags: SiloedTag[]): Promise<TxScopedL2Log[][]> {
2082+
return this.store.getPrivateLogsByTags(tags);
20872083
}
2088-
getPublicLogsByTagsFromContract(
2089-
contractAddress: AztecAddress,
2090-
tags: Tag[],
2091-
logsPerTag?: number,
2092-
): Promise<TxScopedL2Log[][]> {
2093-
return this.store.getPublicLogsByTagsFromContract(contractAddress, tags, logsPerTag);
2084+
getPublicLogsByTagsFromContract(contractAddress: AztecAddress, tags: Tag[]): Promise<TxScopedL2Log[][]> {
2085+
return this.store.getPublicLogsByTagsFromContract(contractAddress, tags);
20942086
}
20952087
getPublicLogs(filter: LogFilter): Promise<GetPublicLogsResponse> {
20962088
return this.store.getPublicLogs(filter);

‎yarn-project/archiver/src/archiver/archiver_store.ts‎

Lines changed: 11 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -206,27 +206,17 @@ export interface ArchiverDataStore {
206206
getTotalL1ToL2MessageCount(): Promise<bigint>;
207207

208208
/**
209-
* Gets all private logs that match any of the received tags (i.e. logs with their first field equal to a SiloedTag).
210-
* @param tags - The SiloedTags to filter the logs by.
211-
* @param logsPerTag - The number of logs to return per tag. Defaults to everything
212-
* @returns For each received tag, an array of matching private logs is returned. An empty array implies no logs match
213-
* that tag.
214-
*/
215-
getPrivateLogsByTags(tags: SiloedTag[], logsPerTag?: number): Promise<TxScopedL2Log[][]>;
216-
217-
/**
218-
* Gets all public logs that match any of the received tags from the specified contract (i.e. logs with their first field equal to a Tag).
219-
* @param contractAddress - The contract that emitted the public logs.
220-
* @param tags - The Tags to filter the logs by.
221-
* @param logsPerTag - The number of logs to return per tag. Defaults to everything
222-
* @returns For each received tag, an array of matching public logs is returned. An empty array implies no logs match
223-
* that tag.
224-
*/
225-
getPublicLogsByTagsFromContract(
226-
contractAddress: AztecAddress,
227-
tags: Tag[],
228-
logsPerTag?: number,
229-
): Promise<TxScopedL2Log[][]>;
209+
/**
210+
* Gets all private logs that match any of the `tags`. For each tag, an array of matching logs is returned. An empty
211+
* array implies no logs match that tag.
212+
*/
213+
getPrivateLogsByTags(tags: SiloedTag[]): Promise<TxScopedL2Log[][]>;
214+
215+
/**
216+
* Gets all public logs that match any of the `tags` from the specified contract. For each tag, an array of matching
217+
* logs is returned. An empty array implies no logs match that tag.
218+
*/
219+
getPublicLogsByTagsFromContract(contractAddress: AztecAddress, tags: Tag[]): Promise<TxScopedL2Log[][]>;
230220

231221
/**
232222
* Gets public logs based on the provided filter.

‎yarn-project/archiver/src/archiver/kv_archiver_store/kv_archiver_store.ts‎

Lines changed: 4 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -318,21 +318,17 @@ export class KVArchiverDataStore implements ArchiverDataStore, ContractDataSourc
318318
return this.#messageStore.getL1ToL2Messages(checkpointNumber);
319319
}
320320

321-
getPrivateLogsByTags(tags: SiloedTag[], logsPerTag?: number): Promise<TxScopedL2Log[][]> {
321+
getPrivateLogsByTags(tags: SiloedTag[]): Promise<TxScopedL2Log[][]> {
322322
try {
323-
return this.#logStore.getPrivateLogsByTags(tags, logsPerTag);
323+
return this.#logStore.getPrivateLogsByTags(tags);
324324
} catch (err) {
325325
return Promise.reject(err);
326326
}
327327
}
328328

329-
getPublicLogsByTagsFromContract(
330-
contractAddress: AztecAddress,
331-
tags: Tag[],
332-
logsPerTag?: number,
333-
): Promise<TxScopedL2Log[][]> {
329+
getPublicLogsByTagsFromContract(contractAddress: AztecAddress, tags: Tag[]): Promise<TxScopedL2Log[][]> {
334330
try {
335-
return this.#logStore.getPublicLogsByTagsFromContract(contractAddress, tags, logsPerTag);
331+
return this.#logStore.getPublicLogsByTagsFromContract(contractAddress, tags);
336332
} catch (err) {
337333
return Promise.reject(err);
338334
}

‎yarn-project/archiver/src/archiver/kv_archiver_store/log_store.ts‎

Lines changed: 9 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -284,46 +284,27 @@ export class LogStore {
284284
}
285285

286286
/**
287-
* Gets all private logs that match any of the received tags (i.e. logs with their first field equal to a SiloedTag).
288-
* @param tags - The SiloedTags to filter the logs by.
289-
* @param limitPerTag - The maximum number of logs to return per tag.
290-
* @returns For each received tag, an array of matching private logs is returned. An empty array implies no logs match
291-
* that tag.
287+
* Gets all private logs that match any of the `tags`. For each tag, an array of matching logs is returned. An empty
288+
* array implies no logs match that tag.
292289
*/
293-
async getPrivateLogsByTags(tags: SiloedTag[], limitPerTag?: number): Promise<TxScopedL2Log[][]> {
294-
if (limitPerTag !== undefined && limitPerTag <= 0) {
295-
throw new TypeError('limitPerTag needs to be greater than 0');
296-
}
290+
async getPrivateLogsByTags(tags: SiloedTag[]): Promise<TxScopedL2Log[][]> {
297291
const logs = await Promise.all(tags.map(tag => this.#privateLogsByTag.getAsync(tag.toString())));
298-
return logs.map(
299-
logBuffers => logBuffers?.slice(0, limitPerTag).map(logBuffer => TxScopedL2Log.fromBuffer(logBuffer)) ?? [],
300-
);
292+
293+
return logs.map(logBuffers => logBuffers?.map(logBuffer => TxScopedL2Log.fromBuffer(logBuffer)) ?? []);
301294
}
302295

303296
/**
304-
* Gets all public logs that match any of the received tags from the specified contract (i.e. logs with their first field equal to a Tag).
305-
* @param contractAddress - The contract that emitted the public logs.
306-
* @param tags - The Tags to filter the logs by.
307-
* @param limitPerTag - The maximum number of logs to return per tag.
308-
* @returns For each received tag, an array of matching public logs is returned. An empty array implies no logs match that tag.
297+
* Gets all public logs that match any of the `tags` from the specified contract. For each tag, an array of matching
298+
* logs is returned. An empty array implies no logs match that tag.
309299
*/
310-
async getPublicLogsByTagsFromContract(
311-
contractAddress: AztecAddress,
312-
tags: Tag[],
313-
limitPerTag?: number,
314-
): Promise<TxScopedL2Log[][]> {
315-
if (limitPerTag !== undefined && limitPerTag <= 0) {
316-
throw new TypeError('limitPerTag needs to be greater than 0');
317-
}
300+
async getPublicLogsByTagsFromContract(contractAddress: AztecAddress, tags: Tag[]): Promise<TxScopedL2Log[][]> {
318301
const logs = await Promise.all(
319302
tags.map(tag => {
320303
const key = `${contractAddress.toString()}_${tag.value.toString()}`;
321304
return this.#publicLogsByContractAndTag.getAsync(key);
322305
}),
323306
);
324-
return logs.map(
325-
logBuffers => logBuffers?.slice(0, limitPerTag).map(logBuffer => TxScopedL2Log.fromBuffer(logBuffer)) ?? [],
326-
);
307+
return logs.map(logBuffers => logBuffers?.map(logBuffer => TxScopedL2Log.fromBuffer(logBuffer)) ?? []);
327308
}
328309

329310
/**

‎yarn-project/aztec-node/src/aztec-node/server.ts‎

Lines changed: 4 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -695,16 +695,12 @@ export class AztecNodeService implements AztecNode, AztecNodeAdmin, Traceable {
695695
return this.contractDataSource.getContract(address);
696696
}
697697

698-
public getPrivateLogsByTags(tags: SiloedTag[], logsPerTag?: number): Promise<TxScopedL2Log[][]> {
699-
return this.logsSource.getPrivateLogsByTags(tags, logsPerTag);
698+
public getPrivateLogsByTags(tags: SiloedTag[]): Promise<TxScopedL2Log[][]> {
699+
return this.logsSource.getPrivateLogsByTags(tags);
700700
}
701701

702-
public getPublicLogsByTagsFromContract(
703-
contractAddress: AztecAddress,
704-
tags: Tag[],
705-
logsPerTag?: number,
706-
): Promise<TxScopedL2Log[][]> {
707-
return this.logsSource.getPublicLogsByTagsFromContract(contractAddress, tags, logsPerTag);
702+
public getPublicLogsByTagsFromContract(contractAddress: AztecAddress, tags: Tag[]): Promise<TxScopedL2Log[][]> {
703+
return this.logsSource.getPublicLogsByTagsFromContract(contractAddress, tags);
708704
}
709705

710706
/**

‎yarn-project/stdlib/src/interfaces/archiver.ts‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -115,11 +115,11 @@ export const ArchiverApiSchema: ApiSchemaFor<ArchiverApi> = {
115115
getL2Tips: z.function().args().returns(L2TipsSchema),
116116
getPrivateLogsByTags: z
117117
.function()
118-
.args(z.array(SiloedTag.schema), optional(schemas.Integer))
118+
.args(z.array(SiloedTag.schema))
119119
.returns(z.array(z.array(TxScopedL2Log.schema))),
120120
getPublicLogsByTagsFromContract: z
121121
.function()
122-
.args(schemas.AztecAddress, z.array(Tag.schema), optional(schemas.Integer))
122+
.args(schemas.AztecAddress, z.array(Tag.schema))
123123
.returns(z.array(z.array(TxScopedL2Log.schema))),
124124
getPublicLogs: z.function().args(LogFilterSchema).returns(GetPublicLogsResponseSchema),
125125
getContractClassLogs: z.function().args(LogFilterSchema).returns(GetContractClassLogsResponseSchema),

‎yarn-project/stdlib/src/interfaces/aztec-node.ts‎

Lines changed: 8 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -338,29 +338,16 @@ export interface AztecNode
338338
getContractClassLogs(filter: LogFilter): Promise<GetContractClassLogsResponse>;
339339

340340
/**
341-
* Gets all private logs that match any of the received tags (i.e. logs with their first field equal to a SiloedTag).
342-
* @param tags - The SiloedTags to filter the logs by.
343-
* @param logsPerTag - How many logs to return per tag. Default 10 logs are returned for each tag
344-
* @returns For each received tag, an array of matching private logs and metadata (e.g. tx hash) is returned. An empty
345-
* array implies no logs match that tag. There can be multiple logs for 1 tag because tag reuse can happen
346-
* --> e.g. when sending a note from multiple unsynched devices.
341+
* Gets all private logs that match any of the `tags`. For each tag, an array of matching logs is returned. An empty
342+
* array implies no logs match that tag.
347343
*/
348-
getPrivateLogsByTags(tags: SiloedTag[], logsPerTag?: number): Promise<TxScopedL2Log[][]>;
344+
getPrivateLogsByTags(tags: SiloedTag[]): Promise<TxScopedL2Log[][]>;
349345

350346
/**
351-
* Gets all public logs that match any of the received tags from the specified contract (i.e. logs with their first field equal to a Tag).
352-
* @param contractAddress - The contract that emitted the public logs.
353-
* @param tags - The Tags to filter the logs by.
354-
* @param logsPerTag - How many logs to return per tag. Default 10 logs are returned for each tag
355-
* @returns For each received tag, an array of matching public logs and metadata (e.g. tx hash) is returned. An empty
356-
* array implies no logs match that tag. There can be multiple logs for 1 tag because tag reuse can happen
357-
* --> e.g. when sending a note from multiple unsynched devices.
347+
* Gets all public logs that match any of the `tags` from the specified contract. For each tag, an array of matching
348+
* logs is returned. An empty array implies no logs match that tag.
358349
*/
359-
getPublicLogsByTagsFromContract(
360-
contractAddress: AztecAddress,
361-
tags: Tag[],
362-
logsPerTag?: number,
363-
): Promise<TxScopedL2Log[][]>;
350+
getPublicLogsByTagsFromContract(contractAddress: AztecAddress, tags: Tag[]): Promise<TxScopedL2Log[][]>;
364351

365352
/**
366353
* Method to submit a transaction to the p2p pool.
@@ -496,7 +483,6 @@ export interface AztecNode
496483
getAllowedPublicSetup(): Promise<AllowedElement[]>;
497484
}
498485

499-
export const MAX_LOGS_PER_TAG = 10;
500486
const MAX_SIGNATURES_PER_REGISTER_CALL = 100;
501487
const MAX_SIGNATURE_LEN = 10000;
502488

@@ -618,19 +604,12 @@ export const AztecNodeApiSchema: ApiSchemaFor<AztecNode> = {
618604

619605
getPrivateLogsByTags: z
620606
.function()
621-
.args(
622-
z.array(SiloedTag.schema).max(MAX_RPC_LEN),
623-
optional(z.number().gte(1).lte(MAX_LOGS_PER_TAG).default(MAX_LOGS_PER_TAG)),
624-
)
607+
.args(z.array(SiloedTag.schema).max(MAX_RPC_LEN))
625608
.returns(z.array(z.array(TxScopedL2Log.schema))),
626609

627610
getPublicLogsByTagsFromContract: z
628611
.function()
629-
.args(
630-
schemas.AztecAddress,
631-
z.array(Tag.schema).max(MAX_RPC_LEN),
632-
optional(z.number().gte(1).lte(MAX_LOGS_PER_TAG).default(MAX_LOGS_PER_TAG)),
633-
)
612+
.args(schemas.AztecAddress, z.array(Tag.schema).max(MAX_RPC_LEN))
634613
.returns(z.array(z.array(TxScopedL2Log.schema))),
635614

636615
sendTx: z.function().args(Tx.schema).returns(z.void()),

‎yarn-project/stdlib/src/interfaces/l2_logs_source.ts‎

Lines changed: 6 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -12,28 +12,16 @@ import type { GetContractClassLogsResponse, GetPublicLogsResponse } from './get_
1212
*/
1313
export interface L2LogsSource {
1414
/**
15-
* Gets all private logs that match any of the received tags (i.e. logs with their first field equal to a SiloedTag).
16-
* @param tags - The SiloedTags to filter the logs by.
17-
* @param logsPerTag - The maximum number of logs to return for each tag. Default returns everything
18-
* @returns For each received tag, an array of matching private logs is returned. An empty array implies no logs match
19-
* that tag.
15+
* Gets all private logs that match any of the `tags`. For each tag, an array of matching logs is returned. An empty
16+
* array implies no logs match that tag.
2017
*/
21-
getPrivateLogsByTags(tags: SiloedTag[], logsPerTag?: number): Promise<TxScopedL2Log[][]>;
18+
getPrivateLogsByTags(tags: SiloedTag[]): Promise<TxScopedL2Log[][]>;
2219

2320
/**
24-
* Gets all public logs that match any of the received tags from the specified contract (i.e. logs with their first
25-
* field equal to a Tag).
26-
* @param contractAddress - The contract that emitted the public logs.
27-
* @param tags - The Tags to filter the logs by.
28-
* @param logsPerTag - The maximum number of logs to return for each tag. Default returns everything
29-
* @returns For each received tag, an array of matching public logs is returned. An empty array implies no logs match
30-
* that tag.
21+
* Gets all public logs that match any of the `tags` from the specified contract. For each tag, an array of matching
22+
* logs is returned. An empty array implies no logs match that tag.
3123
*/
32-
getPublicLogsByTagsFromContract(
33-
contractAddress: AztecAddress,
34-
tags: Tag[],
35-
logsPerTag?: number,
36-
): Promise<TxScopedL2Log[][]>;
24+
getPublicLogsByTagsFromContract(contractAddress: AztecAddress, tags: Tag[]): Promise<TxScopedL2Log[][]>;
3725

3826
/**
3927
* Gets public logs based on the provided filter.

0 commit comments

Comments
 (0)