Skip to content

Commit af46a37

Browse files
authored
fix: Unify separators in unique_key construction (#1569)
### Description - Use only pipes (`|`) as separators when constructing `unique_key` values. - Since the changes affect only the `unique_key` of the `always_enqueue` requests, it is not a breaking change. - The random string component is placed at the beginning, so an `always_enqueue` unique key looks like this: ``` unique_key='7VnDVJgE3TrjbDl86|GET|e3b0c442|e3b0c442|https://crawlee.dev/' ``` This format is slightly better for debugging, as the individual parts of the unique key (including the URL) are easier to identify. ### Issues - Closes: #1512 ### Testing - Current tests pass.
1 parent c8345ae commit af46a37

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

src/crawlee/_request.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -299,7 +299,7 @@ def from_url(
299299
)
300300

301301
if always_enqueue:
302-
unique_key = f'{unique_key}_{crypto_random_object_id()}'
302+
unique_key = f'{crypto_random_object_id()}|{unique_key}'
303303

304304
request = cls(
305305
url=url,

0 commit comments

Comments
 (0)