Skip to content

Conversation

@neuroevolutus
Copy link
Contributor

@neuroevolutus neuroevolutus commented Feb 25, 2025

I added a function called make_source_annotated_token to simplify the repeated logic in the source extent tests, but if it's unnecessary I can easily revert those commits.

Edit: Linking the PR for easy, future reference: #241.

Copy link
Owner

@tekknolagi tekknolagi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm pending fixes

scrapscript.py Outdated
result.source_extent.end.byteno = self.current_token_source_extent.end.byteno

return result
return make_source_annotated_token(cls, copy.deepcopy(self.current_token_source_extent), *args)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think if you use dataclasses, you get a copy for free. Try calling .replace() (no arguments)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the suggestion; actually, it seems that replace is a free-standing function defined in module dataclasses, so I had to do replace(self.current_token_source_extent). However, this failed the tests since it did not seem to do a deep copy of the dataclass fields.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, riparoni

int_ast = make_source_annotated_object(Int, source_extent, 1)
self.assertTrue(parse(Peekable(iter([int_lit]))).source_extent == int_ast.source_extent)
int_lit = make_source_annotated_token(IntLit, source_extent, 1)
self.assertTrue(parse(Peekable(iter([int_lit]))).source_extent == source_extent)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

assertEqual, please (here and everywhere)

@tekknolagi
Copy link
Owner

You may also find it useful to convert make_source_annotated_token a method on Token called with_source or something that returns a new token with that attribute added

Copy link
Owner

@tekknolagi tekknolagi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sweet, thanks

@tekknolagi tekknolagi merged commit 84a2bf6 into tekknolagi:trunk Feb 26, 2025
53 of 55 checks passed
@neuroevolutus neuroevolutus deleted the source-tracking-fixes branch February 26, 2025 21:18
@tekknolagi tekknolagi deployed to scrapscript-pr-244 February 26, 2025 21:19 — with GitHub Actions Active
@tekknolagi tekknolagi linked an issue Feb 26, 2025 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Maintain line and character number after parsing

2 participants