Skip to content

update JuliaSyntax to v1 #121

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ericphanson merged 5 commits into from
Nov 10, 2025
Merged

update JuliaSyntax to v1 #121

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
17e0304
update JuliaSyntax to v1
ericphanson Nov 10, 2025
71ab1e0
fix bug and add test
ericphanson Nov 10, 2025
e5d9323
fix random typos
ericphanson Nov 10, 2025
f8fe3a2
fix test setup
ericphanson Nov 10, 2025
91e12a4
fix deprecation
ericphanson Nov 10, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions Project.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
name = "PackageAnalyzer"
uuid = "e713c705-17e4-4cec-abe0-95bf5bf3e10c"
authors = ["Mosè Giordano <mose@gnu.org>"]
version = "3.1.0"
version = "3.2.0"

[deps]
AbstractTrees = "1520ce14-60c1-5f80-bbc7-55ef81b5835c"
Expand All @@ -28,7 +28,7 @@ Git = "1.2.1"
GitHub = "5.4"
Legolas = "0.5"
JSON3 = "1.5.1"
JuliaSyntax = "0.4"
JuliaSyntax = "1"
LicenseCheck = "0.2"
RegistryInstances = "0.1"
julia = "1.10"
Expand Down
2 changes: 2 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,8 @@ Package Flux:

See the [docs](https://JuliaEcosystem.github.io/PackageAnalyzer.jl/dev/) for more!

One special feature of PackageAnalyzer is a custom JuliaSyntax-based line-of-code counting implementation which correctly distinguishes between Julia docstrings, comments, and code.

## Talks and blog posts

Check out our JuliaCon 2023 talk:
Expand Down
8 changes: 4 additions & 4 deletions src/LineCategories.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
module CategorizeLines
export LineCategories, LineCategory, Blank, Code, Docstring, Comment, categorize_lines!

using JuliaSyntax: haschildren, children, SourceFile, kind, @K_str, source_line
using JuliaSyntax: JuliaSyntax, children, SourceFile, kind, @K_str, source_line

# Every line will have a single category. This way the total number across all categories
# equals the total number of lines. This is useful for debugging and is reassuring to users.
Expand Down Expand Up @@ -32,7 +32,7 @@ Currently:
@enum LineCategory Blank Code Comment Docstring

# We will store the categories assigned to each line in a file with the following structure.
# This keeps the `SourceFile` to facillitate printing.
# This keeps the `SourceFile` to facilitate printing.
"""
LineCategories(path)

Expand Down Expand Up @@ -84,11 +84,11 @@ function categorize_lines!(d::LineCategories, node, source, nesting=0, pos=1, pa
k = kind(node)

# Recurse over children
is_leaf = !haschildren(node)
is_leaf = JuliaSyntax.is_leaf(node)
if !is_leaf
new_nesting = nesting + 1
p = pos
for x in children(node)
for x in something(children(node), ())
categorize_lines!(d, x, source, new_nesting, p, k)
p += x.span
end
Expand Down
10 changes: 5 additions & 5 deletions src/analyze.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,8 @@ PackageV1 Pluto:
* tree hash: db1306745717d127037c5697436b04cfb9d7b3dd
* Julia code in `src`: 8337 lines
* Julia code in `test`: 5448 lines (39.5% of `test` + `src`)
* documention in `docs`: 0 lines (0.0% of `docs` + `src`)
* documention in README: 118 lines
* documentation in `docs`: 0 lines (0.0% of `docs` + `src`)
* documentation in README: 118 lines
* has license(s) in file: MIT
* filename: LICENSE
* OSI approved: true
Expand Down Expand Up @@ -102,8 +102,8 @@ PackageV1 DataFrames:
* tree hash: db2a9cb664fcea7836da4b414c3278d71dd602d2
* Julia code in `src`: 15628 lines
* Julia code in `test`: 21089 lines (57.4% of `test` + `src`)
* documention in `docs`: 6270 lines (28.6% of `docs` + `src`)
* documention in README: 21 lines
* documentation in `docs`: 6270 lines (28.6% of `docs` + `src`)
* documentation in README: 21 lines
* has license(s) in file: MIT
* filename: LICENSE.md
* OSI approved: true
Expand Down Expand Up @@ -138,7 +138,7 @@ end
analyze_manifest([path_to_manifest]; registries=reachable_registries(),
auth=github_auth(), sleep=0)

Convienence function to run [`find_packages_in_manifest`](@ref) then [`analyze`](@ref) on the results. Positional argument `path_to_manifest` defaults to `joinpath(dirname(Base.active_project()), "Manifest.toml")`.
Convenience function to run [`find_packages_in_manifest`](@ref) then [`analyze`](@ref) on the results. Positional argument `path_to_manifest` defaults to `joinpath(dirname(Base.active_project()), "Manifest.toml")`.
"""
function analyze_manifest(args...; registries=reachable_registries(),
auth=github_auth(), sleep=0)
Expand Down
8 changes: 5 additions & 3 deletions src/count_loc.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,14 +94,16 @@ struct GreenNodeWrapper
end

function AbstractTrees.children(wrapper::GreenNodeWrapper)
return map(n -> GreenNodeWrapper(n, wrapper.source), JuliaSyntax.children(wrapper.node))
return map(n -> GreenNodeWrapper(n, wrapper.source), something(JuliaSyntax.children(wrapper.node), ()))
end

const EMPTY_TREE = JuliaSyntax.GreenNode(JuliaSyntax.SyntaxHead(K"toplevel", 0), 0, JuliaSyntax.GreenNode{JuliaSyntax.SyntaxHead}[])

function parse_green_node(file_path)
file = read(file_path, String)
parsed = @maybecatch(JuliaSyntax.parseall(JuliaSyntax.GreenNode, file; ignore_trivia=false, filename=file_path),
"Error during `JuliaSyntax.parseall` of $(file_path)",
JuliaSyntax.GreenNode(JuliaSyntax.SyntaxHead(K"toplevel", 0), 0, ()))
EMPTY_TREE)
return GreenNodeWrapper(parsed, JuliaSyntax.SourceFile(file; filename=file_path))
end

Expand All @@ -127,7 +129,7 @@ function count_julia_lines_of_code(dir)
table = LinesOfCodeV2[]
for path in readdir(dir; join=true)
n_files = 0
# Could be a LittleDict for efficency, but that would require
# Could be a LittleDict for efficiency, but that would require
# a dependency on OrderedCollections.jl
counts = Dict{LineCategory,Int}(Comment => 0,
Blank => 0,
Expand Down
2 changes: 1 addition & 1 deletion src/utilities.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
"""
PackageAnalyzer.github_auth(token::String="")

Obtain a GitHub authetication. Use the `token` argument if it is non-empty,
Obtain a GitHub authentication. Use the `token` argument if it is non-empty,
otherwise use the `GITHUB_TOKEN` and `GITHUB_AUTH` environment variables, if set
and of length 40. If all these methods fail, return an anonymous
authentication.
Expand Down
14 changes: 14 additions & 0 deletions test/runtests.jl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -451,3 +451,17 @@ end
finally
PackageAnalyzer.CATCH_EXCEPTIONS[] = catch_exceptions_value
end

# This test relies on catching exceptions
try
PackageAnalyzer.CATCH_EXCEPTIONS[] = true
@testset "Graceful handling of unparsable files" begin
# This codebase includes an uncommented `TODO` which fails to parse
# in `sample/test1.jl` we should just skip that file instead of erroing out
pkg = analyze("Pluto"; version=v"0.18.0")
l_src = PackageAnalyzer.sum_julia_loc(pkg, "src")
@test l_src > 6000
end
catch
PackageAnalyzer.CATCH_EXCEPTIONS[] = catch_exceptions_value
end
Loading