Skip to content

Conversation

@doorgan
Copy link

@doorgan doorgan commented Jan 6, 2026

Trying to integrate Spitfire into Expert I found a few issues:

  • Spitfire.parse_with_comments("@x ]") crashes instead of returning an error:
** (CaseClauseError) no case clause matching: {:error, :no_fuel_remaining}
    (spitfire 0.2.1) lib/spitfire.ex:174: Spitfire.parse_with_comments/2
    iex:27: (file)
  • Spitfire.parse("[«]") crashes(this one was triggered by one of Expert's test helpers):
** (FunctionClauseError) no function clause matching in Spitfire.peek_token/1

    The following arguments were given to Spitfire.peek_token/1:

        # 1
        %{
          tokens: [nil | :eot],
          literal_encoder: nil,
          errors: [
            {[line: 1, column: 1], "missing closing bracket for list"},
            {[], "unknown token: eot"}
          ],
          current_token: {:"]", nil},
          peek_token: nil,
          nesting: 0,
          fuel: 150
        }

    Attempted function clauses (showing 7 out of 7):

        defp peek_token(%{peek_token: {:stab_op, _, token}})
        defp peek_token(%{peek_token: {type, _, _, _}}) when type === :list_heredoc or type === :bin_heredoc
        defp peek_token(%{peek_token: {token, _, _}})
        defp peek_token(%{peek_token: {token, _}})
        defp peek_token(%{peek_token: {token, _, _, _, _, _, _}})
        defp peek_token(%{peek_token: :eof})
        defp peek_token(%{tokens: :eot})

    (spitfire 0.2.1) lib/spitfire.ex:2390: Spitfire.peek_token/1
    (spitfire 0.2.1) lib/spitfire.ex:298: anonymous fn/7 in Spitfire.parse_expression/6
    (spitfire 0.2.1) lib/spitfire/while.ex:49: Spitfire.While.do_while/2
    (spitfire 0.2.1) lib/spitfire.ex:196: anonymous fn/1 in Spitfire.parse_program/1
    (spitfire 0.2.1) lib/spitfire/while.ex:5: Spitfire.While2.recurse/3
    (spitfire 0.2.1) lib/spitfire.ex:195: Spitfire.parse_program/1
    (spitfire 0.2.1) lib/spitfire.ex:140: Spitfire.parse/2
    iex:31: (file)

This PR fixes those

@mhanberg
Copy link
Contributor

mhanberg commented Jan 6, 2026

I'm not sure this exactly the right fix, as the error cases showing 'unknown token: eot' is exposing an implementation detail that isn't a real syntax error.

I think you can maybe split the struct error into its own PR.

I'll dig into it more, how does the tokenizer actually tokenize that special << character?

@doorgan
Copy link
Author

doorgan commented Jan 6, 2026

I'll dig into it more, how does the tokenizer actually tokenize that special << character?

As [171] which corresponds to "«", which seems correct

I think you can maybe split the struct error into its own PR.

Will do. Also for that one I'm not 100% either that the behavior I chose is the one we really want either, we can discuss further there

@doorgan doorgan force-pushed the doorgan/expert-bugs branch from 79b5b25 to 5ed6d78 Compare January 6, 2026 16:37
@doorgan doorgan changed the title fix: infinite recursion and out of fuel errors fix: parsing crashes on code with errors Jan 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants