Regular expressions in tables being affected by the refactor

[RRead91]

Describe the bug

When running the reformatter on a feature file where a table has a "", the reformatter adds an extra .

To Reproduce

Steps to reproduce the behaviour

Run the refactor on the following:

Feature: Demo

  Scenario Outline: demo
     Given I have a regex value "report_\S+\.json"
       And A table with
       | report_\S+\.csv |
       And an outline value <regex>

    Examples:
        | regex            |
        | report_\S+\.html |

Expected behaviour

The feature file is analysed and the regular expressions aren't affected

Actual behaviour

The regex in the tables has been changed, adding extra \s

Feature: Demo issue

  Scenario Outline: demo refactor gherkin issue
    Given I have a regex value "report_\S+\.json"
      And A table with
      | report_\\S+\\.csv |
      And an outline value <regex>

    Examples:
      | regex              |
      | report_\\S+\\.html |

Environment

reformat-gherkin, version 3.0.1
Python 3.12.9
MacOS 15.3.1

[ducminh-phan]

@RRead91 I would argue that this is a feature. The \-character will be considered as an escape in a table cell iff it is followed by a |-character, a \-character or an n (source).

So the non-escape \ characters before S and . are "normalized" into \\ after reformating. And from gherkin's point of view, the two are identical, as can be seen by their AST:

AST

{
  "feature": {
    "tags": [],
    "location": {
      "line": 1,
      "column": 1
    },
    "language": "en",
    "keyword": "Feature",
    "name": "Demo",
    "description": "",
    "children": [
      {
        "scenario": {
          "id": "7",
          "tags": [],
          "location": {
            "line": 3,
            "column": 3
          },
          "keyword": "Scenario Outline",
          "name": "demo",
          "description": "",
          "steps": [
            {
              "id": "0",
              "location": {
                "line": 4,
                "column": 6
              },
              "keyword": "Given ",
              "keywordType": "Context",
              "text": "I have a regex value \"report_\\S+\\.json\""
            },
            {
              "id": "2",
              "location": {
                "line": 5,
                "column": 8
              },
              "keyword": "And ",
              "keywordType": "Conjunction",
              "text": "A table with",
              "dataTable": {
                "location": {
                  "line": 6,
                  "column": 8
                },
                "rows": [
                  {
                    "id": "1",
                    "location": {
                      "line": 6,
                      "column": 8
                    },
                    "cells": [
                      {
                        "location": {
                          "line": 6,
                          "column": 10
                        },
                        "value": "report_\\S+\\.csv"
                      }
                    ]
                  }
                ]
              }
            },
            {
              "id": "3",
              "location": {
                "line": 7,
                "column": 8
              },
              "keyword": "And ",
              "keywordType": "Conjunction",
              "text": "an outline value <regex>"
            }
          ],
          "examples": [
            {
              "id": "6",
              "tags": [],
              "location": {
                "line": 9,
                "column": 5
              },
              "keyword": "Examples",
              "name": "",
              "description": "",
              "tableHeader": {
                "id": "4",
                "location": {
                  "line": 10,
                  "column": 9
                },
                "cells": [
                  {
                    "location": {
                      "line": 10,
                      "column": 11
                    },
                    "value": "regex"
                  }
                ]
              },
              "tableBody": [
                {
                  "id": "5",
                  "location": {
                    "line": 11,
                    "column": 9
                  },
                  "cells": [
                    {
                      "location": {
                        "line": 11,
                        "column": 11
                      },
                      "value": "report_\\S+\\.html"
                    }
                  ]
                }
              ]
            }
          ]
        }
      }
    ]
  },
  "comments": []
}

AST generation snippet

import json

from gherkin.parser import Parser


content = r"""
Feature: Demo

  Scenario Outline: demo
     Given I have a regex value "report_\S+\.json"
       And A table with
       | report_\S+\.csv |
       And an outline value <regex>

    Examples:
        | regex            |
        | report_\S+\.html |
""".strip()

parser = Parser()
ast = parser.parse(content)
with open("ast.json", "w") as f:
    json.dump(ast, f, indent=2)

[RRead91]

Hi @ducminh-phan, Thanks for responding to my issue, I think I've worked out the problem I'm facing is due to both the reformatter adding the extra escape characters followed by the framework (behave in my case) doing it again. Here is what it looks like:

behave --tags demo --dry-run
@demo
Feature: Demo # features/demo.feature:2

  Scenario Outline: demo -- @1.1                  # features/demo.feature:12
    Given I have a regex value "report_\S+\.json" # steps/demo.py:6
    And A table with                              # steps/demo.py:11
      | report_\\\\S+\\\\.csv |
    And an outline value report_\\S+\\.html       # steps/demo.py:16

The table ends up going from 2 to 4 in the data table

This isn't the fault of the reformatter but is there any chance a change could be added to ignore adding extra escape characters?

[ducminh-phan]

Could you share the files and the steps to reproduce? I would want to understand more about the problem.

When looking at behave's source code, it seems like they write their own parser, while reformat-gherkin uses gherkin-official. So there could be some discrepancy here. In that case, I think it would be best to report the issue to behave.