SNOW-1764178: Adding multiline/tabbing to generated SQL statements

[sfc-gh-daviwang]

1. Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR.

   Fixes SNOW-1764178

2. Fill out the following pre-review checklist:

   • I am adding a new automated test(s) to verify correctness of my new code
   • If this test skips Local Testing mode, I'm requesting review from @snowflakedb/local-testing
   • I am adding new logging messages
   • I am adding a new telemetry message
   • I am adding new credentials
   • I am adding a new dependency
   • If this is a new feature/behavior, I'm adding the Local Testing parity changes.
   • I acknowledge that I have ensured my changes to be thread-safe. Follow the link for more information: Thread-safe Developer Guidelines
   • If adding any arguments to public Snowpark APIs or creating new public Snowpark APIs, I acknowledge that I have ensured my changes include AST support. Follow the link for more information: AST Support Guidelines

3. Please describe how your code solves the related issue.

Added NEW_LINE and TAB character to analyzer_utils.py so generated SQL includes newlines and SQL formatting. We do not have nested tabbing for child queries b/c doing so would make us unable to include "\n" in identifiers. We only use tabbing to indent column names, group bys, and order bys. This setting is protected by the parameter PYTHON_SNOWPARK_GENERATE_MULTILINE_QUERIES in https://github.com/snowflakedb/snowflake/pull/294205, and is automatically enabled due to this being a low risk change. This behavior is disabled for snowpark pandas tests. Also added additional unit tests to verify formatting.

An example of the new behavior:

SELECT  * 
 FROM (
( SELECT 
    "ID" AS "l_0002_ID", 
    "NAME" AS "NAME"
 FROM (
 SELECT $1 AS "ID", $2 AS "NAME" FROM  VALUES (1 :: INT, 'A' :: STRING), (2 :: INT, 'B' :: STRING)
)) AS SNOWPARK_LEFT 
INNER JOIN 
( SELECT 
    "ID" AS "r_0003_ID", 
    "VALUE" AS "VALUE"
 FROM (
 SELECT $1 AS "ID", $2 AS "VALUE" FROM  VALUES (1 :: INT, 10 :: INT), (2 :: INT, 20 :: INT)
)) AS SNOWPARK_RIGHT
 ON ("l_0002_ID" = "r_0003_ID")
)

Old behavior (all one line):

SELECT  *  FROM (( SELECT "ID" AS "l_0000_ID", "NAME" AS "NAME" FROM ( SELECT "ID", "NAME" FROM ( SELECT $1 AS "ID", $2 AS "NAME" FROM  VALUES (1 :: INT, 'A' :: STRING), (2 :: INT, 'B' :: STRING)))) AS SNOWPARK_LEFT INNER JOIN ( SELECT "ID" AS "r_0001_ID", "VALUE" AS "VALUE" FROM ( SELECT "ID", "VALUE" FROM ( SELECT $1 AS "ID", $2 AS "VALUE" FROM  VALUES (1 :: INT, 10 :: INT), (2 :: INT, 20 :: INT)))) AS SNOWPARK_RIGHT ON ("l_0000_ID" = "r_0001_ID"))```

[github-actions]

All contributors have signed the CLA ✍️ ✅
_{Posted by the CLA Assistant Lite bot.}

[sfc-gh-snowflakedb-snyk-sa]

🎉 Snyk checks have passed. No issues have been found so far.

✅ security/snyk check is complete. No issues have been found. (View Details)

✅ license/snyk check is complete. No issues have been found. (View Details)

[sfc-gh-daviwang]

I have read the CLA Document and I hereby sign the CLA

[sfc-gh-joshi]

Who's the intended audience for this change? I'd prefer is this could be guarded by some kind of configuration flag, as I'm worried this could make logging noisier than it already is, especially in Snowpark pandas, where queries can get very large very fast. It already takes browsers a very long time to load the result page of a CI run, and having more newlines would likely exacerbate that.

We do not have nested tabbing for child queries b/c doing so would make us unable to include "\n" in identifiers.

Why exactly is this a limitation? Nested indentation would help readability a lot.

[sfc-gh-daviwang]

Who's the intended audience for this change? I'd prefer is this could be guarded by some kind of configuration flag, as I'm worried this could make logging noisier than it already is, especially in Snowpark pandas, where queries can get very large very fast. It already takes browsers a very long time to load the result page of a CI run, and having more newlines would likely exacerbate that.

Some clients want this change so their queries are more readable. Also this is part of a series of changes where we want to attach errors to specific lines of SQL, which isn't possible if the entire SQL query is on one line. Adding a configuration flag makes sense, I put a PR to parameter protect this feature, and will adjust the code accordingly.

Why exactly is this a limitation? Nested indentation would help readability a lot.

The main issue is that there isn't an easy way to differentiate between a "\n" character that a customer input as opposed to one we created for formatting purposes. Without an easy way to differentiate these, our indentation can break current behavior (eg. test_drop_columns_special_names in test_dataframe.py). I've discussed this a bunch offline already with @sfc-gh-aalam and we came to the conclusion that this approach makes the most sense. Feel free to reach out on slack if you would like to discuss.