Feat/add snowflake support (#44)

* Add snowflake grammar support

* Update rule helper tool

* Update rules

* Cleanup

---------

Co-authored-by: gretard <[email protected]>

Author: gretard

.gitignore

@@ -33,3 +33,4 @@ examples/1-tsql/tools
 
 **/tools/**
 
+src/sonar-sql-plugin/examples/**

README.md

@@ -17,6 +17,7 @@ Currently plug-in supports:
   - [PostgreSQL](https://github.com/tshprecher/antlr_psql)
   - [PostgreSQL](https://github.com/antlr/grammars-v4/tree/master/sql/postgresql)
   - [VSQL](https://github.com/gretard/antlr4-grammar-vsql)
+  - [SNOWFLAKE](https://github.com/antlr/grammars-v4/tree/master/sql/snowflake)
 - Reporting of issues found by:
   - [SQLCodeGuard](https://www.red-gate.com/products/sql-development/sql-code-guard/index) 
   - [MSBuild](https://msdn.microsoft.com/en-us/library/dd172133(v=vs.100).aspx)
@@ -34,8 +35,8 @@ Tutorials:
 
 ## Requirements ##
 Different plugin versions supports the following:
-- 1.0.0 - Sonarqube 7.4+versions
-- 1.2.0 - Sonarqube 9+versions
+- 1.0.0 - Sonarqube 7.4+ versions
+- 1.2.0 - Sonarqube 9+ versions
 
 ## Installation ##
 1. Download and install SonarQube
@@ -97,6 +98,17 @@ sonar.sources=src
 sonar.language=sql
 sonar.sql.dialect=vsql
 ```
+### SNOWFLAKE ###
+Sonar settings for snowflake. You can check example at [here](https://github.com/gretard/sonar-sql-plugin/tree/master/examples/4-vsql)
+```
+sonar.projectKey=examples.sql.snowflake.project
+sonar.projectName=examples.sql.snowflake.project
+sonar.projectVersion=1.1
+sonar.sources=src
+# optional
+sonar.language=sql
+sonar.sql.dialect=snowflake
+```
 
 ### Custom rules example ###
 This is an example for sonar settings for project which uses custom plugin rules from local directory (located at ./rules directory). You can check full example at [here](https://github.com/gretard/sonar-sql-plugin/tree/master/examples/6-pssql-with-custom-rules)
@@ -140,9 +152,9 @@ Please configure additional properties:
 `sonar.lang.patterns.plsqlopen=na`
 
 ## Using command line tools
-With the plugin - there is additional cli tool available (they are not required for sonar execution):
+With the plugin - there is additional cli tool available (it does not require sonar execution):
 
- - **rulesHelper.jar** - command line helper tool for writing custom sql rules
+ - **rulesHelper.jar** - command line helper tool for working with plugin and custom sql rules
 
 
 ### rulesHelper
@@ -155,19 +167,30 @@ Full help info:
 
 ```
 Please pass the following: 
-	action (print or  verify)
-	type (text or  file)
-	value (sql string or path to folder) 
-	dialect (tsql, pssql, mysql, pssql, pssqlv2) 
-Example:
-print text "SELECT * FROM dbo.test;" tsql
+	action  (print, verify or analyze)
+	type    (text or file)
+	value   (sql string or path to rules file/folder) 
+	dialect (tsql, pssql, mysql, pssql, pssqlv2, snowflake)
+	folder  (folder to analyze, only applicable when using analyze action)
 
-Example:
-verify file "c:/tests/customRules.rules;" mysql
+Example to print AST tree:
+	print text "SELECT * FROM dbo.test;" tsql
+
+Example to verify custom rules definitions:
+	verify file "c:/tests/customRules.rules;" mysql
+
+Example to execute custom rules and plugin rules against specified folder:
+	analyze file "c:/tests/customRules.rules;" snowflake "c:\docs\src"
+
+Example to execute sql analysis againt specified folder:
+	analyze file "NA" snowflake "c:\docs\src"
 ```
 
 
 ## Contributing ##
+### Building locally
+Run: ```mvn versions:display-dependency-updates spotless:check spotless:apply install```
+
 ### Developing locally
 Added container definitions for easy development with VSCode. Download the [remote containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers) and let it figure out the maven targets. 
 <img width="1917" alt="vscode_remote_containers_extension_maven" src="https://user-images.githubusercontent.com/3657015/125957363-653c9f6f-b5cc-4a3c-96ef-9dc18d0f8bfb.png">
@@ -214,3 +237,5 @@ docker run \
 3. Implement sql rules, example VSQLRules
 4. Register rules at SQLDialectRules. This step is optional as plugin will support custom rules from user project provided in xml format.
 5. Update ./src/external/README.md with references to your added grammar
+
+Example commit for adding Snowflake grammar: https://github.com/gretard/sonar-sql-plugin/commit/e3296a5d1c69a031f24358aad87a4e46c46ea785

azure-pipelines.yml

@@ -1,17 +1,23 @@
 strategy:
   matrix:
     linux:
-      imageName: 'ubuntu-16.04'
+      imageName: 'ubuntu-latest'
     mac:
-      imageName: 'macos-10.13'
+      imageName: 'macos-latest'
     windows:
-      imageName: 'vs2017-win2016'
+      imageName: 'windows-latest'
 
 pool:
   vmImage: $(imageName)
   demands: maven
 
 steps:
+- task: JavaToolInstaller@0
+  inputs:
+    versionSpec: '11'
+    jdkArchitectureOption: 'x64'
+    jdkSourceOption: 'PreInstalled'
+
 - task: Maven@3
   displayName: 'Maven sonar-sql-plugin install'
   inputs:

docs/customRulesSetup.md

@@ -45,6 +45,7 @@ Mandatory fields for rule:
 		- FailIfNotFound - reports code issue if no node was found violating rule
 		- FailIfLessFound/FailIfMoreFound - reports code issue if rule was not violated less or more times than value defined by times value
 		- SkipIfFound/SkipIfNotFound - skips code issue reporting at all if rule was satisfied or not
+        - SkipIfLessFound/SkipIfMoreFound - skips code issue reporting at all if rule was matches less or more times than value defined by times value
 	- textCheckType - defines how to search for particular text in nodes, can be:
 		- Contains - will be checked if node contains some defined text
 		- Regexp - will be checked if node's text matches regexp

docs/pluginRules.md

@@ -3,16 +3,16 @@
 SLEEP/WAITFOR is used<h2>Code examples</h2><h3>Non-compliant</h3><pre><code>WAITFOR '10:00:00';</code></pre>
 
 ## C002 - SELECT * is used
-<p>Supported dialects: MYSQL,PSSQLV2,TSQL</p>
+<p>Supported dialects: MYSQL,SNOWFLAKE,PSSQLV2,TSQL</p>
 <h2>Description</h2><p>SELECT * is used. Please list names.</p><h2>Code examples</h2><h3>Non-compliant</h3><pre><code>SELECT t1.*, t2.* from dbo.test as t1 inner join dbo.test2 as t2 on t1.id=t2.id;</code></pre><h3>Compliant</h3><pre><code>SELECT name, surname from dbo.test;</code></pre><pre><code>SELECT name, surname, 1 * 3 from dbo.test;</code></pre>
 
 ## C003 - INSERT statement without columns listed
-<p>Supported dialects: MYSQL,PSSQLV2,TSQL</p>
+<p>Supported dialects: MYSQL,SNOWFLAKE,PSSQLV2,TSQL</p>
 <h2>Description</h2><p>INSERT statement does not have columns listed. Always use a column list in your INSERT statements.</p><h2>Code examples</h2><h3>Non-compliant</h3><pre><code>INSERT INTO dbo.test VALUES (1,2);
 INSERT INTO dbo.test2 VALUES (1,2);</code></pre><h3>Compliant</h3><pre><code>INSERT INTO dbo.test (a,b) VALUES (1,2);</code></pre>
 
 ## C004 - ORDER BY clause contains positional references
-<p>Supported dialects: MYSQL,PSSQL,PSSQLV2,TSQL</p>
+<p>Supported dialects: MYSQL,PSSQL,SNOWFLAKE,PSSQLV2,TSQL</p>
 <h2>Description</h2><p>Do not use column numbers in the ORDER BY clause. Always use column names in an order by clause. Avoid positional references.</p><h2>Code examples</h2><h3>Non-compliant</h3><pre><code>SELECT * from dbo.test order by 1, 2;</code></pre><h3>Compliant</h3><pre><code>SELECT * from dbo.test order by name;</code></pre>
 
 ## C005 - EXECUTE/EXEC for dynamic query is used
@@ -24,8 +24,8 @@ INSERT INTO dbo.test2 VALUES (1,2);</code></pre><h3>Compliant</h3><pre><code>INS
 <h2>Description</h2><p>Use of NOLOCK might cause data inconsistency problems.</p><h2>Code examples</h2><h3>Non-compliant</h3><pre><code>SELECT name, surname from dbo.test WITH (NOLOCK);</code></pre><h3>Compliant</h3><pre><code>SELECT name, surname from dbo.test;</code></pre>
 
 ## C009 - Non-sargable statement used
-<p>Supported dialects: MYSQL,PSSQL,PSSQLV2,TSQL</p>
-<h2>Description</h2><p>Use of non-sargeable arguments might cause performance problems.</p><h2>Code examples</h2><h3>Non-compliant</h3><pre><code>SELECT name, surname from dbo.test where year(date) > 2008 and month = 12;</code></pre><pre><code>SELECT name, surname from dbo.test where name like '%red' </code></pre><h3>Compliant</h3><pre><code>SELECT MAX(RateChangeDate)  FROM HumanResources.EmployeePayHistory WHERE BusinessEntityID = 1</code></pre><pre><code>SELECT name, surname from dbo.test where date between 2008-10-10 and 2010-10-10;</code></pre><pre><code>SELECT max(price) from dbo.items;</code></pre>
+<p>Supported dialects: VSQL,MYSQL,PSSQL,SNOWFLAKE,PSSQLV2,TSQL</p>
+<h2>Description</h2><p>Use of non-sargeable arguments might cause performance problems.</p><h2>Code examples</h2><h3>Non-compliant</h3><pre><code>SELECT name, surname from dbo.test where year(date) > 2008 and month = 12;</code></pre><pre><code>SELECT name, surname from dbo.test where name like '%red' </code></pre><h3>Compliant</h3><pre><code>SELECT MAX(RateChangeDate)  FROM HumanResources.EmployeePayHistory WHERE BusinessEntityID = 1</code></pre><pre><code>SELECT name, surname from dbo.test where date between 2008-10-10 and 2010-10-10;</code></pre><pre><code>SELECT max(price) from dbo.items;</code></pre><pre><code>select department_id, count(*) from employees group by department_id having count (*) < 10;</code></pre>
 
 ## C010 - Defined primary key is not using recommended naming convention
 <p>Supported dialects: TSQL</p>
@@ -44,39 +44,47 @@ PRIMARY KEY (Id)
 <h2>Description</h2><p>Defined foreign key is not using recommended naming convention to start with FK_.</p><h2>Code examples</h2><h3>Non-compliant</h3><pre><code>ALTER TABLE dbo.Orders ADD CONSTRAINT ClientId FOREIGN KEY (ClientId) REFERENCES dbo.Clients(Id);  </code></pre><h3>Compliant</h3><pre><code>ALTER TABLE dbo.Orders ADD CONSTRAINT FK_ClientId FOREIGN KEY (ClientId) REFERENCES dbo.Clients(Id); </code></pre>
 
 ## C012 - Comparison operator (=, <>, !=) to check if value is null used
-<p>Supported dialects: MYSQL,PSSQL,PSSQLV2,TSQL</p>
+<p>Supported dialects: MYSQL,PSSQL,SNOWFLAKE,PSSQLV2,TSQL</p>
 <h2>Description</h2><p>It is not advisable to use comparison operator to check if value is null as comparison operators return UNKNOWN when either or both arguments are NULL. Please use IS NULL or IS NOT NULL instead.</p><h2>Code examples</h2><h3>Non-compliant</h3><pre><code>SELECT * from dbo.test where name = null and surname = 'Test' ;</code></pre><pre><code>SELECT * from dbo.test where name != null;</code></pre><pre><code>SELECT * from dbo.test where name <> null;</code></pre><h3>Compliant</h3><pre><code>SELECT * from dbo.test where name IS NULL;</code></pre><pre><code>SELECT * from dbo.test where name IS NOT NULL;</code></pre><pre><code>SELECT * from dbo.test where name = 'test';</code></pre>
 
 ## C013 - Defined index name is not using recommended naming convention
 <p>Supported dialects: TSQL</p>
 <h2>Description</h2><p>Defined index name is not using recommended naming convention to start with IX_.</p><h2>Code examples</h2><h3>Non-compliant</h3><pre><code>CREATE UNIQUE INDEX Test_Name on dbo.test (Name);</code></pre><h3>Compliant</h3><pre><code>CREATE UNIQUE INDEX IX_Test_Name on dbo.test (Name);</code></pre>
 
 ## C014 - OR verb is used in a WHERE clause
-<p>Supported dialects: MYSQL,PSSQL,PSSQLV2,TSQL</p>
+<p>Supported dialects: MYSQL,PSSQL,SNOWFLAKE,PSSQLV2,TSQL</p>
 <h2>Description</h2><p>It is  advisable to consider using UNION/UNION ALL operator instead of OR verb in the WHERE clause.</p><h2>Code examples</h2><h3>Non-compliant</h3><pre><code>SELECT name, surname, count from dbo.test where name = 'Test' OR surname = 'Testor';</code></pre><h3>Compliant</h3><pre><code>SELECT name, surname, count from dbo.test where name = 'or' and surname = 'TestOR';</code></pre>
 
 ## C015 - UNION operator is used
-<p>Supported dialects: MYSQL,PSSQL,PSSQLV2,TSQL</p>
+<p>Supported dialects: MYSQL,PSSQL,SNOWFLAKE,PSSQLV2,TSQL</p>
 <h2>Description</h2><p>It is  advisable to consider using UNION ALL operator instead of UNION.</p><h2>Code examples</h2><h3>Non-compliant</h3><pre><code>SELECT name, surname, count from dbo.test union SELECT name, surname, count from dbo.test2;</code></pre><h3>Compliant</h3><pre><code>SELECT name, surname, count from dbo.test union all SELECT name, surname, count from dbo.test2;</code></pre>
 
 ## C016 - IN/NOT IN is used for a subquery
-<p>Supported dialects: MYSQL,PSSQL,PSSQLV2,TSQL</p>
+<p>Supported dialects: MYSQL,PSSQL,SNOWFLAKE,PSSQLV2,TSQL</p>
 <h2>Description</h2><p>Consider using EXISTS/NOT EXISTS operator instead of IN for a subquery.</p><h2>Code examples</h2><h3>Non-compliant</h3><pre><code>SELECT name, surname, count from dbo.test where locationID in (select id from dbo.locations);</code></pre><h3>Compliant</h3><pre><code>SELECT name, surname, count from dbo.test where locationID in (1,2,3);</code></pre><pre><code>SELECT name, surname, count from dbo.test where exists (select 1 from dbo.locations where id = locationID);</code></pre>
 
 ## C017 - ORDER BY clause does not contain order (ASC/DESC)
-<p>Supported dialects: MYSQL,PSSQL,PSSQLV2,TSQL</p>
+<p>Supported dialects: MYSQL,PSSQL,SNOWFLAKE,PSSQLV2,TSQL</p>
 <h2>Description</h2><p>It is advisable to specidy order how rows should be ordered.</p><h2>Code examples</h2><h3>Non-compliant</h3><pre><code>SELECT name, surname from dbo.test order by name, surname asc;</code></pre><h3>Compliant</h3><pre><code>SELECT name, surname from dbo.test order by name desc, surname asc;</code></pre>
 
 ## C020 - HINT is used
 <p>Supported dialects: VSQL</p>
-HINT is used. Consider rewriting this statement.<h2>Code examples</h2><h3>Non-compliant</h3><pre><code>SELECT /*+DIRECT*/ * FROM test_table1;</code></pre><pre><code>CREATE TABLE test_table2 AS SELECT /*+DIRECT*/ * FROM test_table1;</code></pre><h3>Compliant</h3><pre><code>SELECT * FROM test_table1;</code></pre>
+<h2>Description</h2><p>HINT is used. Consider rewriting this statement.</p><h2>Code examples</h2><h3>Non-compliant</h3><pre><code>SELECT /*+DIRECT*/ * FROM test_table1;</code></pre><pre><code>CREATE TABLE test_table2 AS SELECT /*+DIRECT*/ * FROM test_table1;</code></pre><h3>Compliant</h3><pre><code>SELECT * FROM test_table1;</code></pre>
 
 ## C021 - COMMIT is missing
 <p>Supported dialects: VSQL</p>
-COMMIT is missing after UPDATE/DELETE statement. If you run script without autocommit - your changes might be lost.<h2>Code examples</h2><h3>Non-compliant</h3><pre><code>SELECT 1; DELETE FROM temp1; SELECT 2;</code></pre><pre><code>COMMIT; DELETE FROM temp1;</code></pre><h3>Compliant</h3><pre><code>SELECT 1; DELETE FROM temp1; COMMIT; SELECT 2; </code></pre>
+<h2>Description</h2><p>COMMIT is missing after UPDATE/DELETE statement. If you run script without autocommit - your changes might be lost.</p><h2>Code examples</h2><h3>Non-compliant</h3><pre><code>SELECT 1; DELETE FROM temp1; SELECT 2;</code></pre><pre><code>COMMIT; DELETE FROM temp1;</code></pre><h3>Compliant</h3><pre><code>SELECT 1; DELETE FROM temp1; COMMIT; SELECT 2; </code></pre>
+
+## C022 - Non-materialized view found
+<p>Supported dialects: SNOWFLAKE</p>
+<h2>Description</h2><p>Consider using materialized view.</p><h2>Code examples</h2><h3>Non-compliant</h3><pre><code>CREATE VIEW V1 (C1) AS SELECT 1 as C1;</code></pre><h3>Compliant</h3><pre><code>CREATE MATERIALIZED VIEW V1 (C1) AS SELECT 1 as C1;</code></pre>
+
+## C023 - Cartesian join found
+<p>Supported dialects: VSQL,MYSQL,SNOWFLAKE,PSSQLV2,TSQL</p>
+<h2>Description</h2><p>Cartesian join without explicit JOIN clause. Consider using explicit JOIN clause.</p><h2>Code examples</h2><h3>Non-compliant</h3><pre><code>SELECT name, order_id FROM customers, orders WHERE customers.id = orders.customer_id;</code></pre><h3>Compliant</h3><pre><code>SELECT name, surname FROM customers c</code></pre><pre><code>SELECT name, bank_name FROM customers c  inner join banks b on c.bank_id = b.id</code></pre>
 
 ## C030 - File does not start with multiline/header comment
-<p>Supported dialects: null</p>
+<p>Supported dialects: TSQL,PSSQL,MYSQL,VSQL,PSSQLV2,SNOWFLAKE</p>
 File does not start with multiline/header comment.<h2>Code examples</h2><h3>Non-compliant</h3><pre><code>SELECT * FROM test_table1;</code></pre><pre><code>SELECT * FROM test_table1; /*additionalComment*/ </code></pre><h3>Compliant</h3><pre><code>/* AUTHOR: test
 Date: 2020-01-01
  */