Merge pull request #45 from fivetran/release/v0.7.0

Release/v0.7.0

Author: fivetran-catfritz

.buildkite/pipeline.yml

@@ -1,4 +1,19 @@
 steps:
+  - label: ":postgres: Run Tests - Postgres"
+    key: "run-dbt-postgres"
+    plugins:
+      - docker#v3.13.0:
+          image: "python:3.8"
+          shell: [ "/bin/bash", "-e", "-c" ]
+          environment:
+            - "BASH_ENV=/tmp/.bashrc"
+            - "CI_POSTGRES_DBT_DBNAME"
+            - "CI_POSTGRES_DBT_HOST"
+            - "CI_POSTGRES_DBT_PASS"
+            - "CI_POSTGRES_DBT_USER"
+    commands: |
+      bash .buildkite/scripts/run_models.sh postgres
+      
   - label: ":snowflake-db: Run Tests - Snowflake"
     key: "run_dbt_snowflake"
     plugins:
@@ -43,7 +58,7 @@ steps:
     commands: |
       bash .buildkite/scripts/run_models.sh redshift
 
-  - label: ":bricks: Run Tests - Databricks"
+  - label: ":databricks: Run Tests - Databricks"
     key: "run_dbt_databricks"
     plugins:
       - docker#v3.13.0:

CHANGELOG.md

@@ -1,3 +1,23 @@
+# dbt_github v0.7.0
+
+## 🚨 Breaking Change 🚨
+- Updated the following models to aggregate at the `repository` grain in addition to their time period grain. ([#42](https://github.com/fivetran/dbt_github/pull/42), [#43](https://github.com/fivetran/dbt_github/pull/43))
+  - `github__daily_metrics`
+  - `github__weekly_metrics`
+  - `github__monthly_metrics`
+  - `github__quarterly_metrics`
+## 🎉 Features
+- Added column `requested_reviewers` to provide a list of users that were requested to review on a pull request. This is to supplement the column `reviewers`, which provides a list of users that have submitted a reivew, whether or not they were requested to. ([#44](https://github.com/fivetran/dbt_github/pull/44))
+- PostgreSQL compatibility! ([#44](https://github.com/fivetran/dbt_github/pull/44))
+## 🔧 Bug Fix
+- Updated model `int_github__pull_request_reviewers` so that the list of reviewers generated does not contain duplicate usernames. ([#44](https://github.com/fivetran/dbt_github/pull/44))
+## 🚘 Under the Hood
+- For the metrics models that were updated, added unique-combination-of-column tests for the combination of the time period and repository. ([#44](https://github.com/fivetran/dbt_github/pull/44))
+- Removed uniqueness tests on time period in metrics models in favor of the combo test. ([#44](https://github.com/fivetran/dbt_github/pull/44))
+- Removed ordering in metrics models to improve efficiency. ([#44](https://github.com/fivetran/dbt_github/pull/44))
+## 📝 Contributors 
+- @onimsha ([#42](https://github.com/fivetran/dbt_github/pull/42))
+
 # dbt_github v0.6.0
 [PR #35](https://github.com/fivetran/dbt_github/pull/35) includes the following breaking changes:
 ## 🚨 Breaking Changes 🚨:

README.md

@@ -27,18 +27,18 @@ The following table provides a detailed list of all models materialized within t
 | -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- |
 | [github__issues](https://fivetran.github.io/dbt_github/#!/model/model.github.github__issues)     | Each record represents a GitHub issue, enriched with data about its assignees, milestones, and time comparisons.                                             |
 | [github__pull_requests](https://fivetran.github.io/dbt_github/#!/model/model.github.github__pull_requests)     | Each record represents a GitHub pull request, enriched with data about its repository, reviewers, and durations between review requests, merges and reviews. |
-| [github__daily_metrics](https://fivetran.github.io/dbt_github/#!/model/model.github.github__daily_metrics)     | Each record represents a single day, enriched with metrics about PRs and issues that were created and closed during that period.                              |
-| [github__weekly_metrics](https://fivetran.github.io/dbt_github/#!/model/model.github.github__weekly_metrics)    | Each record represents a single week, enriched with metrics about PRs and issues that were created and closed during that period.                             |
-| [github__monthly_metrics](https://fivetran.github.io/dbt_github/#!/model/model.github.github__monthly_metrics)   | Each record represents a single month, enriched with metrics about PRs and issues that were created and closed during that period.                            |
-| [github__quarterly_metrics](https://fivetran.github.io/dbt_github/#!/model/model.github.github__quarterly_metrics) | Each record represents a single quarter, enriched with metrics about PRs and issues that were created and closed during that period.                          |
+| [github__daily_metrics](https://fivetran.github.io/dbt_github/#!/model/model.github.github__daily_metrics)     | Each record represents a single day and repository, enriched with metrics about PRs and issues that were created and closed during that period.                              |
+| [github__weekly_metrics](https://fivetran.github.io/dbt_github/#!/model/model.github.github__weekly_metrics)    | Each record represents a single week and repository, enriched with metrics about PRs and issues that were created and closed during that period.                             |
+| [github__monthly_metrics](https://fivetran.github.io/dbt_github/#!/model/model.github.github__monthly_metrics)   | Each record represents a single month and repository, enriched with metrics about PRs and issues that were created and closed during that period.                            |
+| [github__quarterly_metrics](https://fivetran.github.io/dbt_github/#!/model/model.github.github__quarterly_metrics) | Each record represents a single quarter and repository, enriched with metrics about PRs and issues that were created and closed during that period.                          |
 <!--section-end-->
 
 # 🎯 How do I use the dbt package?
 ## Step 1: Prerequisites
 To use this dbt package, you must have the following:
 
 - At least one Fivetran Github connector syncing data into your destination.
-- A **BigQuery**, **Snowflake**, **Redshift**, or **Databricks** destination.
+- A **BigQuery**, **Snowflake**, **Redshift**, **PostgreSQL**, or **Databricks** destination.
 
 ### Databricks Dispatch Configuration
 If you are using a Databricks destination with this package you will need to add the below (or a variation of the below) dispatch configuration within your `dbt_project.yml`. This is required in order for the package to accurately search for macros within the `dbt-labs/spark_utils` then the `dbt-labs/dbt_utils` packages respectively.
@@ -55,7 +55,7 @@ Include the following github package version in your `packages.yml` file.
 ```yaml
 packages:
   - package: fivetran/github
-    version: [">=0.6.0", "<0.7.0"]
+    version: [">=0.7.0", "<0.8.0"]
 ```
 
 ## Step 3: Define database and schema variables
@@ -125,7 +125,7 @@ packages:
       version: [">=1.0.0", "<2.0.0"]
 
     - package: fivetran/github_source
-      version: [">=0.6.0", "<0.7.0"]
+      version: [">=0.7.0", "<0.8.0"]
     
     - package: dbt-labs/spark_utils
       version: [">=0.3.0", "<0.4.0"]

dbt_project.yml

@@ -1,6 +1,6 @@
 config-version: 2
 name: 'github'
-version: '0.6.0'
+version: '0.7.0'
 require-dbt-version: [">=1.3.0", "<2.0.0"]
 models:
   github:

docs/catalog.json

@@ -1,5 +1,5 @@
 name: 'github_integration_tests'
-version: '0.6.0'
+version: '0.7.0'
 config-version: 2
 profile: 'integration_tests'
 vars:

docs/index.html

@@ -174,17 +174,24 @@ models:
       - name: reviewers
         description: List of Users who reviewed the pull request
 
+      - name: requested_reviewers
+        description: List of Users who were requested to review the pull request
+
       - name: number_of_reviews
         description: Number of times a pull request was reviewed
 
   - name: github__daily_metrics
     description: >
       Summary numbers for issues and pull requests by day
+    tests:
+      - dbt_utils.unique_combination_of_columns:
+          combination_of_columns:
+            - repository
+            - day
     columns:
       - name: day
         description: The reporting day
         tests:
-          - unique
           - not_null
       - name: number_issues_opened
         description: The total number of issues created during this time period
@@ -207,6 +214,11 @@ models:
       - name: number_prs_closed_without_merge
         description: The total number of pull requests closed without a merge during this time period
 
+      - name: repository
+        description: The name of the repository
+        tests:
+          - not_null
+
       - name: sum_days_pr_open
         description: The total number of days a pull request opened during this time period was open
 
@@ -217,11 +229,15 @@ models:
   - name: github__weekly_metrics
     description: >
       Summary numbers for issues and pull requests by week
+    tests:
+      - dbt_utils.unique_combination_of_columns:
+          combination_of_columns:
+            - repository
+            - week
     columns:
       - name: week
         description: The reporting week
         tests:
-          - unique
           - not_null
       - name: number_issues_opened
         description: The total number of issues created during this time period
@@ -249,15 +265,24 @@ models:
 
       - name: longest_days_pr_open
         description: The longest number of days a pull request opened during this time period was open
+
+      - name: repository
+        description: The name of the repository
+        tests:
+          - not_null
 
   - name: github__monthly_metrics
     description: >
       Summary numbers for issues and pull requests by month
+    tests:
+      - dbt_utils.unique_combination_of_columns:
+          combination_of_columns:
+            - repository
+            - month
     columns:
       - name: month
         description: The reporting month
         tests:
-          - unique
           - not_null
       - name: number_issues_opened
         description: The total number of issues created during this time period
@@ -285,15 +310,24 @@ models:
 
       - name: longest_days_pr_open
         description: The longest number of days a pull request opened during this time period was open
+
+      - name: repository
+        description: The name of the repository
+        tests:
+          - not_null
 
   - name: github__quarterly_metrics
     description: >
       Summary numbers for issues and pull requests by quarter
+    tests:
+      - dbt_utils.unique_combination_of_columns:
+          combination_of_columns:
+            - repository
+            - quarter
     columns:
       - name: quarter
         description: The reporting quarter
         tests:
-          - unique
           - not_null
       - name: number_issues_opened
         description: The total number of issues created during this time period
@@ -321,4 +355,8 @@ models:
 
       - name: longest_days_pr_open
         description: The longest number of days a pull request opened during this time period was open
-                
+
+      - name: repository
+        description: The name of the repository
+        tests:
+          - not_null