Skip to content

Commit e7cbf9e

Browse files
authored
docs: update README to MDS
Changed name for Marketing Data Sync and added credit to Google Project
1 parent c32367d commit e7cbf9e

File tree

1 file changed

+40
-13
lines changed

1 file changed

+40
-13
lines changed

README.md

Lines changed: 40 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,21 @@
1-
# Megalista
1+
# MDS - Marketing Data Sync
22

3-
Sample integration code for onboarding offline/CRM data from BigQuery as custom audiences or offline conversions in Google Ads, Google Analytics 360, Google Display & Video 360 and Google Campaign Manager.
3+
Solution based on the [Google Megalista project](https://github.com/google/megalista).
44

5-
**Disclaimer:** This is not an officially supported Google product.
5+
<div align="center">
6+
<img src="https://raw.githubusercontent.com/DP6/templates-centro-de-inovacoes/main/public/images/centro_de_inovacao_dp6.png" height="100px" />
7+
</div>
8+
9+
<p align="center">
10+
<a href="#badge">
11+
<img alt="semantic-release" src="https://img.shields.io/badge/%20%20%F0%9F%93%A6%F0%9F%9A%80-semantic--release-e10079.svg">
12+
</a>
13+
<a href="https://www.codacy.com/gh/DP6/marketing-data-sync/dashboard?utm_source=github.com&amp;utm_medium=referral&amp;utm_content=DP6/marketing-data-sync&amp;utm_campaign=Badge_Grade">
14+
<img alt="Code quality" src="https://app.codacy.com/project/badge/Grade/4bb28565a8b241abae247e6e84778723"/>
15+
</a>
16+
</p>
17+
18+
Sample integration code for onboarding offline/CRM data from BigQuery as custom audiences or offline conversions in Google Ads, Google Analytics 360, Google Display & Video 360, Google Campaign Manager and Facebook Ads.
619

720
## Supported integrations
821
- **Google Ads**
@@ -25,7 +38,7 @@ Sample integration code for onboarding offline/CRM data from BigQuery as custom
2538
- S2S Offline events API (conversion upload), to be used for audience creation and in-app events with Google Ads and DV360 [[details]](https://support.appsflyer.com/hc/en-us/articles/207034486-API-de-eventos-de-servidor-para-servidor-S2S-mobile-para-mobile)
2639

2740
## How does it work
28-
Megalista was design to separate the configuration of conversion/audience upload rules from the engine, giving more freedom for non-technical teams (i.e. Media and Business Inteligence) to setup multiple upload rules on their own.
41+
MDS was design to separate the configuration of conversion/audience upload rules from the engine, giving more freedom for non-technical teams (i.e. Media and Business Inteligence) to setup multiple upload rules on their own.
2942

3043
The solution consists in #1 a Google Spreadsheet (template) in which all rules are defined by mapping a data source (BigQuery Table) to a destination (data upload endpoint) and #2, an apache beam workflow running on Google Dataflow, scheduled to upload the data in batch mode.
3144

@@ -46,7 +59,7 @@ The solution consists in #1 a Google Spreadsheet (template) in which all rules a
4659
- **Google Cloud SDK**
4760

4861
### Access Requirements
49-
Those are the minimum roles necessary to deploy Megalista:
62+
Those are the minimum roles necessary to deploy MDS:
5063
- OAuth Config Editor
5164
- BigQuery User
5265
- BigQuery Job User
@@ -81,23 +94,23 @@ In order to create it, follow these steps:
8194
- On the **OAuth Consent Screen** and configure an *Application name*
8295
- Then, go to the **Credentials** and create an *OAuth client Id* with Application type set as *Desktop App*
8396
- This will generate a *Client Id* and a *Client secret*
84-
- Run the **generate_megalist_token.sh** script in this folder providing these two values and follow the instructions
85-
- Sample: `./generate_megalist_token.sh client_id client_secret`
97+
- Run the **generate_mds_token.sh** script in this folder providing these two values and follow the instructions
98+
- Sample: `./generate_mds_token.sh client_id client_secret`
8699
- This will generate the *Access Token* and the *Refresh token*
87100

88101
### Creating a bucket on Cloud Storage
89102
This bucket will hold the deployed code for this solution. To create it, navigate to the *Storage* link on the top-left menu on GCP and click on *Create bucket*. You can use Regional location and Standard data type for this bucket.
90103

91-
## Running Megalista
104+
## Running MDS
92105

93106
We recommend first running it locally and make sure that everything works.
94107
Make some sample tables on BigQuery for one of the uploaders and make sure that the data is getting correctly to the destination.
95108
After that is done, upload the Dataflow template to GCP and try running it manually via the UI to make sure it works.
96-
Lastly, configure the Cloud Scheduler to run Megalista in the frequency desired and you'll have a fully functional data integration pipeline.
109+
Lastly, configure the Cloud Scheduler to run MDS in the frequency desired and you'll have a fully functional data integration pipeline.
97110

98111
### Running locally
99112
```bash
100-
python3 megalist_dataflow/main.py \
113+
python3 mds_dataflow/main.py \
101114
--runner DirectRunner \
102115
--developer_token ${GOOGLE_ADS_DEVELOPER_TOKEN} \
103116
--setup_sheet_id ${CONFIGURATION_SHEET_ID} \
@@ -119,15 +132,15 @@ To execute the pipeline, use the following steps:
119132
- Go to **Dataflow** on GCP console
120133
- Click on *Create job from template*
121134
- On the template selection dropdown, select *Custom template*
122-
- Find the *megalist* file on the bucket you've created, on the templates folder
135+
- Find the *mds* file on the bucket you've created, on the templates folder
123136
- Fill in the parameters required and execute
124137

125138
### Scheduling pipeline
126139
To schedule daily/hourly runs, go to **Cloud Scheduler**:
127140
- Click on *create job*
128141
- Add a name and frequency as desired
129142
- For *target* set as HTTP
130-
- Configure a *POST* for url: https://dataflow.googleapis.com/v1b3/projects/${YOUR_PROJECT_ID}/locations/${LOCATION}/templates:launch?gcsPath=gs://${BUCKET_NAME}/templates/megalist, replacing the params with the actual values
143+
- Configure a *POST* for url: https://dataflow.googleapis.com/v1b3/projects/${YOUR_PROJECT_ID}/locations/${LOCATION}/templates:launch?gcsPath=gs://${BUCKET_NAME}/templates/mds, replacing the params with the actual values
131144
- For a sample on the *body* of the request, check **cloud_config/scheduler.json**
132145
- Add OAuth Headers
133146
- Scope: https://www.googleapis.com/auth/cloud-platform
@@ -142,4 +155,18 @@ It's recommended to create a new Service Account to be used with the Cloud Sched
142155

143156

144157
## Usage
145-
Every upload method expects as source a BigQuery data with specific fields, in addition to specific configuration metadata. For details on how to setup your upload routines, refer to the [Megalista Wiki](https://github.com/google/megalista/wiki) or the [Megalista user guide](https://github.com/google/megalista/blob/main/documentation/Megalista%20-%20Technical%20User%20Guide%20-%20EXTERNAL.pdf).
158+
Every upload method expects as source a BigQuery data with specific fields, in addition to specific configuration metadata. For details on how to setup your upload routines, refer to the [MDS Wiki](https://github.com/dp6/marketing-data-sync/wiki) or the [MDS user guide](https://github.com/dp6/marketing-data-sync/blob/main/documentation/mds%20-%20Technical%20User%20Guide%20-%20EXTERNAL.pdf).
159+
160+
### Mandatory requirements
161+
162+
Only contributions that meet the following requirements will be accepted:
163+
164+
- [Commit pattern](https://www.conventionalcommits.org/en/v1.0.0/)
165+
166+
## Support:
167+
168+
**DP6 Koopa-troopa Team**
169+
170+
_e-mail: <[email protected]>_
171+
172+
<img src="https://raw.githubusercontent.com/DP6/templates-centro-de-inovacoes/main/public/images/koopa.png" height="100" />

0 commit comments

Comments
 (0)