Skip to content

Commit faa5a22

Browse files
committed
Adds Running Megalista section into README.md
Change-Id: I4f5d57ee42cd46f2e1024ebf6080c1c5c4a98e59
1 parent 9660942 commit faa5a22

File tree

1 file changed

+22
-0
lines changed

1 file changed

+22
-0
lines changed

README.md

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -88,6 +88,28 @@ In order to create it, follow these steps:
8888
### Creating a bucket on Cloud Storage
8989
This bucket will hold the deployed code for this solution. To create it, navigate to the *Storage* link on the top-left menu on GCP and click on *Create bucket*. You can use Regional location and Standard data type for this bucket.
9090

91+
## Running Megalista
92+
93+
We recommend first running it locally and make sure that everything works.
94+
Make some sample tables on BigQuery for one of the uploaders and make sure that the data is getting correctly to the destination.
95+
After that is done, upload the Dataflow template to GCP and try running it manually via the UI to make sure it works.
96+
Lastly, configure the Cloud Scheduler to run Megalista in the frequency desired and you'll have a fully functional data integration pipeline.
97+
98+
### Running locally
99+
```bash
100+
python3 megalist_dataflow/main.py \
101+
--runner DirectRunner \
102+
--developer_token ${GOOGLE_ADS_DEVELOPER_TOKEN} \
103+
--setup_sheet_id ${CONFIGURATION_SHEET_ID} \
104+
--refresh_token ${REFRESH_TOKEN} \
105+
--access_token ${ACCESS_TOKEN} \
106+
--client_id ${CLIENT_ID} \
107+
--client_secret ${CLIENT_SECRET} \
108+
--project ${GCP_PROJECT_ID} \
109+
--region us-central1 \
110+
--temp_location gs://{$GCS_BUCKET}/tmp
111+
```
112+
91113
### Deploying Pipeline
92114
To deploy, use the following command:
93115
`./deploy_cloud.sh project_id bucket_name region_name`

0 commit comments

Comments
 (0)