|
38 | 38 | "\n", |
39 | 39 | "To start go to https://cloud.google.com/ and click on “Get Started For Free\". This is a two step sign up process where you will need to provide your name, address and a credit card. The starter account is free and it comes with $300 credit that you can use. For this step you will need to provide a Google Account ( i.e. your Gmail account) to sign in.\n", |
40 | 40 | "\n", |
41 | | - "After completing the sign up process you will be redirected to [Google Cloud Platform welcome page](https://console.cloud.google.com/home/dashboard). click on the \"Home\" tab and make a note of your Project ID." |
| 41 | + "After completing the sign up process you will be redirected to [Google Cloud Platform welcome page](https://console.cloud.google.com/home/dashboard). click on the \"Home\" tab and make a note of your Project ID and Project number. (see [Identifying projects](https://cloud.google.com/resource-manager/docs/creating-managing-projects#identifying_projects))" |
42 | 42 | ] |
43 | 43 | }, |
44 | 44 | { |
|
49 | 49 | }, |
50 | 50 | "outputs": [], |
51 | 51 | "source": [ |
52 | | - "GCP_PROJECT_ID = 'YOUR_PROJECT_ID'" |
| 52 | + "GCP_PROJECT_ID = 'YOUR_PROJECT_ID'\n", |
| 53 | + "PROJECT_NUMBER = 'YOUR_PROJECT_NUMBER'" |
53 | 54 | ] |
54 | 55 | }, |
55 | 56 | { |
|
63 | 64 | "\n", |
64 | 65 | "* 2.1. Auth for Kaggle notebooks\n", |
65 | 66 | "* 2.2. Auth for Colab notebook\n", |
66 | | - "* 2.3. Auth for Cloud AI Notebooks - No action needed move to step 3." |
| 67 | + "* 2.3. Auth for Cloud AI Notebooks - Not supported." |
67 | 68 | ] |
68 | 69 | }, |
69 | 70 | { |
|
145 | 146 | "id": "DMqNVFx6-E_F" |
146 | 147 | }, |
147 | 148 | "source": [ |
148 | | - "Use your Billing Account_ID from above and run the following to link your billing account with your project." |
| 149 | + "Use your Billing Account_ID from above and run the following to link your billing account with your project. \n", |
| 150 | + "\n", |
| 151 | + "Note if you use an existing project you may not see an Account_ID, this means you do not have the proper permissions to run the following commands, contact your admin or create a new project." |
149 | 152 | ] |
150 | 153 | }, |
151 | 154 | { |
|
158 | 161 | "outputs": [], |
159 | 162 | "source": [ |
160 | 163 | "BILLING_ACCOUNT_ID = 'YOUR_BILLING_ACCOUNT_ID'\n", |
| 164 | + "\n", |
161 | 165 | "!gcloud beta billing projects link $GCP_PROJECT_ID --billing-account $BILLING_ACCOUNT_ID" |
162 | 166 | ] |
163 | 167 | }, |
|
204 | 208 | "outputs": [], |
205 | 209 | "source": [ |
206 | 210 | "BUCKET_NAME = 'YOUR_BUCKET_NAME'\n", |
| 211 | + "\n", |
207 | 212 | "GCS_BUCKET = f'gs://{BUCKET_NAME}'\n", |
208 | 213 | "!gsutil mb -p $GCP_PROJECT_ID $GCS_BUCKET" |
209 | 214 | ] |
|
216 | 221 | "source": [ |
217 | 222 | "## Create a service account for HP Tuning jobs\n", |
218 | 223 | "This step is required to use HP Tuning on Google Cloud using CloudTuner.\n", |
219 | | - "To [create a service account](https://cloud.google.com/iam/docs/creating-managing-service-accounts#iam-service-accounts-create-gcloud) run the following command and make a note of your service account name." |
| 224 | + "To [create a service account](https://cloud.google.com/iam/docs/creating-managing-service-accounts#iam-service-accounts-create-gcloud) and give it project editor access run the following command and make a note of your service account name." |
220 | 225 | ] |
221 | 226 | }, |
222 | 227 | { |
|
229 | 234 | "outputs": [], |
230 | 235 | "source": [ |
231 | 236 | "SERVICE_ACCOUNT_NAME ='YOUR_SERVICE_ACCOUNT_NAME'\n", |
232 | | - "SERVICE_ACCOUNT_EMAIL = f'{SERVICE_ACCOUNT_NAME}@{GCP_PROJECT_ID}.iam.gserviceaccount.com'\n", |
233 | 237 | "\n", |
234 | | - "!gcloud iam --project $GCP_PROJECT_ID service-accounts create $SERVICE_ACCOUNT_NAME" |
| 238 | + "SERVICE_ACCOUNT_EMAIL = f'{SERVICE_ACCOUNT_NAME}@{GCP_PROJECT_ID}.iam.gserviceaccount.com'\n", |
| 239 | + "!gcloud iam --project $GCP_PROJECT_ID service-accounts create $SERVICE_ACCOUNT_NAME\n", |
| 240 | + "!gcloud projects add-iam-policy-binding $GCP_PROJECT_ID \\\n", |
| 241 | + " --member serviceAccount:$SERVICE_ACCOUNT_EMAIL \\\n", |
| 242 | + " --role=roles/editor" |
235 | 243 | ] |
236 | 244 | }, |
237 | 245 | { |
|
240 | 248 | "id": "a-fNtK6rvGmg" |
241 | 249 | }, |
242 | 250 | "source": [ |
243 | | - "The [`default AI Platform service account`](https://cloud.google.com/ai-platform/training/docs/custom-service-account#default) is identified by an email address with the format `service-PROJECT_NUMBER@cloud-ml.google.com.iam.gserviceaccount.com`. Run the following command to get your PROJECT_NUMBER." |
244 | | - ] |
245 | | - }, |
246 | | - { |
247 | | - "cell_type": "code", |
248 | | - "execution_count": null, |
249 | | - "metadata": { |
250 | | - "id": "4MZGiPZnysMo" |
251 | | - }, |
252 | | - "outputs": [], |
253 | | - "source": [ |
254 | | - "!gcloud projects describe $GCP_PROJECT_ID |grep projectNumber" |
255 | | - ] |
256 | | - }, |
257 | | - { |
258 | | - "cell_type": "markdown", |
259 | | - "metadata": { |
260 | | - "id": "hfS6Erynz9Tx" |
261 | | - }, |
262 | | - "source": [ |
263 | | - "Use the project number above to construct the service account email." |
| 251 | + "The [`default AI Platform service account`](https://cloud.google.com/ai-platform/training/docs/custom-service-account#default) is identified by an email address with the format `service-PROJECT_NUMBER@cloud-ml.google.com.iam.gserviceaccount.com`. Using your Project number from step one, we construct the service account email and grant the [`default AI Platform service account`](https://cloud.google.com/ai-platform/training/docs/custom-service-account#default) admin role (roles/iam.serviceAccountAdmin) on your new service account." |
264 | 252 | ] |
265 | 253 | }, |
266 | 254 | { |
|
271 | 259 | }, |
272 | 260 | "outputs": [], |
273 | 261 | "source": [ |
274 | | - "PROJECT_NUMBER = 'YOUR_PROJECT_NUMBER'\n", |
275 | | - "DEFAULT_AI_PLATFORM_SERVICE_ACCOUNT = f'service-{PROJECT_NUMBER}@cloud-ml.google.com.iam.gserviceaccount.com'" |
276 | | - ] |
277 | | - }, |
278 | | - { |
279 | | - "cell_type": "markdown", |
280 | | - "metadata": { |
281 | | - "id": "ySCk0NIF3lux" |
282 | | - }, |
283 | | - "source": [ |
284 | | - "Grant the [`default AI Platform service account`](https://cloud.google.com/ai-platform/training/docs/custom-service-account#default) admin role (roles/iam.serviceAccountAdmin) on your new service account." |
285 | | - ] |
286 | | - }, |
287 | | - { |
288 | | - "cell_type": "code", |
289 | | - "execution_count": null, |
290 | | - "metadata": { |
291 | | - "id": "l9HL0bYxuzWL" |
292 | | - }, |
293 | | - "outputs": [], |
294 | | - "source": [ |
| 262 | + "DEFAULT_AI_PLATFORM_SERVICE_ACCOUNT = f'service-{PROJECT_NUMBER}@cloud-ml.google.com.iam.gserviceaccount.com'\n", |
| 263 | + "\n", |
295 | 264 | "!gcloud iam --project $GCP_PROJECT_ID service-accounts add-iam-policy-binding \\\n", |
296 | 265 | "--role=roles/iam.serviceAccountAdmin \\\n", |
297 | 266 | "--member=serviceAccount:$DEFAULT_AI_PLATFORM_SERVICE_ACCOUNT \\\n", |
298 | 267 | "$SERVICE_ACCOUNT_EMAIL" |
299 | 268 | ] |
300 | 269 | }, |
301 | 270 | { |
302 | | - "cell_type": "code", |
303 | | - "execution_count": null, |
| 271 | + "cell_type": "markdown", |
304 | 272 | "metadata": { |
305 | | - "id": "g9fJh8RX-E_H", |
306 | | - "trusted": true |
| 273 | + "id": "fqbjpPt_-E_H" |
307 | 274 | }, |
308 | | - "outputs": [], |
309 | 275 | "source": [ |
310 | | - "Finally run the following to allow the service account to impersonate your your users account." |
| 276 | + "## Congratulations !\n", |
| 277 | + "You are now ready to run tensorflow-cloud. Note that these steps only need to be run one time. Once you have your project setup you can reuse the same project and bucket configuration for future runs. For any new notebooks you will need to repeat the step two to add your Google Cloud auth credentials.\n", |
| 278 | + "\n", |
| 279 | + "Make a note of the following values as they are needed to run tensorflow-cloud." |
311 | 280 | ] |
312 | 281 | }, |
313 | 282 | { |
314 | 283 | "cell_type": "code", |
315 | 284 | "execution_count": null, |
316 | 285 | "metadata": { |
317 | | - "id": "UeTwSx75-E_H", |
318 | | - "trusted": true |
| 286 | + "id": "yE1gpL8oUdV8" |
319 | 287 | }, |
320 | 288 | "outputs": [], |
321 | 289 | "source": [ |
322 | | - "!gcloud iam service-accounts --project $GCP_PROJECT_ID add-iam-policy-binding \\\n", |
323 | | - " $SERVICE_ACCOUNT_EMAIL \\\n", |
324 | | - " --member=\"user:[email protected]\" \\\n", |
325 | | - " --role=\"roles/iam.serviceAccountUser\"" |
326 | | - ] |
327 | | - }, |
328 | | - { |
329 | | - "cell_type": "markdown", |
330 | | - "metadata": { |
331 | | - "id": "fqbjpPt_-E_H" |
332 | | - }, |
333 | | - "source": [ |
334 | | - "## Congratulations !\n", |
335 | | - "You are now ready to run tensorflow-cloud. Note that these steps only need to be run one time. Once you have your project setup you can reuse the same project and bucket configuration for future runs. For any new notebooks you will need to repeat the step two to add your Google Cloud auth credentials. " |
| 290 | + "print(f\"Your GCP_PROJECT_ID is: {GCP_PROJECT_ID}\")\n", |
| 291 | + "print(f\"Your SERVICE_ACCOUNT_NAME is: {SERVICE_ACCOUNT_NAME}\")\n", |
| 292 | + "print(f\"Your BUCKET_NAME is: {BUCKET_NAME}\")" |
336 | 293 | ] |
337 | 294 | } |
338 | 295 | ], |
|
345 | 302 | }, |
346 | 303 | "name": "google-cloud-project-setup-instructions.ipynb", |
347 | 304 | "provenance": [ |
| 305 | + { |
| 306 | + "file_id": "/piper/depot/google3/third_party/tensorflow_cloud/examples/google_cloud_project_setup_instructions.ipynb?workspaceId=chavoshi:tfc_examples::citc", |
| 307 | + "timestamp": 1614659451764 |
| 308 | + }, |
348 | 309 | { |
349 | 310 | "file_id": "/piper/depot/google3/third_party/tensorflow_cloud/examples/google_cloud_project_setup_instructions.ipynb?workspaceId=chavoshi:tensorflow_cloud::citc", |
350 | 311 | "timestamp": 1613079100054 |
|
0 commit comments