Skip to content

Commit 69890f7

Browse files
johnnygrecojohnnygreco
committed
updates
1 parent eab4d67 commit 69890f7

20 files changed

+856
-1162
lines changed

‎nemo/NeMo-Data-Designer/managed-service-tutorials/getting-started/1-the-basics.ipynb‎

Lines changed: 10 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -6,11 +6,7 @@
66
"source": [
77
"# 🎨 NeMo Data Designer 101: The Basics\n",
88
"\n",
9-
"> ⚠️ **Warning**: NeMo Data Designer is current in Early Release and is not recommended for production use.\n",
10-
"\n",
11-
"<br>\n",
12-
"\n",
13-
"In this notebook, we will demonstrate the basics of Data Designer by generating a simple product review dataset."
9+
"In this notebook, we will demonstrate the basics of Data Designer by generating a simple product review dataset.\n"
1410
]
1511
},
1612
{
@@ -24,7 +20,7 @@
2420
},
2521
{
2622
"cell_type": "code",
27-
"execution_count": 1,
23+
"execution_count": null,
2824
"metadata": {},
2925
"outputs": [],
3026
"source": [
@@ -41,7 +37,7 @@
4137
" SubcategorySamplerParams,\n",
4238
" UniformSamplerParams,\n",
4339
" ModelConfig,\n",
44-
" InferenceParameters\n",
40+
" InferenceParameters,\n",
4541
")\n"
4642
]
4743
},
@@ -55,9 +51,9 @@
5551
"- In this notebook, we connect to the [managed service of data designer](https://build.nvidia.com/nemo/data-designer). Alternatively, you can connect to your own instance of data designer by following the deployment instructions [here](https://docs.nvidia.com/nemo/microservices/latest/set-up/deploy-as-microservices/data-designer/docker-compose.html).\n",
5652
"- If you have an instance of data designer running locally, you can connect to it as follows\n",
5753
"\n",
58-
" ```python\n",
59-
" data_designer_client = DataDesignerClient(client=NeMoMicroservices(base_url=\"http://localhost:8080\"))\n",
60-
" ```\n"
54+
" ```python\n",
55+
" data_designer_client = DataDesignerClient(client=NeMoMicroservices(base_url=\"http://localhost:8080\"))\n",
56+
" ```\n"
6157
]
6258
},
6359
{
@@ -83,7 +79,7 @@
8379
"source": [
8480
"data_designer_client = NeMoDataDesignerClient(\n",
8581
" base_url=\"https://ai.api.nvidia.com/v1/nemo/dd\",\n",
86-
" default_headers={\"Authorization\": f\"Bearer {api_key}\"} # auto-generated API KEY\n",
82+
" default_headers={\"Authorization\": f\"Bearer {api_key}\"}, # auto-generated API KEY\n",
8783
")\n"
8884
]
8985
},
@@ -106,10 +102,10 @@
106102
"cell_type": "markdown",
107103
"metadata": {},
108104
"source": [
109-
"**Note**: \n",
105+
"**Note**:\n",
110106
"The NeMo Data Designer Managed service has models available for you to use as well. You can use these models by referencing the appropriate model_alias for them.\n",
111107
"\n",
112-
"Please visit https://build.nvidia.com/nemo/data-designer to see the full list of models and their model aliases."
108+
"Please visit https://build.nvidia.com/nemo/data-designer to see the full list of models and their model aliases.\n"
113109
]
114110
},
115111
{
@@ -138,7 +134,7 @@
138134
" max_tokens=1024,\n",
139135
" temperature=0.6,\n",
140136
" top_p=0.95,\n",
141-
" )\n",
137+
" ),\n",
142138
" ),\n",
143139
" ]\n",
144140
")\n"

‎nemo/NeMo-Data-Designer/managed-service-tutorials/getting-started/2-structured-outputs-and-jinja-expressions.ipynb‎

Lines changed: 10 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -6,15 +6,13 @@
66
"source": [
77
"# 🎨 NeMo Data Designer 101: Structured Outputs and Jinja Expressions\n",
88
"\n",
9-
"> ⚠️ **Warning**: NeMo Data Designer is current in Early Release and is not recommended for production use.\n",
10-
">\n",
119
"> **Note**: In order to run this notebook, you must have the NeMo Data Designer microservice deployed locally via docker compose. See the [deployment guide](http://docs.nvidia.com/nemo/microservices/latest/set-up/deploy-as-microservices/data-designer/docker-compose.html) for more details.\n",
1210
"\n",
1311
"<br>\n",
1412
"\n",
1513
"In this notebook, we will continue our exploration of Data Designer, demonstrating more advanced data generation using structured outputs and Jinja expressions.\n",
1614
"\n",
17-
"If this is your first time using Data Designer, we recommend starting with the [first notebook](./1-the-basics.ipynb) in this 101 series."
15+
"If this is your first time using Data Designer, we recommend starting with the [first notebook](./1-the-basics.ipynb) in this 101 series.\n"
1816
]
1917
},
2018
{
@@ -57,9 +55,9 @@
5755
"- In this notebook, we connect to the [managed service of data designer](https://build.nvidia.com/nemo/data-designer). Alternatively, you can connect to your own instance of data designer by following the deployment instructions [here](https://docs.nvidia.com/nemo/microservices/latest/set-up/deploy-as-microservices/data-designer/docker-compose.html).\n",
5856
"- If you have an instance of data designer running locally, you can connect to it as follows\n",
5957
"\n",
60-
" ```python\n",
61-
" data_designer_client = DataDesignerClient(client=NeMoMicroservices(base_url=\"http://localhost:8080\"))\n",
62-
" ```\n"
58+
" ```python\n",
59+
" data_designer_client = DataDesignerClient(client=NeMoMicroservices(base_url=\"http://localhost:8080\"))\n",
60+
" ```\n"
6361
]
6462
},
6563
{
@@ -85,7 +83,7 @@
8583
"source": [
8684
"data_designer_client = NeMoDataDesignerClient(\n",
8785
" base_url=\"https://ai.api.nvidia.com/v1/nemo/dd\",\n",
88-
" default_headers={\"Authorization\": f\"Bearer {api_key}\"} # auto-generated API KEY\n",
86+
" default_headers={\"Authorization\": f\"Bearer {api_key}\"}, # auto-generated API KEY\n",
8987
")\n"
9088
]
9189
},
@@ -108,10 +106,10 @@
108106
"cell_type": "markdown",
109107
"metadata": {},
110108
"source": [
111-
"**Note**: \n",
109+
"**Note**:\n",
112110
"The NeMo Data Designer Managed service has models available for you to use as well. You can use these models by referencing the appropriate model_alias for them.\n",
113111
"\n",
114-
"Please visit https://build.nvidia.com/nemo/data-designer to see the full list of models and their model aliases."
112+
"Please visit https://build.nvidia.com/nemo/data-designer to see the full list of models and their model aliases.\n"
115113
]
116114
},
117115
{
@@ -282,11 +280,11 @@
282280
" sampler_type=SamplerType.CATEGORY,\n",
283281
" params=CategorySamplerParams(\n",
284282
" values=[\"rambling\", \"brief\", \"detailed\", \"structured with bullet points\"],\n",
285-
" weights=[1, 2, 2, 1]\n",
283+
" weights=[1, 2, 2, 1],\n",
286284
" ),\n",
287285
" conditional_params={\n",
288286
" \"target_age_range == '18-25'\": CategorySamplerParams(values=[\"rambling\"]),\n",
289-
" }\n",
287+
" },\n",
290288
" )\n",
291289
")\n",
292290
"\n",
@@ -402,8 +400,7 @@
402400
"\n",
403401
"- [Seeding synthetic data generation with an external dataset](./3-seeding-with-a-dataset.ipynb)\n",
404402
"\n",
405-
"- [Using Custom Model Configs](./4-custom-model-configs.ipynb)\n",
406-
"\n"
403+
"- [Using Custom Model Configs](./4-custom-model-configs.ipynb)\n"
407404
]
408405
}
409406
],

‎nemo/NeMo-Data-Designer/managed-service-tutorials/getting-started/3-seeding-with-a-dataset.ipynb‎

Lines changed: 9 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -6,15 +6,13 @@
66
"source": [
77
"# 🎨 NeMo Data Designer 101: Seeding Synthetic Data Generation with an External Dataset\n",
88
"\n",
9-
"> ⚠️ **Warning**: NeMo Data Designer is current in Early Release and is not recommended for production use.\n",
10-
">\n",
119
"> **Note**: In order to run this notebook, you must have the NeMo Data Designer microservice deployed locally via docker compose. See the [deployment guide](http://docs.nvidia.com/nemo/microservices/latest/set-up/deploy-as-microservices/data-designer/docker-compose.html) for more details.\n",
1210
"\n",
1311
"<br>\n",
1412
"\n",
1513
"In this notebook, we will demonstrate how to seed synthetic data generation in Data Designer with an external dataset.\n",
1614
"\n",
17-
"If this is your first time using Data Designer, we recommend starting with the [first notebook](./1-the-basics.ipynb) in this 101 series."
15+
"If this is your first time using Data Designer, we recommend starting with the [first notebook](./1-the-basics.ipynb) in this 101 series.\n"
1816
]
1917
},
2018
{
@@ -51,9 +49,9 @@
5149
"- In this notebook, we connect to the [managed service of data designer](https://build.nvidia.com/nemo/data-designer). Alternatively, you can connect to your own instance of data designer by following the deployment instructions [here](https://docs.nvidia.com/nemo/microservices/latest/set-up/deploy-as-microservices/data-designer/docker-compose.html).\n",
5250
"- If you have an instance of data designer running locally, you can connect to it as follows\n",
5351
"\n",
54-
" ```python\n",
55-
" data_designer_client = DataDesignerClient(client=NeMoMicroservices(base_url=\"http://localhost:8080\"))\n",
56-
" ```\n"
52+
" ```python\n",
53+
" data_designer_client = DataDesignerClient(client=NeMoMicroservices(base_url=\"http://localhost:8080\"))\n",
54+
" ```\n"
5755
]
5856
},
5957
{
@@ -79,7 +77,7 @@
7977
"source": [
8078
"data_designer_client = NeMoDataDesignerClient(\n",
8179
" base_url=\"https://ai.api.nvidia.com/v1/nemo/dd\",\n",
82-
" default_headers={\"Authorization\": f\"Bearer {api_key}\"} # auto-generated API KEY\n",
80+
" default_headers={\"Authorization\": f\"Bearer {api_key}\"}, # auto-generated API KEY\n",
8381
")\n"
8482
]
8583
},
@@ -102,10 +100,10 @@
102100
"cell_type": "markdown",
103101
"metadata": {},
104102
"source": [
105-
"**Note**: \n",
103+
"**Note**:\n",
106104
"The NeMo Data Designer Managed service has models available for you to use as well. You can use these models by referencing the appropriate model_alias for them.\n",
107105
"\n",
108-
"Please visit https://build.nvidia.com/nemo/data-designer to see the full list of models and their model aliases."
106+
"Please visit https://build.nvidia.com/nemo/data-designer to see the full list of models and their model aliases.\n"
109107
]
110108
},
111109
{
@@ -138,7 +136,7 @@
138136
"\n",
139137
"- In this dataset, the `input_text` represents the `patient_summary` and the `output_text` represents the `diagnosis`\n",
140138
"\n",
141-
"**Note**: At this time, we only support using a single file as the seed. If you have multiple files you would like to use as seeds, it is recommended you consolidated these into a single file. \n"
139+
"**Note**: At this time, we only support using a single file as the seed. If you have multiple files you would like to use as seeds, it is recommended you consolidated these into a single file.\n"
142140
]
143141
},
144142
{
@@ -155,7 +153,7 @@
155153
"config_builder.with_seed_dataset(\n",
156154
" dataset_reference=SeedDatasetReference(\n",
157155
" dataset=\"gretelai/symptom_to_diagnosis/train.jsonl\",\n",
158-
" datastore_settings={\"endpoint\": \"https://huggingface.co\"}\n",
156+
" datastore_settings={\"endpoint\": \"https://huggingface.co\"},\n",
159157
" ),\n",
160158
" sampling_strategy=\"shuffle\",\n",
161159
")\n"

0 commit comments

Comments
 (0)