|
59 | 59 | "if os.name == 'nt':\n", |
60 | 60 | " import osgeo\n", |
61 | 61 | "\n", |
62 | | - "from importlib import reload\n", |
63 | | - "\n", |
64 | 62 | "# DL stuff\n", |
65 | 63 | "import matplotlib.pyplot as plt\n", |
66 | 64 | "import torch\n", |
|
170 | 168 | "source": [ |
171 | 169 | "## Model setup\n", |
172 | 170 | "\n", |
173 | | - "Here we instantiate the autoencoder class and verify that the output shape is what we expect. Autoencoders compress input data to a latent space and then reconstruct the input from the compressed representation. Usually, we are interested in the compressed result, but during training we are trying to recreate the input exactly. Therefore, the output tensor should match the shape of the input tensor.\n", |
174 | | - "\n", |
175 | | - "Input tensor:\n", |
176 | | - "\n", |
177 | | - "<svg width=\"177\" height=\"167\" style=\"stroke:rgb(0,0,0);stroke-width:1\"> <!-- Horizontal lines -->\n", |
178 | | - " <line x1=\"10\" y1=\"0\" x2=\"27\" y2=\"17\" style=\"stroke-width:2\" />\n", |
179 | | - " <line x1=\"10\" y1=\"100\" x2=\"27\" y2=\"117\" style=\"stroke-width:2\" /> <!-- Vertical lines -->\n", |
180 | | - " <line x1=\"10\" y1=\"0\" x2=\"10\" y2=\"100\" style=\"stroke-width:2\" />\n", |
181 | | - " <line x1=\"27\" y1=\"17\" x2=\"27\" y2=\"117\" style=\"stroke-width:2\" /> <!-- Colored Rectangle -->\n", |
182 | | - " <polygon points=\"10.0,0.0 27.578209484906253,17.578209484906253 27.578209484906253,117.57820948490625 10.0,100.0\"\n", |
183 | | - " style=\"fill:#ECB172A0;stroke-width:0\" /> <!-- Horizontal lines -->\n", |
184 | | - " <line x1=\"10\" y1=\"0\" x2=\"110\" y2=\"0\" style=\"stroke-width:2\" />\n", |
185 | | - " <line x1=\"27\" y1=\"17\" x2=\"127\" y2=\"17\" style=\"stroke-width:2\" /> <!-- Vertical lines -->\n", |
186 | | - " <line x1=\"10\" y1=\"0\" x2=\"27\" y2=\"17\" style=\"stroke-width:2\" />\n", |
187 | | - " <line x1=\"110\" y1=\"0\" x2=\"127\" y2=\"17\" style=\"stroke-width:2\" /> <!-- Colored Rectangle -->\n", |
188 | | - " <polygon points=\"10.0,0.0 110.0,0.0 127.57820948490625,17.578209484906253 27.578209484906253,17.578209484906253\"\n", |
189 | | - " style=\"fill:#ECB172A0;stroke-width:0\" /> <!-- Horizontal lines -->\n", |
190 | | - " <line x1=\"27\" y1=\"17\" x2=\"127\" y2=\"17\" style=\"stroke-width:2\" />\n", |
191 | | - " <line x1=\"27\" y1=\"117\" x2=\"127\" y2=\"117\" style=\"stroke-width:2\" /> <!-- Vertical lines -->\n", |
192 | | - " <line x1=\"27\" y1=\"17\" x2=\"27\" y2=\"117\" style=\"stroke-width:2\" />\n", |
193 | | - " <line x1=\"127\" y1=\"17\" x2=\"127\" y2=\"117\" style=\"stroke-width:2\" /> <!-- Colored Rectangle -->\n", |
194 | | - " <polygon\n", |
195 | | - " points=\"27.578209484906253,17.578209484906253 127.57820948490625,17.578209484906253 127.57820948490625,117.57820948490625 27.578209484906253,117.57820948490625\"\n", |
196 | | - " style=\"fill:#ECB172A0;stroke-width:0\" /> <!-- Text --> <text x=\"77.578209\" y=\"137.578209\" font-size=\"1.0rem\"\n", |
197 | | - " font-weight=\"100\" text-anchor=\"middle\">x: 32</text> <text x=\"147.578209\" y=\"67.578209\" font-size=\"1.0rem\"\n", |
198 | | - " font-weight=\"100\" text-anchor=\"middle\" transform=\"rotate(0,147.578209,67.578209)\">y: 32</text> <text x=\"8.789105\"\n", |
199 | | - " y=\"128.789105\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\"\n", |
200 | | - " transform=\"rotate(45,8.789105,128.789105)\">band: 1</text>\n", |
201 | | - "</svg>\n", |
202 | | - "\n", |
203 | | - "Output tensor:\n", |
204 | | - "\n", |
205 | | - "<svg width=\"177\" height=\"167\" style=\"stroke:rgb(0,0,0);stroke-width:1\"> <!-- Horizontal lines -->\n", |
206 | | - " <line x1=\"10\" y1=\"0\" x2=\"27\" y2=\"17\" style=\"stroke-width:2\" />\n", |
207 | | - " <line x1=\"10\" y1=\"100\" x2=\"27\" y2=\"117\" style=\"stroke-width:2\" /> <!-- Vertical lines -->\n", |
208 | | - " <line x1=\"10\" y1=\"0\" x2=\"10\" y2=\"100\" style=\"stroke-width:2\" />\n", |
209 | | - " <line x1=\"27\" y1=\"17\" x2=\"27\" y2=\"117\" style=\"stroke-width:2\" /> <!-- Colored Rectangle -->\n", |
210 | | - " <polygon points=\"10.0,0.0 27.578209484906253,17.578209484906253 27.578209484906253,117.57820948490625 10.0,100.0\"\n", |
211 | | - " style=\"fill:#ECB172A0;stroke-width:0\" /> <!-- Horizontal lines -->\n", |
212 | | - " <line x1=\"10\" y1=\"0\" x2=\"110\" y2=\"0\" style=\"stroke-width:2\" />\n", |
213 | | - " <line x1=\"27\" y1=\"17\" x2=\"127\" y2=\"17\" style=\"stroke-width:2\" /> <!-- Vertical lines -->\n", |
214 | | - " <line x1=\"10\" y1=\"0\" x2=\"27\" y2=\"17\" style=\"stroke-width:2\" />\n", |
215 | | - " <line x1=\"110\" y1=\"0\" x2=\"127\" y2=\"17\" style=\"stroke-width:2\" /> <!-- Colored Rectangle -->\n", |
216 | | - " <polygon points=\"10.0,0.0 110.0,0.0 127.57820948490625,17.578209484906253 27.578209484906253,17.578209484906253\"\n", |
217 | | - " style=\"fill:#ECB172A0;stroke-width:0\" /> <!-- Horizontal lines -->\n", |
218 | | - " <line x1=\"27\" y1=\"17\" x2=\"127\" y2=\"17\" style=\"stroke-width:2\" />\n", |
219 | | - " <line x1=\"27\" y1=\"117\" x2=\"127\" y2=\"117\" style=\"stroke-width:2\" /> <!-- Vertical lines -->\n", |
220 | | - " <line x1=\"27\" y1=\"17\" x2=\"27\" y2=\"117\" style=\"stroke-width:2\" />\n", |
221 | | - " <line x1=\"127\" y1=\"17\" x2=\"127\" y2=\"117\" style=\"stroke-width:2\" /> <!-- Colored Rectangle -->\n", |
222 | | - " <polygon\n", |
223 | | - " points=\"27.578209484906253,17.578209484906253 127.57820948490625,17.578209484906253 127.57820948490625,117.57820948490625 27.578209484906253,117.57820948490625\"\n", |
224 | | - " style=\"fill:#ECB172A0;stroke-width:0\" /> <!-- Text --> <text x=\"77.578209\" y=\"137.578209\" font-size=\"1.0rem\"\n", |
225 | | - " font-weight=\"100\" text-anchor=\"middle\">x: 32</text> <text x=\"147.578209\" y=\"67.578209\" font-size=\"1.0rem\"\n", |
226 | | - " font-weight=\"100\" text-anchor=\"middle\" transform=\"rotate(0,147.578209,67.578209)\">y: 32</text> <text x=\"8.789105\"\n", |
227 | | - " y=\"128.789105\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\"\n", |
228 | | - " transform=\"rotate(45,8.789105,128.789105)\">band: 1</text>\n", |
229 | | - "</svg>" |
230 | | - ] |
231 | | - }, |
232 | | - { |
233 | | - "cell_type": "code", |
234 | | - "execution_count": null, |
235 | | - "metadata": {}, |
236 | | - "outputs": [], |
237 | | - "source": [ |
238 | | - "import dask.array.svg as svg\n", |
239 | | - "from IPython.display import HTML \n", |
240 | | - "\n", |
241 | | - "svg.svg(((50,), (10,), (10,)), size=100).replace(\"\\n\", \"\")" |
| 171 | + "Here we instantiate the autoencoder class and verify that the output shape is what we expect. Autoencoders compress input data to a latent space and then reconstruct the input from the compressed representation. Usually, we are interested in the compressed result, but during training we are trying to recreate the input exactly. Therefore, the output tensor should match the shape of the input tensor.\n" |
242 | 172 | ] |
243 | 173 | }, |
244 | 174 | { |
|
497 | 427 | "cell_type": "markdown", |
498 | 428 | "metadata": {}, |
499 | 429 | "source": [ |
500 | | - "To be clear, we started with the usual x/y/band tensor from the input dataset.\n", |
501 | | - "\n", |
502 | | - "<svg width=\"177\" height=\"167\" style=\"stroke:rgb(0,0,0);stroke-width:1\"> <!-- Horizontal lines -->\n", |
503 | | - " <line x1=\"10\" y1=\"0\" x2=\"27\" y2=\"17\" style=\"stroke-width:2\" />\n", |
504 | | - " <line x1=\"10\" y1=\"100\" x2=\"27\" y2=\"117\" style=\"stroke-width:2\" /> <!-- Vertical lines -->\n", |
505 | | - " <line x1=\"10\" y1=\"0\" x2=\"10\" y2=\"100\" style=\"stroke-width:2\" />\n", |
506 | | - " <line x1=\"27\" y1=\"17\" x2=\"27\" y2=\"117\" style=\"stroke-width:2\" /> <!-- Colored Rectangle -->\n", |
507 | | - " <polygon points=\"10.0,0.0 27.578209484906253,17.578209484906253 27.578209484906253,117.57820948490625 10.0,100.0\"\n", |
508 | | - " style=\"fill:#ECB172A0;stroke-width:0\" /> <!-- Horizontal lines -->\n", |
509 | | - " <line x1=\"10\" y1=\"0\" x2=\"110\" y2=\"0\" style=\"stroke-width:2\" />\n", |
510 | | - " <line x1=\"27\" y1=\"17\" x2=\"127\" y2=\"17\" style=\"stroke-width:2\" /> <!-- Vertical lines -->\n", |
511 | | - " <line x1=\"10\" y1=\"0\" x2=\"27\" y2=\"17\" style=\"stroke-width:2\" />\n", |
512 | | - " <line x1=\"110\" y1=\"0\" x2=\"127\" y2=\"17\" style=\"stroke-width:2\" /> <!-- Colored Rectangle -->\n", |
513 | | - " <polygon points=\"10.0,0.0 110.0,0.0 127.57820948490625,17.578209484906253 27.578209484906253,17.578209484906253\"\n", |
514 | | - " style=\"fill:#ECB172A0;stroke-width:0\" /> <!-- Horizontal lines -->\n", |
515 | | - " <line x1=\"27\" y1=\"17\" x2=\"127\" y2=\"17\" style=\"stroke-width:2\" />\n", |
516 | | - " <line x1=\"27\" y1=\"117\" x2=\"127\" y2=\"117\" style=\"stroke-width:2\" /> <!-- Vertical lines -->\n", |
517 | | - " <line x1=\"27\" y1=\"17\" x2=\"27\" y2=\"117\" style=\"stroke-width:2\" />\n", |
518 | | - " <line x1=\"127\" y1=\"17\" x2=\"127\" y2=\"117\" style=\"stroke-width:2\" /> <!-- Colored Rectangle -->\n", |
519 | | - " <polygon\n", |
520 | | - " points=\"27.578209484906253,17.578209484906253 127.57820948490625,17.578209484906253 127.57820948490625,117.57820948490625 27.578209484906253,117.57820948490625\"\n", |
521 | | - " style=\"fill:#ECB172A0;stroke-width:0\" /> <!-- Text --> <text x=\"77.578209\" y=\"137.578209\" font-size=\"1.0rem\"\n", |
522 | | - " font-weight=\"100\" text-anchor=\"middle\">x: 32</text> <text x=\"147.578209\" y=\"67.578209\" font-size=\"1.0rem\"\n", |
523 | | - " font-weight=\"100\" text-anchor=\"middle\" transform=\"rotate(0,147.578209,67.578209)\">y: 32</text> <text x=\"8.789105\"\n", |
524 | | - " y=\"128.789105\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\"\n", |
525 | | - " transform=\"rotate(45,8.789105,128.789105)\">band: 1</text>\n", |
526 | | - "</svg>\n", |
527 | | - "\n", |
528 | | - "...and end up with a tensor that has singleton x/y dimensions and a new, 64-element channel dimension.\n", |
529 | | - "\n", |
530 | | - "<svg width=\"141\" height=\"131\" style=\"stroke:rgb(0,0,0);stroke-width:1\"> <!-- Horizontal lines -->\n", |
531 | | - " <line x1=\"10\" y1=\"0\" x2=\"68\" y2=\"58\" style=\"stroke-width:2\" />\n", |
532 | | - " <line x1=\"10\" y1=\"23\" x2=\"68\" y2=\"81\" style=\"stroke-width:2\" /> <!-- Vertical lines -->\n", |
533 | | - " <line x1=\"10\" y1=\"0\" x2=\"10\" y2=\"23\" style=\"stroke-width:2\" />\n", |
534 | | - " <line x1=\"68\" y1=\"58\" x2=\"68\" y2=\"81\" style=\"stroke-width:2\" /> <!-- Colored Rectangle -->\n", |
535 | | - " <polygon\n", |
536 | | - " points=\"10.0,0.0 68.82352941176471,58.82352941176471 68.82352941176471,81.8863520766876 10.0,23.062822664922887\"\n", |
537 | | - " style=\"fill:#ECB172A0;stroke-width:0\" /> <!-- Horizontal lines -->\n", |
538 | | - " <line x1=\"10\" y1=\"0\" x2=\"33\" y2=\"0\" style=\"stroke-width:2\" />\n", |
539 | | - " <line x1=\"68\" y1=\"58\" x2=\"91\" y2=\"58\" style=\"stroke-width:2\" /> <!-- Vertical lines -->\n", |
540 | | - " <line x1=\"10\" y1=\"0\" x2=\"68\" y2=\"58\" style=\"stroke-width:2\" />\n", |
541 | | - " <line x1=\"33\" y1=\"0\" x2=\"91\" y2=\"58\" style=\"stroke-width:2\" /> <!-- Colored Rectangle -->\n", |
542 | | - " <polygon\n", |
543 | | - " points=\"10.0,0.0 33.06282266492289,0.0 91.8863520766876,58.82352941176471 68.82352941176471,58.82352941176471\"\n", |
544 | | - " style=\"fill:#ECB172A0;stroke-width:0\" /> <!-- Horizontal lines -->\n", |
545 | | - " <line x1=\"68\" y1=\"58\" x2=\"91\" y2=\"58\" style=\"stroke-width:2\" />\n", |
546 | | - " <line x1=\"68\" y1=\"81\" x2=\"91\" y2=\"81\" style=\"stroke-width:2\" /> <!-- Vertical lines -->\n", |
547 | | - " <line x1=\"68\" y1=\"58\" x2=\"68\" y2=\"81\" style=\"stroke-width:2\" />\n", |
548 | | - " <line x1=\"91\" y1=\"58\" x2=\"91\" y2=\"81\" style=\"stroke-width:2\" /> <!-- Colored Rectangle -->\n", |
549 | | - " <polygon\n", |
550 | | - " points=\"68.82352941176471,58.82352941176471 91.8863520766876,58.82352941176471 91.8863520766876,81.8863520766876 68.82352941176471,81.8863520766876\"\n", |
551 | | - " style=\"fill:#ECB172A0;stroke-width:0\" /> <!-- Text --> <text x=\"80.354941\" y=\"101.886352\" font-size=\"1.0rem\"\n", |
552 | | - " font-weight=\"100\" text-anchor=\"middle\">y: 1</text> <text x=\"111.886352\" y=\"70.354941\" font-size=\"1.0rem\"\n", |
553 | | - " font-weight=\"100\" text-anchor=\"middle\" transform=\"rotate(0,111.886352,70.354941)\">x: 1</text> <text x=\"29.411765\"\n", |
554 | | - " y=\"72.474587\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\"\n", |
555 | | - " transform=\"rotate(45,29.411765,72.474587)\">channel: 64</text>\n", |
556 | | - "</svg>\n", |
| 430 | + "To be clear, we started with the usual x/y/band tensor from the input dataset, and end up with a tensor that has singleton x/y dimensions and a new, 64-element channel dimension.\n", |
557 | 431 | "\n", |
558 | 432 | "We can go through the same process as before to see how we put together the `predict_on_array` call. Both the `x` and `y` dimensions change size and are used by the batch generator, so they go in `resample_dims`. The remaining dimension, `channel`, is a new dimension and goes in the `new_dim` list." |
559 | 433 | ] |
|
687 | 561 | "\n", |
688 | 562 | "grisdale_similarity.plot.imshow()\n", |
689 | 563 | "plt.scatter(grisdale[\"x\"], grisdale[\"y\"], marker=\"*\", c=\"purple\", edgecolor=\"black\", s=200)\n", |
| 564 | + "plt.title(\"Cosine similarity with Grisdale, WA\")\n", |
690 | 565 | "plt.show()" |
691 | 566 | ] |
692 | 567 | }, |
|
0 commit comments