Skip to content

Commit a248883

Browse files
gedankenstueckegedankenstuecke
committed
start support 4 download files into container
1 parent f34fae8 commit a248883

File tree

2 files changed

+111
-7
lines changed

2 files changed

+111
-7
lines changed

‎Dockerfile‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,4 +66,6 @@ RUN mkdir -p ${HOME}/output
6666
RUN mkdir -p ${HOME}/pgdata
6767
WORKDIR ${HOME}
6868

69+
RUN git clone https://github.com/geofabrik/sendfile_osm_oauth_protector
70+
6971
ENTRYPOINT ["./entrypoint-new.sh"]

‎make-images.ipynb‎

Lines changed: 109 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -23,11 +23,113 @@
2323
"\n",
2424
"**The right file**: When you've picked your region, you will likely see two similarly named files listed: `regionname-internal.osh.pbf` and `regionname-latest-internal.osm.pbf`. The file you will want to download is the `regionname-internal.osh.pbf` (i.e. the one that includes `osh` and **no `latest`**).\n",
2525
"\n",
26-
"Once you have downloaded your file onto your computer, you need to upload it into your container:\n",
26+
"Once you have downloaded your file onto your computer, you need to upload it into your container. There are a few options that you can consider, depending on how you are using this notebook:\n",
27+
"\n",
28+
"#### Uploading a file you already have downloaded onto your computer\n",
2729
"\n",
2830
"* If you've launched this notebook in _MyBinder_, you can then open up the filebrowser on the top of the left hand side (The _file_ icon, or by pressing CTRL+SHIFT+F). \n",
29-
"* If you've launched this notebook locally, you're likely in the _notebook_ interface, switch back to the _tree_ view and upload your file there (or put it into the shared folder which is accessible by both the container and your normal operating system). \n",
31+
"* If you've launched this notebook locally, you're likely in the _notebook_ interface, switch back to the _tree_ view and upload your file there (or put it into the shared folder which is accessible by both the container and your normal operating system).\n",
32+
"\n",
33+
"#### Downloading a file directly from Geofabrik \n",
34+
"\n",
35+
"This option is particularly useful if you use this notebook on MyBinder, as it allows you to directly download the OSM history file into your Binder, without having to first download it onto your computer and then re-uploading it to the Binder, speeding things up considerably. As the OSM history files are only accessible with an OSM account, you will unfortunately have to fill out the form below, including your OSM username & password. \n",
36+
"\n",
37+
"**Note:** Your username & password will only be used to get a cookie that allows downloading the files and won't be permanently stored (and are not accessible to anyone else). If you are using _MyBinder_, your virtual machine will automatically shut down after more than 10 minutes of inactivity, destroying all data within the virtual machine in the process. If you are running this in a Docker container on your own computer the container will also destroy itself after shutting it down. \n",
38+
"\n",
39+
"If you decide to use the direct download from Geofabrik, enter the necessary details here: "
40+
]
41+
},
42+
{
43+
"cell_type": "code",
44+
"execution_count": 13,
45+
"id": "7d8cfd4c-913e-4983-92ef-c40567421331",
46+
"metadata": {},
47+
"outputs": [
48+
{
49+
"data": {
50+
"application/vnd.jupyter.widget-view+json": {
51+
"model_id": "03b1bf2d3a5447b3823c3b4007441b86",
52+
"version_major": 2,
53+
"version_minor": 0
54+
},
55+
"text/plain": [
56+
"VBox(children=(Text(value='', description='OSM username:', placeholder='Steve'), Text(value='', description='O…"
57+
]
58+
},
59+
"execution_count": 13,
60+
"metadata": {},
61+
"output_type": "execute_result"
62+
}
63+
],
64+
"source": [
65+
"import ipywidgets as widgets\n",
66+
"\n",
67+
"osm_user = widgets.Text(\n",
68+
" placeholder='Steve',\n",
69+
" description='OSM username:',\n",
70+
" disabled=False \n",
71+
")\n",
72+
"\n",
73+
"osm_password = widgets.Text(\n",
74+
" placeholder='your password',\n",
75+
" description='OSM password:',\n",
76+
" disabled=False \n",
77+
")\n",
3078
"\n",
79+
"file_url = widgets.Text(\n",
80+
" placeholder='https://osm-internal.download.geofabrik.de/south-america/argentina-internal.osh.pbf',\n",
81+
" description='Link to OSM history file:',\n",
82+
" disabled=False \n",
83+
")\n",
84+
"widgets.VBox([\n",
85+
" osm_user,\n",
86+
" osm_password,\n",
87+
" file_url\n",
88+
" ])"
89+
]
90+
},
91+
{
92+
"cell_type": "markdown",
93+
"id": "15b753ea-4489-4620-9c19-e46e59f0afeb",
94+
"metadata": {},
95+
"source": [
96+
"Once you have added the details above, you can download your file by running the cell below this one:"
97+
]
98+
},
99+
{
100+
"cell_type": "code",
101+
"execution_count": 16,
102+
"id": "582dbd2b-bd06-4dcf-934a-112c1de79fbf",
103+
"metadata": {},
104+
"outputs": [
105+
{
106+
"name": "stdout",
107+
"output_type": "stream",
108+
"text": [
109+
"--2024-12-20 21:19:10-- https://osm-internal.download.geofabrik.de/south-america/uruguay-internal.osh.pbf\n",
110+
"Resolving osm-internal.download.geofabrik.de (osm-internal.download.geofabrik.de)... 65.109.22.172, 2a01:4f9:5a:1426::2\n",
111+
"connected. to osm-internal.download.geofabrik.de (osm-internal.download.geofabrik.de)|65.109.22.172|:443... \n",
112+
"HTTP request sent, awaiting response... 200 OK\n",
113+
"Length: 87939811 (84M) [application/octet-stream]\n",
114+
"Saving to: ‘uruguay-internal.osh.pbf’\n",
115+
"\n",
116+
"uruguay-internal.os 100%[===================>] 83.87M 190KB/s in 3m 3s \n",
117+
"\n",
118+
"2024-12-20 21:22:15 (468 KB/s) - ‘uruguay-internal.osh.pbf’ saved [87939811/87939811]\n",
119+
"\n"
120+
]
121+
}
122+
],
123+
"source": [
124+
"!python sendfile_osm_oauth_protector/oauth_cookie_client.py -u \"{osm_user.value}\" -p \"{osm_password.value}\" -c \"https://osm-internal.download.geofabrik.de/get_cookie\" -f netscape -o geofabrik_cookie.txt;wget --max-redirect 0 --load-cookies geofabrik_cookie.txt {file_url.value}"
125+
]
126+
},
127+
{
128+
"cell_type": "markdown",
129+
"id": "bfe0b5b8-cebd-4975-8447-f2189459dbcb",
130+
"metadata": {},
131+
"source": [
132+
"Once the file is completely downloaded (or if you uploaded the file using one of the other methods), you can continue with the next step below. \n",
31133
"\n",
32134
"## Starting to make the comparison images. \n",
33135
"\n",
@@ -38,22 +140,22 @@
38140
},
39141
{
40142
"cell_type": "code",
41-
"execution_count": 3,
143+
"execution_count": 17,
42144
"id": "cf6cc023-ea39-46eb-885e-86a184dc1850",
43145
"metadata": {},
44146
"outputs": [
45147
{
46148
"data": {
47149
"application/vnd.jupyter.widget-view+json": {
48-
"model_id": "405e31685af24471838cd6587a52f032",
150+
"model_id": "9637ee17e1b040d5857c2d36553015f8",
49151
"version_major": 2,
50152
"version_minor": 0
51153
},
52154
"text/plain": [
53-
"VBox(children=(Dropdown(description='File:', options=('external/argentina-internal.osh.pbf', 'external/argenti…"
155+
"VBox(children=(Dropdown(description='File:', options=('testfile.osh.pbf', 'uruguay-internal.osh.pbf', 'argenti…"
54156
]
55157
},
56-
"execution_count": 3,
158+
"execution_count": 17,
57159
"metadata": {},
58160
"output_type": "execute_result"
59161
}
@@ -63,7 +165,7 @@
63165
"import glob\n",
64166
"\n",
65167
"filename = widgets.Dropdown(\n",
66-
" options=glob.glob(\"*/*.pbf\"),\n",
168+
" options=glob.glob(\"**/*.pbf\",recursive=True),\n",
67169
" description='File:',\n",
68170
" disabled=False,\n",
69171
")\n",

0 commit comments

Comments
 (0)