You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@@ -40,12 +40,7 @@ FROM { 'filename' | PROGRAM 'command' | STDIN | Azure_blob_url}
40
40
41
41
### Arguments
42
42
#### Azure_blob_url
43
-
Allows unstructured data to be stored and accessed at a massive scale in block blobs. Objects in blob storage can be accessed from anywhere in the world via HTTP or HTTPS. The storage client libraries are available for multiple languages, including .NET, Java, Node.js, Python, PHP, and Ruby. User or client applications can access blobs via following methods
44
-
1. URLs
45
-
1. The Azure Storage REST API
46
-
1. Azure PowerShell
47
-
1. Azure CLI
48
-
1. An Azure storage client library
43
+
Allows unstructured data to be stored and accessed at a massive scale in block blobs. Objects in blob storage can be accessed from anywhere in the world via HTTPS. The storage client libraries are available for multiple languages, including .NET, Java, Node.js, Python, PHP, and Ruby.
49
44
50
45
### Option
51
46
#### format
@@ -64,14 +59,14 @@ Function allows adding access to a storage account.
64
59
```postgresql
65
60
azure_storage.account_add
66
61
(account_name_p text
67
-
,account_key_p text);
62
+
,account_key_p text);
68
63
```
69
64
### Arguments
70
65
#### account_name_p
71
-
An Azure blob storage (ABS) account contains all of your ABS objects: blobs, files, queues, and tables. The storage account provides a unique namespace for your ABS that is accessible from anywhere in the world over HTTP or HTTPS.
66
+
An Azure blob storage (ABS) account contains all of your ABS objects: blobs, files, queues, and tables. The storage account provides a unique namespace for your ABS that is accessible from anywhere in the world over HTTPS.
72
67
73
68
#### account_key_p
74
-
Your Azure blob storage (ABS) access keys are similar to a root password for your storage account. Always be careful to protect your access keys. Use Azure Key Vault to manage and rotate your keys securely. The account key is stored in a table that is only accessible by the postgres superuser. To see which storage accounts exist, use the account_list
69
+
Your Azure blob storage (ABS) access keys are similar to a root password for your storage account. Always be careful to protect your access keys. Use Azure Key Vault to manage and rotate your keys securely. The account key is stored in a table that is accessible by the postgres superuser, azure_storage_admin and all roles granted those admin permissions. To see which storage accounts exist, use the function account_list.
75
70
76
71
## azure_storage.account_remove
77
72
Function allows revoking account access to storage account.
@@ -83,7 +78,7 @@ azure_storage.account_remove
83
78
84
79
### Arguments
85
80
#### account_name_p
86
-
Azure blob storage (ABS) account contains all of your ABS objects: blobs, files, queues, and tables. The storage account provides a unique namespace for your ABS that is accessible from anywhere in the world over HTTP or HTTPS.
81
+
Azure blob storage (ABS) account contains all of your ABS objects: blobs, files, queues, and tables. The storage account provides a unique namespace for your ABS that is accessible from anywhere in the world over HTTPS.
87
82
88
83
## azure_storage.account_user_add
89
84
The function allows adding access for a role to a storage account.
@@ -96,7 +91,7 @@ azure_storage.account_add
96
91
97
92
### Arguments
98
93
#### account_name_p
99
-
An Azure blob storage (ABS) account contains all of your ABS objects: blobs, files, queues, and tables. The storage account provides a unique namespace for your ABS that is accessible from anywhere in the world over HTTP or HTTPS.
94
+
An Azure blob storage (ABS) account contains all of your ABS objects: blobs, files, queues, and tables. The storage account provides a unique namespace for your ABS that is accessible from anywhere in the world over HTTPS.
100
95
101
96
#### user_p
102
97
Role created by user visible on the cluster.
@@ -109,13 +104,13 @@ The function allows removing access for a role to a storage account.
109
104
110
105
```postgresql
111
106
azure_storage.account_remove
112
-
(account_name_p text
113
-
,user_p regrole);
107
+
(account_name_p text
108
+
,user_p regrole);
114
109
```
115
110
116
111
### Arguments
117
112
#### account_name_p
118
-
An Azure blob storage (ABS) account contains all of your ABS objects: blobs, files, queues, and tables. The storage account provides a unique namespace for your ABS that is accessible from anywhere in the world over HTTP or HTTPS.
113
+
An Azure blob storage (ABS) account contains all of your ABS objects: blobs, files, queues, and tables. The storage account provides a unique namespace for your ABS that is accessible from anywhere in the world over HTTPS.
119
114
120
115
#### user_p
121
116
Role created by user visible on the cluster.
@@ -125,15 +120,15 @@ The function lists the account & role having access to Azure blob storage.
125
120
126
121
```postgresql
127
122
azure_storage.account_list
128
-
( OUT account_name text
129
-
, OUT allowed_users regrole[]
130
-
)
123
+
(OUT account_name text
124
+
,OUT allowed_users regrole[]
125
+
)
131
126
Returns TABLE;
132
127
```
133
128
134
129
### Arguments
135
130
#### account_name
136
-
Azure blob storage (ABS) account contains all of your ABS objects: blobs, files, queues, and tables. The storage account provides a unique namespace for your ABS that is accessible from anywhere in the world over HTTP or HTTPS.
131
+
Azure blob storage (ABS) account contains all of your ABS objects: blobs, files, queues, and tables. The storage account provides a unique namespace for your ABS that is accessible from anywhere in the world over HTTPS.
137
132
138
133
#### allowed_users
139
134
Lists the users having access to the Azure blob storage.
@@ -146,23 +141,24 @@ The function lists the available blob files within a user container with their p
146
141
147
142
```postgresql
148
143
azure_storage.blob_list
149
-
(account_name text
150
-
, container_name text
151
-
, prefix text DEFAULT ''::text
152
-
, OUT path text
153
-
, OUT bytes bigint
154
-
, OUT last_modified timestamp with time zone
155
-
, OUT etag text
156
-
, OUT content_type text
157
-
, OUT content_encoding text
158
-
, OUT content_hash text
159
-
)
144
+
(account_name text
145
+
,container_name text
146
+
,prefix text DEFAULT ''::text
147
+
,OUT path text
148
+
,OUT bytes bigint
149
+
,OUT last_modified timestamp with time zone
150
+
,OUT etag text
151
+
,OUT content_type text
152
+
,OUT content_encoding text
153
+
,OUT content_hash text
154
+
)
160
155
Returns SETOF record;
161
156
```
162
157
163
158
### Arguments
164
159
#### account_name
165
-
The `storage account name` provides a unique namespace for your Azure storage data that's accessible from anywhere in the world over HTTP or HTTPS.
160
+
The `storage account name` provides a unique namespace for your Azure storage data that's accessible from anywhere in the world over HTTPS.
161
+
166
162
#### container_name
167
163
A container organizes a set of blobs, similar to a directory in a file system. A storage account can include an unlimited number of containers, and a container can store an unlimited number of blobs.
168
164
A container name must be a valid DNS name, as it forms part of the unique URI used to address the container or its blobs. Follow these rules when naming a container:
@@ -172,8 +168,9 @@ A container name must be a valid DNS name, as it forms part of the unique URI us
172
168
* Two or more consecutive dash characters aren't permitted in container names.
returns file from blob container with matching string initials.
179
176
#### path
@@ -204,31 +201,31 @@ The function allows loading the content of file \ files from within the containe
204
201
```postgresql
205
202
azure_storage.blob_get
206
203
(account_name text
207
-
,container_name text
208
-
,path text
209
-
,decoder text DEFAULT 'auto'::text
210
-
,compression text DEFAULT 'auto'::text
211
-
,options jsonb DEFAULT NULL::jsonb
204
+
,container_name text
205
+
,path text
206
+
,decoder text DEFAULT 'auto'::text
207
+
,compression text DEFAULT 'auto'::text
208
+
,options jsonb DEFAULT NULL::jsonb
212
209
)
213
210
RETURNS SETOF record;
214
211
```
215
212
There's an overloaded version of function, containing rec parameter that allows you to conveniently define the output format record.
216
213
```postgresql
217
214
azure_storage.blob_get
218
215
(account_name text
219
-
,container_name text
220
-
,path text
221
-
,rec anyelement
222
-
,decoder text DEFAULT 'auto'::text
223
-
,compression text DEFAULT 'auto'::text
224
-
,options jsonb DEFAULT NULL::jsonb
216
+
,container_name text
217
+
,path text
218
+
,rec anyelement
219
+
,decoder text DEFAULT 'auto'::text
220
+
,compression text DEFAULT 'auto'::text
221
+
,options jsonb DEFAULT NULL::jsonb
225
222
)
226
223
RETURNS SETOF anyelement;
227
224
```
228
225
229
226
### Arguments
230
227
#### account
231
-
The storage account provides a unique namespace for your Azure Storage data that's accessible from anywhere in the world over HTTP or HTTPS.
228
+
The storage account provides a unique namespace for your Azure Storage data that's accessible from anywhere in the world over HTTPS.
232
229
#### container
233
230
A container organizes a set of blobs, similar to a directory in a file system. A storage account can include an unlimited number of containers, and a container can store an unlimited number of blobs.
234
231
A container name must be a valid DNS name, as it forms part of the unique URI used to address the container or its blobs.
@@ -263,15 +260,15 @@ The function acts as a utility function called as a parameter within blob_get, w
263
260
264
261
```postgresql
265
262
azure_storage.options_csv_get
266
-
(delimiter text DEFAULT NULL::text
267
-
, null_string text DEFAULT NULL::text
268
-
, header boolean DEFAULT NULL::boolean
269
-
, quote text DEFAULT NULL::text
270
-
, escape text DEFAULT NULL::text
271
-
, force_not_null text[] DEFAULT NULL::text[]
272
-
, force_null text[] DEFAULT NULL::text[]
273
-
, content_encoding text DEFAULT NULL::text
274
-
)
263
+
(delimiter text DEFAULT NULL::text
264
+
,null_string text DEFAULT NULL::text
265
+
,header boolean DEFAULT NULL::boolean
266
+
,quote text DEFAULT NULL::text
267
+
,escape text DEFAULT NULL::text
268
+
,force_not_null text[] DEFAULT NULL::text[]
269
+
,force_null text[] DEFAULT NULL::text[]
270
+
,content_encoding text DEFAULT NULL::text
271
+
)
275
272
Returns jsonb;
276
273
```
277
274
@@ -308,16 +305,16 @@ The function acts as a utility function called as a parameter within blob_get.
308
305
309
306
```postgresql
310
307
azure_storage.options_copy
311
-
(delimiter text DEFAULT NULL::text
312
-
, null_string text DEFAULT NULL::text
313
-
, header boolean DEFAULT NULL::boolean
314
-
, quote text DEFAULT NULL::text
315
-
, escape text DEFAULT NULL::text
316
-
, force_quote text[] DEFAULT NULL::text[]
317
-
, force_not_null text[] DEFAULT NULL::text[]
318
-
, force_null text[] DEFAULT NULL::text[]
319
-
, content_encoding text DEFAULT NULL::text
320
-
)
308
+
(delimiter text DEFAULT NULL::text
309
+
,null_string text DEFAULT NULL::text
310
+
,header boolean DEFAULT NULL::boolean
311
+
,quote text DEFAULT NULL::text
312
+
,escape text DEFAULT NULL::text
313
+
,force_quote text[] DEFAULT NULL::text[]
314
+
,force_not_null text[] DEFAULT NULL::text[]
315
+
,force_null text[] DEFAULT NULL::text[]
316
+
,content_encoding text DEFAULT NULL::text
317
+
)
321
318
Returns jsonb;
322
319
```
323
320
@@ -357,10 +354,10 @@ The function acts as a utility function called as a parameter within blob_get. I
357
354
358
355
```postgresql
359
356
azure_storage.options_tsv
360
-
(delimiter text DEFAULT NULL::text
361
-
, null_string text DEFAULT NULL::text
362
-
, content_encoding text DEFAULT NULL::text
363
-
)
357
+
(delimiter text DEFAULT NULL::text
358
+
,null_string text DEFAULT NULL::text
359
+
,content_encoding text DEFAULT NULL::text
360
+
)
364
361
Returns jsonb;
365
362
```
366
363
@@ -382,8 +379,7 @@ The function acts as a utility function called as a parameter within blob_get. I
382
379
383
380
```postgresql
384
381
azure_storage.options_binary
385
-
(content_encoding text DEFAULT NULL::text
386
-
)
382
+
(content_encoding text DEFAULT NULL::text)
387
383
Returns jsonb;
388
384
```
389
385
@@ -403,17 +399,17 @@ Now you can list containers set to Private and Blob access levels for that stora
403
399
The examples used make use of sample Azure storage account `(pgquickstart)` with custom files uploaded for adding to coverage of different use cases. We can start by creating table used across the set of example used.
404
400
```sql
405
401
CREATETABLEIF NOT EXISTS public.events
406
-
(
407
-
event_id bigint,
408
-
event_type text,
409
-
event_public boolean,
410
-
repo_id bigint,
411
-
payload jsonb,
412
-
repo jsonb,
413
-
user_id bigint,
414
-
org jsonb,
415
-
created_at timestamp without time zone
416
-
)
402
+
(
403
+
event_id bigint
404
+
,event_type text
405
+
,event_public boolean
406
+
,repo_id bigint
407
+
,payload jsonb
408
+
,repo jsonb
409
+
,user_id bigint
410
+
,org jsonb
411
+
,created_at timestamp without time zone
412
+
);
417
413
```
418
414
419
415
### Adding access key of storage account (mandatory for access level = private)
@@ -423,7 +419,7 @@ The example illustrates adding of access key for the storage account to get acce
In your storage account, open **Access keys**. Copy the **Storage account name** and copy the **Key** from **key1** section (you have to select **Show** next to the key first).
422
+
> In your storage account, open **Access keys**. Copy the **Storage account name** and copy the **Key** from **key1** section (you have to select **Show** next to the key first).
427
423
428
424
:::image type="content" source="media/howto-ingestion/azure-blob-storage-account-key.png" alt-text="Screenshot of Security + networking > Access keys section of an Azure Blob Storage page in the Azure portal." border="true":::
429
425
@@ -455,7 +451,7 @@ SELECT * FROM azure_storage.blob_list('pgquickstart','publiccontainer','e');
@@ -510,7 +506,7 @@ SELECT * FROM azure_storage.blob_get
510
506
,'events-compressed'
511
507
, NULL::events
512
508
, decoder :='csv'
513
-
, compression :='gzip')
509
+
, compression :='gzip')
514
510
LIMIT5;
515
511
```
516
512
@@ -531,18 +527,19 @@ LIMIT 5;
531
527
The example illustrates the use of `options` argument for processing files with headers, custom separators, escape characters, etc., you can either use the COPY command or pass COPY options to the blob_get function using the `azure_storage.options_copy` function.
0 commit comments