Skip to content

Commit 18c7724

Browse files
HugoberryHugoberry
committed
trim the docs
1 parent e9f08b7 commit 18c7724

File tree

1 file changed

+12
-181
lines changed

1 file changed

+12
-181
lines changed

‎README.md‎

Lines changed: 12 additions & 181 deletions
Original file line numberDiff line numberDiff line change
@@ -111,21 +111,12 @@ SELECT UNNEST(pbix2vpax('Adventure Works DW 2020.pbix').Tables).TableName;
111111
SELECT pbix2vpax('Adventure Works DW 2020.pbix');
112112
```
113113

114-
**Count tables and columns:**
115-
```sql
116-
SELECT
117-
list_count(pbix2vpax('Adventure Works DW 2020.pbix').Tables) as table_count,
118-
list_count(pbix2vpax('Adventure Works DW 2020.pbix').Columns) as column_count;
119-
```
120-
121114
**Analyze table sizes:**
122115
```sql
123116
SELECT
124117
tab.TableName,
125118
tab.RowsCount,
126-
tab.TableSize / 1024.0 / 1024.0 as TableSizeMB,
127-
tab.ColumnsSize / 1024.0 / 1024.0 as ColumnsSizeMB,
128-
tab.RelationshipsSize / 1024.0 / 1024.0 as RelationshipsSizeMB
119+
tab.TableSize / 1024.0 / 1024.0 as TableSizeMB
129120
FROM (SELECT UNNEST(pbix2vpax('Adventure Works DW 2020.pbix').Tables) as tab)
130121
ORDER BY tab.TableSize DESC;
131122
```
@@ -159,47 +150,8 @@ ORDER BY MeasureCount DESC;
159150
**Examine relationships:**
160151
```sql
161152
SELECT
162-
rel.FromTableName,
163-
rel.FromFullColumnName,
164-
rel.ToTableName,
165-
rel.ToFullColumnName,
166-
rel.FromCardinalityType,
167-
rel.ToCardinalityType,
168-
rel.IsActive,
169-
rel.CrossFilteringBehavior,
170-
rel.UsedSize / 1024.0 / 1024.0 as RelationshipSizeMB
171-
FROM (SELECT UNNEST(pbix2vpax('Adventure Works DW 2020.pbix').Relationships) as rel)
172-
ORDER BY rel.UsedSize DESC;
173-
```
174-
175-
**Check encoding efficiency:**
176-
```sql
177-
SELECT
178-
col.TableName,
179-
col.ColumnName,
180-
col.Encoding,
181-
col.ColumnCardinality,
182-
col.TotalSize / 1024.0 / 1024.0 as SizeMB,
183-
CASE
184-
WHEN col.ColumnCardinality > 0
185-
THEN col.TotalSize::DOUBLE / col.ColumnCardinality
186-
ELSE 0
187-
END as BytesPerValue
188-
FROM (SELECT UNNEST(pbix2vpax('Adventure Works DW 2020.pbix').Columns) as col)
189-
WHERE col.ColumnCardinality > 0
190-
ORDER BY BytesPerValue DESC
191-
LIMIT 10;
192-
```
193-
194-
**Analyze hierarchies:**
195-
```sql
196-
SELECT
197-
hier.TableName,
198-
hier.UserHierarchyName,
199-
hier.Levels,
200-
hier.UsedSize / 1024.0 / 1024.0 as HierarchySizeMB
201-
FROM (SELECT UNNEST(pbix2vpax('Adventure Works DW 2020.pbix').UserHierarchies) as hier)
202-
ORDER BY hier.UsedSize DESC;
153+
rel.*
154+
FROM (SELECT UNNEST(pbix2vpax('Adventure Works DW 2020.pbix').Relationships) as rel);
203155
```
204156

205157
**Partition refresh status:**
@@ -210,7 +162,8 @@ SELECT
210162
part.RecordCount,
211163
part.SegmentCount,
212164
part.RefreshedTime,
213-
part.RecordsPerSegment
165+
part.RecordsPerSegment,
166+
paart.QueryDefinition
214167
FROM (SELECT UNNEST(pbix2vpax('Adventure Works DW 2020.pbix').Partitions) as part)
215168
ORDER BY part.RecordCount DESC;
216169
```
@@ -220,12 +173,12 @@ ORDER BY part.RecordCount DESC;
220173
**Model size breakdown:**
221174
```sql
222175
WITH vpax AS (
223-
SELECT pbix2vpax('Adventure Works DW 2020.pbix') as analysis
176+
SELECT UNNEST(pbix2vpax('Adventure Works DW 2020.pbix').Tables) as t
224177
),
225178
table_sizes AS (
226179
SELECT
227-
UNNEST(vpax.analysis.Tables).TableName as TableName,
228-
UNNEST(vpax.analysis.Tables).TableSize as TableSize
180+
t.TableName,
181+
t.TableSize
229182
FROM vpax
230183
)
231184
SELECT
@@ -236,43 +189,6 @@ FROM table_sizes
236189
ORDER BY TableSize DESC;
237190
```
238191

239-
**High cardinality columns analysis:**
240-
```sql
241-
WITH vpax AS (
242-
SELECT pbix2vpax('Adventure Works DW 2020.pbix') as analysis
243-
)
244-
SELECT
245-
col.TableName,
246-
col.ColumnName,
247-
col.ColumnCardinality,
248-
col.TotalSize / 1024.0 / 1024.0 as SizeMB,
249-
col.Encoding,
250-
CASE
251-
WHEN col.ColumnCardinality > 1000000 THEN 'Very High'
252-
WHEN col.ColumnCardinality > 100000 THEN 'High'
253-
WHEN col.ColumnCardinality > 10000 THEN 'Medium'
254-
ELSE 'Low'
255-
END as CardinalityLevel
256-
FROM (SELECT UNNEST(vpax.analysis.Columns) as col FROM vpax)
257-
WHERE col.ColumnCardinality > 0
258-
ORDER BY col.ColumnCardinality DESC;
259-
```
260-
261-
**Relationship network analysis:**
262-
```sql
263-
WITH vpax AS (
264-
SELECT pbix2vpax('Adventure Works DW 2020.pbix') as analysis
265-
)
266-
SELECT
267-
rel.FromTableName,
268-
rel.ToTableName,
269-
COUNT(*) as RelationshipCount,
270-
SUM(rel.UsedSize) / 1024.0 / 1024.0 as TotalRelationshipSizeMB
271-
FROM (SELECT UNNEST(vpax.analysis.Relationships) as rel FROM vpax)
272-
GROUP BY rel.FromTableName, rel.ToTableName
273-
ORDER BY RelationshipCount DESC;
274-
```
275-
276192
**Measure catalog with expressions:**
277193
```sql
278194
SELECT
@@ -287,25 +203,11 @@ WHERE NOT meas.IsHidden
287203
ORDER BY meas.TableName, meas.MeasureName;
288204
```
289205

290-
**Column type distribution:**
291-
```sql
292-
WITH vpax AS (
293-
SELECT pbix2vpax('Adventure Works DW 2020.pbix') as analysis
294-
)
295-
SELECT
296-
col.DataType,
297-
COUNT(*) as ColumnCount,
298-
SUM(col.TotalSize) / 1024.0 / 1024.0 as TotalSizeMB,
299-
AVG(col.ColumnCardinality) as AvgCardinality
300-
FROM (SELECT UNNEST(vpax.analysis.Columns) as col FROM vpax)
301-
GROUP BY col.DataType
302-
ORDER BY ColumnCount DESC;
303-
```
304206

305207
**Export to JSON for external analysis:**
306208
```sql
307209
COPY (
308-
SELECT to_json(pbix2vpax('Adventure Works DW 2020.pbix'))
210+
SELECT pbix2vpax('Adventure Works DW 2020.pbix')
309211
) TO 'model_analysis.json';
310212
```
311213

@@ -348,82 +250,11 @@ SELECT * FROM model_b
348250
ORDER BY ModelName, TableSize DESC;
349251
```
350252

351-
**Find columns that could benefit from optimization:**
352-
```sql
353-
WITH vpax AS (
354-
SELECT pbix2vpax('Adventure Works DW 2020.pbix') as analysis
355-
)
356-
SELECT
357-
col.TableName,
358-
col.ColumnName,
359-
col.Encoding,
360-
col.ColumnCardinality,
361-
col.TotalSize / 1024.0 / 1024.0 as SizeMB,
362-
CASE
363-
WHEN col.Encoding = 'HASH' AND col.ColumnCardinality < 1000
364-
THEN 'Consider VALUE encoding'
365-
WHEN col.ColumnCardinality > 1000000
366-
THEN 'Very high cardinality - review if needed'
367-
ELSE 'OK'
368-
END as Recommendation
369-
FROM (SELECT UNNEST(vpax.analysis.Columns) as col FROM vpax)
370-
WHERE col.TotalSize > 1048576 -- Larger than 1MB
371-
ORDER BY col.TotalSize DESC
372-
LIMIT 20;
373-
```
374-
375-
**Relationship health check:**
376-
```sql
377-
SELECT
378-
rel.FromTableName,
379-
rel.ToTableName,
380-
rel.RelationshipType,
381-
rel.IsActive,
382-
rel.CrossFilteringBehavior,
383-
CASE
384-
WHEN NOT rel.IsActive THEN 'Inactive relationship'
385-
WHEN rel.CrossFilteringBehavior = 'BothDirections' THEN 'Bidirectional - review performance'
386-
WHEN NOT rel.RelyOnReferentialIntegrity THEN 'RI not assumed - may impact performance'
387-
ELSE 'OK'
388-
END as HealthCheck
389-
FROM (SELECT UNNEST(pbix2vpax('Adventure Works DW 2020.pbix').Relationships) as rel)
390-
ORDER BY rel.FromTableName, rel.ToTableName;
391-
```
392-
393-
## Running the tests
394-
Different tests can be created for DuckDB extensions. The primary way of testing DuckDB extensions should be the SQL tests in `./test/sql`. These SQL tests can be run using:
395-
```sh
396-
make test
397-
```
398-
399-
### Installing the deployed binaries
400-
You will need to do two things to install your extension binaries from S3. Firstly, DuckDB should be launched with the
401-
`allow_unsigned_extensions` option set to true. How to set this will depend on the client you're using. Some examples:
253+
## Installing the extension
402254

403-
CLI:
404-
```shell
405-
duckdb -unsigned
406-
```
407-
408-
Python:
409-
```python
410-
con = duckdb.connect(':memory:', config={'allow_unsigned_extensions' : 'true'})
411-
```
412-
413-
NodeJS:
414-
```js
415-
db = new duckdb.Database(':memory:', {"allow_unsigned_extensions": "true"});
416-
```
417-
418-
Secondly, you must set the repository endpoint in DuckDB to the HTTP URL of your bucket + version of the extension
419-
you want to install. To do this, run the following SQL query in DuckDB:
420-
```sql
421-
SET custom_extension_repository='https://duckdb.pbix.info';
422-
```
423-
After running these steps, you can install and load your extension using the regular INSTALL/LOAD commands in DuckDB:
424255
```sql
425-
INSTALL pbix
426-
LOAD pbix
256+
INSTALL pbix FROM community;
257+
LOAD pbix;
427258
```
428259

429260
## Configuration Options

0 commit comments

Comments
 (0)