Skip to content

Commit 6e71ef5

Browse files
simon-moosimon-moo
authored
Merge pull request #14 from uber/henry.wu/pretty_athenareader
Prettify athenareader output
2 parents 80e6133 + 3c8f80c commit 6e71ef5

File tree

15 files changed

+798
-127
lines changed

15 files changed

+798
-127
lines changed

‎ChangeLog.txt‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,7 @@
1+
v1.1.7 - Athenareader output style and format added (May 31, 2020)
2+
3+
- prettify athenareader output
4+
- One bug fix (https://github.com/uber/athenadriver/issues/12)
15

26
v1.1.6 - Pseudo commands, bug fix and more document and sample code (May 25, 2020)
37

‎athenareader/README.md‎

Lines changed: 249 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44

55
## Authentication Method
66

7-
To avoid exposing access keys(Access Key ID and Secret Access Key) in command line, `athenareader` use [AWS CLI Config For Authentication](https://github.com/uber/athenadriver#use-aws-cli-config-for-authentication) method. Please make sure your environment variable **`AWS_SDK_LOAD_CONFIG`** is set.
7+
To avoid exposing access keys(Access Key ID and Secret Access Key) in command line, `athenareader` use [AWS CLI Config For Authentication](https://github.com/uber/athenadriver#use-aws-cli-config-for-authentication) method.
88

99
## How to get/build/install `athenareader`
1010

@@ -14,57 +14,282 @@ go get -u github.com/uber/athenadriver/athenareader
1414

1515
## How to use `athenareader`
1616

17-
```
18-
$ athenareader --help
17+
You can use `athenareader -h` or ` athenareader --help` to the the plain text version of help text below.
18+
19+
You can set options in command line or file `athenareader.config`, which is located at your home directory or the same directory with athenareader binary. When there is overlapping in options, command line option will be preferred.
20+
1921
NAME
20-
athenareader - read athena data from command line
22+
23+
athenareader - Query Athena data and display in command line
2124

2225
SYNOPSIS
23-
athenareader [-v] [-b output_bucket] [-d database_name] [-q query_string_or_file] [-r] [-a] [-m]
26+
27+
athenareader [-v] [-b OUTPUT_BUCKET] [-d DATABASE_NAME] [-q QUERY_STRING_OR_FILE] [-r] \
28+
[-a] [-m] [-y STYLE_NAME] [-o OUTPUT_FORMAT]
2429

2530
DESCRIPTION
31+
32+
```
2633
-a Enable admin mode, so database write(create/drop) is allowed at athenadriver level
2734
-b string
28-
Athena resultset output bucket (default "s3://query-results-bucket-henrywu/")
35+
Athena resultset output bucket (default "s3://qr-athena-query-result-prod/Henry/")
2936
-d string
3037
The database you want to query (default "default")
3138
-m Enable moneywise mode to display the query cost as the first line of the output
39+
-o string
40+
Output format(options: table, markdown, csv, html) (default "csv")
3241
-q string
3342
The SQL query string or a file containing SQL string (default "select 1")
3443
-r Display rows only, don't show the first row as columninfo
3544
-v Print the current version and exit
45+
-y string
46+
Output rendering style (default "default")
47+
```
48+
3649

3750
EXAMPLES
3851

39-
$ athenareader -d sampledb -q "select request_timestamp,elb_name from elb_logs limit 2"
40-
request_timestamp,elb_name
41-
2015-01-03T00:00:00.516940Z,elb_demo_004
42-
2015-01-03T00:00:00.902953Z,elb_demo_004
52+
- A Simple Query with plain style default(csv) output
53+
54+
```
55+
$ athenareader -d sampledb -q "select request_timestamp,elb_name from elb_logs limit 2"
56+
request_timestamp,elb_name
57+
2015-01-03T00:00:00.516940Z,elb_demo_004
58+
2015-01-03T00:00:00.902953Z,elb_demo_004
59+
```
60+
61+
- A Simple Query with stylish `table` output
62+
63+
There are many styles to choose:
64+
65+
StyleDefault, StyleBold, StyleColoredBright, StyleColoredDark,
66+
StyleColoredBlackOnBlueWhite, StyleColoredBlackOnCyanWhite, StyleColoredBlackOnGreenWhite
67+
StyleColoredBlackOnMagentaWhite, StyleColoredBlackOnYellowWhite, StyleColoredBlackOnRedWhite
68+
StyleColoredBlueWhiteOnBlack, StyleColoredCyanWhiteOnBlack, StyleColoredGreenWhiteOnBlack
69+
StyleColoredMagentaWhiteOnBlack, StyleColoredRedWhiteOnBlack, StyleColoredYellowWhiteOnBlack
70+
StyleDouble, StyleLight, StyleRounded
71+
72+
You can choose one with `-y STYLE_NAME`. For example, to use `default` style:
73+
74+
$ athenareader -b s3://henrywutest/ -q 'desc sampledb.elb_logs' -m -o table -y default
75+
query cost: 0.0 USD, scanned data: 0 B, qid: 96daa3d2-db82-4aa6-8c86-0bf05fd772dc
76+
+-------------------------+-----------+---------+
77+
| COL_NAME | DATA_TYPE | COMMENT |
78+
+-------------------------+-----------+---------+
79+
| request_timestamp | string | |
80+
| elb_name | string | |
81+
| request_ip | string | |
82+
| request_port | int | |
83+
| backend_ip | string | |
84+
| backend_port | int | |
85+
| request_processing_time | double | |
86+
| backend_processing_time | double | |
87+
| client_response_time | double | |
88+
| elb_response_code | string | |
89+
| backend_response_code | string | |
90+
| received_bytes | bigint | |
91+
| sent_bytes | bigint | |
92+
| request_verb | string | |
93+
| url | string | |
94+
| protocol | string | |
95+
| user_agent | string | |
96+
| ssl_cipher | string | |
97+
| ssl_protocol | string | |
98+
+-------------------------+-----------+---------+
99+
100+
To use `StyleColoredRedWhiteOnBlack` style:
43101

44-
$ athenareader -d sampledb -q "select request_timestamp,elb_name from elb_logs limit 2" -r
45-
2015-01-05T20:00:01.206255Z,elb_demo_002
46-
2015-01-05T20:00:01.612598Z,elb_demo_008
102+
$ athenareader -b s3://henrywutest/ -q 'desc sampledb.elb_logs' -m -y StyleColoredRedWhiteOnBlack -o table
103+
104+
The output is like:
47105

48-
$ athenareader -d sampledb -b s3://my-athena-query-result -q tools/query.sql
49-
request_timestamp,elb_name
50-
2015-01-06T00:00:00.516940Z,elb_demo_009
106+
![](../resources/style.png)
51107

52108

53-
Add '-m' to enable moneywise mode. The first line will display query cost under moneywise mode.
109+
- A Simple Query with `markdown` output for you to paste the table to where markdown source is needed
110+
111+
```
112+
$ ./athenareader -b s3://henrywutest/ -q 'desc sampledb.elb_logs' -y StyleColoredRedWhiteOnBlack -o markdown
113+
```
114+
115+
| col_name | data_type | comment |
116+
| --- | --- | --- |
117+
| request_timestamp | string | |
118+
| elb_name | string | |
119+
| request_ip | string | |
120+
| request_port | int | |
121+
| backend_ip | string | |
122+
| backend_port | int | |
123+
| request_processing_time | double | |
124+
| backend_processing_time | double | |
125+
| client_response_time | double | |
126+
| elb_response_code | string | |
127+
| backend_response_code | string | |
128+
| received_bytes | bigint | |
129+
| sent_bytes | bigint | |
130+
| request_verb | string | |
131+
| url | string | |
132+
| protocol | string | |
133+
| user_agent | string | |
134+
| ssl_cipher | string | |
135+
| ssl_protocol | string | |
54136

55-
$ athenareader -b s3://athena-query-result -q 'select count(*) as cnt from sampledb.elb_logs' -m
56-
query cost: 0.00184898369752772851 USD
57-
cnt
58-
1356206
59137

138+
- A Simple Query with `html` output, so you can paste the table to where HTML source is needed
139+
140+
```
141+
$ athenareader -b s3://henrywutest/ -q 'desc sampledb.elb_logs' -o html
142+
```
60143

61-
Add '-a' to enable admin mode. Database write is enabled at driver level under admin mode.
144+
<table class="go-pretty-table">
145+
<thead>
146+
<tr>
147+
<th>col_name</th>
148+
<th>data_type</th>
149+
<th>comment</th>
150+
</tr>
151+
</thead>
152+
<tbody>
153+
<tr>
154+
<td>request_timestamp</td>
155+
<td>string</td>
156+
<td>&nbsp;</td>
157+
</tr>
158+
<tr>
159+
<td>elb_name</td>
160+
<td>string</td>
161+
<td>&nbsp;</td>
162+
</tr>
163+
<tr>
164+
<td>request_ip</td>
165+
<td>string</td>
166+
<td>&nbsp;</td>
167+
</tr>
168+
<tr>
169+
<td>request_port</td>
170+
<td>int</td>
171+
<td>&nbsp;</td>
172+
</tr>
173+
<tr>
174+
<td>backend_ip</td>
175+
<td>string</td>
176+
<td>&nbsp;</td>
177+
</tr>
178+
<tr>
179+
<td>backend_port</td>
180+
<td>int</td>
181+
<td>&nbsp;</td>
182+
</tr>
183+
<tr>
184+
<td>request_processing_time</td>
185+
<td>double</td>
186+
<td>&nbsp;</td>
187+
</tr>
188+
<tr>
189+
<td>backend_processing_time</td>
190+
<td>double</td>
191+
<td>&nbsp;</td>
192+
</tr>
193+
<tr>
194+
<td>client_response_time</td>
195+
<td>double</td>
196+
<td>&nbsp;</td>
197+
</tr>
198+
<tr>
199+
<td>elb_response_code</td>
200+
<td>string</td>
201+
<td>&nbsp;</td>
202+
</tr>
203+
<tr>
204+
<td>backend_response_code</td>
205+
<td>string</td>
206+
<td>&nbsp;</td>
207+
</tr>
208+
<tr>
209+
<td>received_bytes</td>
210+
<td>bigint</td>
211+
<td>&nbsp;</td>
212+
</tr>
213+
<tr>
214+
<td>sent_bytes</td>
215+
<td>bigint</td>
216+
<td>&nbsp;</td>
217+
</tr>
218+
<tr>
219+
<td>request_verb</td>
220+
<td>string</td>
221+
<td>&nbsp;</td>
222+
</tr>
223+
<tr>
224+
<td>url</td>
225+
<td>string</td>
226+
<td>&nbsp;</td>
227+
</tr>
228+
<tr>
229+
<td>protocol</td>
230+
<td>string</td>
231+
<td>&nbsp;</td>
232+
</tr>
233+
<tr>
234+
<td>user_agent</td>
235+
<td>string</td>
236+
<td>&nbsp;</td>
237+
</tr>
238+
<tr>
239+
<td>ssl_cipher</td>
240+
<td>string</td>
241+
<td>&nbsp;</td>
242+
</tr>
243+
<tr>
244+
<td>ssl_protocol</td>
245+
<td>string</td>
246+
<td>&nbsp;</td>
247+
</tr>
248+
</tbody>
249+
</table>
62250

63-
$ athenareader -b s3://athena-query-result -q 'DROP TABLE IF EXISTS depreacted_table' -a
251+
252+
- Query without header(the first column row)
253+
254+
```
255+
$ athenareader -d sampledb -q "select request_timestamp,elb_name from elb_logs limit 2" -r
256+
2015-01-05T20:00:01.206255Z,elb_demo_002
257+
2015-01-05T20:00:01.612598Z,elb_demo_008
258+
```
259+
260+
- Put complex query in a local file
261+
262+
```
263+
$ athenareader -d sampledb -b s3://my-athena-query-result -q tools/query.sql
264+
request_timestamp,elb_name
265+
2015-01-06T00:00:00.516940Z,elb_demo_009
266+
```
267+
268+
- Add `-m` to enable `moneywise mode`. The first line will display query cost under moneywise mode.
269+
270+
```
271+
$ athenareader -b s3://athena-query-result -q 'select count(*) as cnt from sampledb.elb_logs' -m
272+
query cost: 0.00184898369752772851 USD
273+
cnt
274+
1356206
275+
```
276+
277+
- Add `-a` to enable `admin mode`. Database write is enabled at driver level under admin mode.
278+
279+
```
280+
$ athenareader -b s3://athena-query-result -q 'DROP TABLE IF EXISTS depreacted_table' -a
281+
282+
$ athenareader -b s3://henrywutest/ -q 'DROP TABLE IF EXISTS depreacted_table' -o html
283+
writing to Athena database is disallowed in read-only mode
284+
```
285+
64286

65287
AUTHOR
288+
66289
Henry Fuheng Wu (henry.wu@uber.com)
67290

68291
REPORTING BUGS
292+
69293
https://github.com/uber/athenadriver
70-
```
294+
295+

‎athenareader/athenareader.config‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
athenareader:
2+
output:
3+
# options are table, markdown, csv, html
4+
render: csv
5+
# page size is the number of lines per page, default 1024
6+
pagesize: 1024
7+
# options are StyleDefault, StyleBold, StyleColoredBright, StyleColoredDark,
8+
# StyleColoredBlackOnBlueWhite, StyleColoredBlackOnCyanWhite, StyleColoredBlackOnGreenWhite
9+
# StyleColoredBlackOnMagentaWhite, StyleColoredBlackOnYellowWhite, StyleColoredBlackOnRedWhite
10+
# StyleColoredBlueWhiteOnBlack, StyleColoredCyanWhiteOnBlack, StyleColoredGreenWhiteOnBlack
11+
# StyleColoredMagentaWhiteOnBlack, StyleColoredRedWhiteOnBlack, StyleColoredYellowWhiteOnBlack
12+
# StyleDouble, StyleLight, StyleRounded
13+
style: StyleColoredYellowWhiteOnBlack
14+
rowonly: false
15+
moneywise: false
16+
17+
input:
18+
bucket: "s3://henrywutest/"
19+
region: "us-east-1"
20+
database: sampledb
21+
admin: false

0 commit comments

Comments
 (0)