Skip to content

Commit a465cff

Browse files
badelebadele
committed
feat: update readme
1 parent a7f78fd commit a465cff

File tree

2 files changed

+40
-23
lines changed

2 files changed

+40
-23
lines changed

‎README.md‎

Lines changed: 36 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -20,13 +20,30 @@ Works on :
2020

2121
## Usage
2222

23+
### View on Grafana
24+
25+
**INFO:** While waiting for [DuckDB](https://duckdb.org/) support in Grafana and
26+
to speed up data access from Grafana (especially for accessing views), the data
27+
is exported to the [PostgreSQL](https://www.postgresql.org/) database.
28+
2329
```
24-
just import # Import data to PostgreSQL (used by grafana)
25-
just chart # open the grafana (admin/admin)
26-
just stop # stop the grafana server
30+
just import # Import data to PostgreSQL (used by Grafana)
31+
just chart # Open Grafana (admin/admin)
32+
just stop # Stop the Grafana server
2733
```
2834

29-
**Note:** The grafana account is `admin/admin`
35+
**Note:** The Grafana account credentials are `admin/admin`.
36+
37+
### For developers: Convert data to Parquet format
38+
39+
Before having a dataset viewable in Grafana, you need to download and convert it
40+
to Parquet format. This reduces size and improves performance in DuckDB. Learn
41+
more about [Parquet](https://parquet.apache.org/).
42+
43+
```bash
44+
just download
45+
just update
46+
```
3047

3148
## Scopes reference
3249

@@ -68,20 +85,21 @@ them with geonames city elements
6885

6986
## Todo
7087

71-
| Status | Category | Scope | Description | Sample Report |
72-
| ------ | ------------------ | ----------- | ----------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------- |
73-
| [x] | Geonames | Cities | [Geonames](https://download.geonames.org/export/dump/) | [International Covid-19](international/covid-19/README.md) / [French Covid-19](countries/french/covid-19/README.md) |
74-
| [x] | bike counter | Montpellier | [Montpellier 3M](https://data.montpellier3m.fr/dataset/comptages-velo-et-pieton-issus-des-eco-compteurs/resource/edf3e04f-9409-40fe-be66) | |
75-
| [x] | vigilo | Montpellier | [Vigilo](https://vigilo.city) | |
76-
| [ ] | Covid | Countries | [Johns Hopkins University](https://github.com/CSSEGISandData/COVID-19) | [International Covid-19](international/covid-19/README.md) / [French Covid-19](countries/french/covid-19/README.md) |
77-
| [ ] | NASA | Countries | [Anormal température](https://data.giss.nasa.gov/gistemp/) | |
78-
| [ ] | Population | Cities | [insee estimation](https://www.insee.fr/fr/statistiques/1893198) | |
79-
| [ ] | Population | Cities | [insee](https://www.insee.fr/fr/information/2008354) | |
80-
| [ ] | Population | Countries | [United nation](https://population.un.org/wpp/Download/Standard/Population/) | |
81-
| [ ] | Rental bike | Montpellier | [Montpellier 3M](https://data.montpellier3m.fr/dataset/courses-des-velos-velomagg-de-montpellier-mediterranee-metropole) | |
82-
| [ ] | Weather | Cities | [European Centre for Medium-Range Weather Forecasts](https://confluence.ecmwf.int/display/WEBAPI/Accessing+ECMWF+data+servers+in+batch) | |
83-
| [ ] | Weather | Cities | [European Climate Assessment & Dataset](https://www.ecad.eu/dailydata/predefinedseries.php) | |
84-
| [ ] | universitetetioslo | Countries | [CO2 emissions](https://folk.universitetetioslo.no/roberan/GCB2020.shtml) | |
88+
| Status | Category | Scope | Description |
89+
| ------ | ------------------ | ----------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
90+
| [x] | Geonames | Cities | [Geonames](https://download.geonames.org/export/dump/) |
91+
| [x] | bike counter | Montpellier | [Montpellier 3M](https://data.montpellier3m.fr/dataset/comptages-velo-et-pieton-issus-des-eco-compteurs/resource/edf3e04f-9409-40fe-be66) |
92+
| [x] | vigilo | Montpellier | [Vigilo](https://vigilo.city) |
93+
| [ ] | Enterprise | Countries | [Sirene](https://www.data.gouv.fr/fr/datasets/base-sirene-des-entreprises-et-de-leurs-etablissements-siren-siret/) |
94+
| [ ] | Covid | Countries | [Johns Hopkins University](https://github.com/CSSEGISandData/COVID-19) |
95+
| [ ] | NASA | Countries | [Anormal température](https://data.giss.nasa.gov/gistemp/) |
96+
| [ ] | Population | Cities | [insee estimation](https://www.insee.fr/fr/statistiques/1893198) |
97+
| [ ] | Population | Cities | [insee](https://www.insee.fr/fr/information/2008354) |
98+
| [ ] | Population | Countries | [United nation](https://population.un.org/wpp/Download/Standard/Population/) |
99+
| [ ] | Rental bike | Montpellier | [Montpellier 3M](https://data.montpellier3m.fr/dataset/courses-des-velos-velomagg-de-montpellier-mediterranee-metropole) |
100+
| [ ] | Weather | Cities | [European Centre for Medium-Range Weather Forecasts](https://confluence.ecmwf.int/display/WEBAPI/Accessing+ECMWF+data+servers+in+batch) |
101+
| [ ] | Weather | Cities | [European Climate Assessment & Dataset](https://www.ecad.eu/dailydata/predefinedseries.php) |
102+
| [ ] | universitetetioslo | Countries | [CO2 emissions](https://folk.universitetetioslo.no/roberan/GCB2020.shtml) |
85103

86104
## Project commands
87105

‎justfile‎

Lines changed: 4 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -26,9 +26,7 @@ precommit-install:
2626
@precommit-check:
2727
pre-commit run --all-files
2828

29-
###############################################################################
3029
# Check requirements
31-
###############################################################################
3230
@requirements-check:
3331
command -v curl >/dev/null 2>&1 || (echo "Please install curl" ; exit 1)
3432
command -v xz >/dev/null 2>&1 || (echo "Please install xz-utils" ; exit 1)
@@ -90,6 +88,7 @@ precommit-install:
9088
# Update documentation
9189
@doc-update FAKEFILENAME:
9290
just docker-run 'python3 ./updatedoc.py'
91+
9392
[private]
9493
@perm-grafana:
9594
mkdir -p grafana-storage
@@ -108,7 +107,7 @@ precommit-install:
108107
# Start grafana
109108
@start: perm-grafana
110109
docker compose up -d
111-
echo "go to http://localhost:3000/dashboards"
110+
echo "go to http://localhost:9300/dashboards"
112111

113112
# Stop grafana
114113
@stop: perm-user
@@ -120,11 +119,11 @@ precommit-install:
120119

121120
# Open browser to grafana page
122121
@chart: start
123-
command -v xdg-open > /dev/null && xdg-open http://localhost:3000 || echo "goto to http://localhost:3000/dashboards"
122+
command -v xdg-open > /dev/null && xdg-open http://localhost:9300 || echo "goto to http://localhost:9300/dashboards"
124123

125124
# Open browser to pgadmin page
126125
@pgadmin: start
127-
command -v xdg-open > /dev/null && xdg-open http://localhost:8080 || echo "goto to http://localhost:8080"
126+
command -v xdg-open > /dev/null && xdg-open http://localhost:9080 || echo "goto to http://localhost:9080"
128127

129128
# Inspect parquet file
130129
@parquet-inspect FILE:

0 commit comments

Comments
 (0)