radar-hdfs-restructure version 0.5.0

Use a plugin architecture to specify:

• path layout: for binning (how many hours for a file) and organisation (project/user/topic/time.csv or topic/project/user/time.csv, project/user/topic.csv, etc.).
• file format: currently csv or json
• compression method: currently gzip or none
• storage driver: currently local, but could be minio or s3.

This makes the module much more extensible for other needs or projects.

Other updates:

• threaded task model
• deduplication now does not change ordering, and does not use another temporary file
• files are now atomically moved from staging directory if possible
• bins and offsets are written from separate thread, using single Accountant class
• settings and factories are propagated through the application with the FileStoreFactory.

Restructure-HDFS-topic version 0.4.0

Changes since version 0.3.2:

• Added command-line parser
• Add temporary file staging to prevent corrupt writes
• Documented proper start script usage
• Added docker distribution

Restructure-HDFS-topic release 0.3.2

Changes since 0.3.1:

• Add projectId to output file structure, fixes #13
• Continue with other files if JsonMappingException occurs, fixes #14
• Set deduplicate to false by default, fixes #16
• If a CSV mapping issue occur, make a new file, fixes #15

Restructure HDFS topic version 0.3.1

Restructure HDFS topic version 0.3.1

Changes since version 0.3:

• hotfix: gzip files are correctly detected

Changes since version 0.2.1:

• use java.nio.file
• deduplicate files after closing

Restructure-HDFS-topic version 0.3

Restructure-HDFS-topic version 0.3

Changes since version 0.2.1:

• use java.nio.file.Path instead of java.io.File
• deduplicate files after closing

HDFS restructure version 0.2.1

HDFS restructure version 0.2.1.

Changes since version 0.2:

• removed IntelliJ files
• skip empty files (fixes #3)
• output schemas with files (fixes #8)
• rewrite (clean) offsets.csv after processing
• process the timestamps of WindowedKey records
• update to Gradle 4.1

RADAR restructure HDFS topics version 0.2

Restructures the repository created by the HDFS connector in the RADAR-Docker repository.

Changes since version 0.1.1:

• Added GZIP compression option
• Show progress bar indicator
• Split FileCache into FileCacheStore and FileCache

Restructure-HDFS version 0.1.1: bugfixes

Restructures the repository created by the HDFS connector in the RADAR-Docker repository.

Changes since version 0.1:

• Times in output files follow UTC timezone
• Fixed bug when reading a Frequency file

First release to restructure RADAR HDFS connector files

Restructures files created by the RADAR HDFS connector to CSV or JSON files.