v0.3.1

Full Changelog: v0.3.0...v0.3.1

v0.3.0

Full Changelog: v0.2.1...v0.3.0

v0.2.1

Full Changelog: v0.2.0...v0.2.1

Changes

• enable auto init of metastore schema;
• fix: issue with config overlay where properties were not getting carried over
• enforce java 17

local-data v0.2.0 - Golang CLI

Overview

For better control and error handling, converted bash-based cli to Golang-based cli. Static config templates have been removed in favor of Go structs.

Changes

• Go-based local-data CLI replacing the legacy Bash implementation
• Got rid of auto download of postgres jar, due to security concerns.
• Profile-based configuration with typed, programmatic config generation
• Homebrew distribution support (brew install danieljhkim/tap/local-data)

local-data-platform v0.1.0 — The Hermetic Hadoop & Hive Sandbox for macOS

Release Title: v0.1.0 — The Hermetic Hadoop & Hive Sandbox for macOS

Overview

This is the initial public release of the local-data-platform, a modular CLI toolset designed to manage a single-machine Hadoop (HDFS + YARN), Hive, and Spark environment on macOS. Unlike standard installations that mutate global configuration directories, this platform uses a unique runtime config overlay system to keep your development environment isolated, organized, and safe for your system.

Core Features

• Modular CLI (local-data): A unified entry point to start, stop, and check the status of HDFS, YARN, and Hive services.
• Hermetic Configuration Overlay: Materializes configuration templates into a $BASE_DIR/conf/current directory, ensuring that your Homebrew-managed Hadoop/Hive config files remain untouched.
• Dual Profile Support: * local: Optimized for Spark and Hive using the local filesystem warehouse.
  • hdfs: A full pseudo-distributed stack including NameNode, DataNode, and YARN.
• Intelligent Service Lifecycle:
  • HDFS: Features automatic NameNode formatting on the first run and safe-mode detection.
  • Hive: Seamless integration with a Postgres-backed metastore, including automatic provisioning of the Postgres JDBC driver.
• "Doctor" Diagnostics: A built-in diagnostic tool to verify required dependencies like Java 17 and Homebrew, recommending fixes for version mismatches.
• Convenience CLI Wrappers: Includes -b suffixed wrappers (e.g., hive-b, hdfs-b, pyspark-b) that automatically inject the platform's environment variables and active profile configs.

Prerequisites

• OS: macOS.
• Java: Java 17 (required).
• Package Manager: Homebrew.
• Core Services: Hadoop, Hive, Spark, and Postgres (for the metastore).

Getting Started

1. Permissions: Run make perms to make entrypoints executable.
2. PATH: Use make path to get the export line for your shell profile.
3. Initialize: Run local-data profile init to instantiate default profiles.
4. Set Profile: Activate your environment with local-data profile set hdfs (or local).
5. Start: Launch the full stack with local-data start.