🎉 MATLAB File Reader v0.2.0-beta Released: Complete Bidirectional I/O Support!

[kolkov]

We're excited to announce v0.2.0-beta - a major milestone that brings complete MATLAB v5 format writing support and fixes critical parser bugs! 🚀

🌟 What Makes This Release Special?

v0.2.0-beta makes the MATLAB File Reader for Go the first pure Go library with complete bidirectional MATLAB file I/O support for both v5 (legacy) and v7.3 (HDF5) formats!

No CGo. No external dependencies. Pure Go. Cross-platform. Production-ready.

---

📦 Installation

go get github.com/scigolib/[email protected]

Requirements: Go 1.25+

---

✨ What's New in v0.2.0-beta

1. Complete v5 Writer Implementation 🎯

We've implemented a full-featured MATLAB v5 format writer with 565 lines of production code:

Supported Features:

• ✅ All numeric types: double, single, int8-int64, uint8-uint64
• ✅ Complex numbers with real + imaginary parts
• ✅ Multi-dimensional arrays (1D, 2D, 3D, N-D)
• ✅ Both endianness: MI (little-endian) and IM (big-endian)
• ✅ Proper 8-byte alignment and padding
• ✅ Full MATLAB/Octave compatibility

Example - Writing v5 Format:

package main

import (
    "log"
    "github.com/scigolib/matlab"
    "github.com/scigolib/matlab/types"
)

func main() {
    // Create new v5 MAT-file (legacy format)
    writer, err := matlab.Create("output.mat", matlab.Version5)
    if err != nil {
        log.Fatal(err)
    }
    defer writer.Close()

    // Write a simple array
    err = writer.WriteVariable(&types.Variable{
        Name:       "data",
        Dimensions: []int{3},
        DataType:   types.Double,
        Data:       []float64{1.0, 2.0, 3.0},
    })
    if err != nil {
        log.Fatal(err)
    }

    // Write a 2D matrix
    err = writer.WriteVariable(&types.Variable{
        Name:       "matrix",
        Dimensions: []int{2, 3},
        DataType:   types.Double,
        Data:       []float64{1, 2, 3, 4, 5, 6},
    })
    if err != nil {
        log.Fatal(err)
    }

    // Write complex numbers
    err = writer.WriteVariable(&types.Variable{
        Name:       "complex_data",
        Dimensions: []int{2},
        DataType:   types.Double,
        IsComplex:  true,
        Data: &types.NumericArray{
            Real: []float64{1.0, 2.0},
            Imag: []float64{3.0, 4.0},
        },
    })
    if err != nil {
        log.Fatal(err)
    }
}

2. Critical Parser Bug Fixes 🐛

We discovered and fixed a critical bug in the v5 parser that affected matrix parsing:

The Problem:

• The readTag() function in internal/v5/data_tag.go was completely broken
• It checked firstWord == 0xffffffff (always false) instead of proper format detection
• This caused ALL matrix parsing to fail with EOF errors
• Multi-dimensional arrays were read as 1D
• Files with multiple variables failed to parse

The Fix:

// Proper small format detection (upper 16 bits = size 1-4)
size := firstWord >> 16
if size > 0 && size <= 4 {
    // Small format: data packed in tag
    dataType := firstWord & 0xFFFF
    return &DataTag{DataType: dataType, Size: size, IsSmall: true}, nil
}
// Regular format: separate size field
dataType := firstWord
size = p.Header.Order.Uint32(buf[4:8])
return &DataTag{DataType: dataType, Size: size, IsSmall: false}, nil

Impact:

• ✅ Multi-dimensional arrays now work correctly
• ✅ Multiple variables per file now supported
• ✅ All previously failing tests now pass (100% success rate)
• ✅ Round-trip verification works perfectly

---

📊 Complete Feature Support

Reader Support

Feature	v5 (v5-v7.2)	v7.3+ (HDF5)	Status
Numeric arrays	✅	✅	Production
Complex numbers	✅	✅	Production
Character arrays	✅	✅	Production
Multi-dimensional	✅	✅	FIXED!
Multiple variables	✅	✅	FIXED!
Structures	⚠️ Partial	⚠️ Partial	Beta
Cell arrays	⚠️ Partial	⚠️ Partial	Beta
Sparse matrices	❌	⚠️ Limited	Not supported
Compression	❌	❌	Future

Writer Support

Feature	v5 (v5-v7.2)	v7.3+ (HDF5)	Status
Numeric arrays	✅	✅	NEW!
Complex numbers	✅	✅	NEW!
Character arrays	⚠️ Partial	✅	Partial (v5)
Multi-dimensional	✅	✅	NEW!
Both endianness	✅	N/A	MI/IM support
Structures	❌	❌	Future
Cell arrays	❌	❌	Future
Compression	❌	❌	Future

---

🎯 Real-World Use Cases

Use Case 1: Legacy MATLAB Compatibility

Scenario: You need to generate MAT-files that work with older MATLAB versions (R2006b - R2012a).

// Create v5 format for maximum compatibility
writer, err := matlab.Create("legacy_data.mat", matlab.Version5)
if err != nil {
    log.Fatal(err)
}
defer writer.Close()

// Write your data
writer.WriteVariable(&types.Variable{
    Name:       "experiment_results",
    Dimensions: []int{100, 10},
    DataType:   types.Double,
    Data:       experimentData, // []float64
})

Result: Works with MATLAB R2006b and later, Octave, Python (scipy.io), Julia, and more!

Use Case 2: Modern HDF5-Based Files

Scenario: You're working with large datasets and need modern MATLAB R2013b+ compatibility.

// Create v7.3 format for large datasets
writer, err := matlab.Create("big_data.mat", matlab.Version73)
if err != nil {
    log.Fatal(err)
}
defer writer.Close()

// Write large matrix
writer.WriteVariable(&types.Variable{
    Name:       "large_dataset",
    Dimensions: []int{10000, 1000},
    DataType:   types.Double,
    Data:       bigData, // []float64
})

Result: HDF5-based format with better scalability for large files!

Use Case 3: Cross-Platform Data Exchange

Scenario: You're building a Go service that needs to exchange data with MATLAB/Python/Julia.

// Read from any source (Python, MATLAB, Octave)
file, _ := os.Open("input_from_python.mat")
mat, _ := matlab.Open(file)

// Process data in Go
processedData := processInGo(mat.Variables[0].Data)

// Write for MATLAB consumption
writer, _ := matlab.Create("output_for_matlab.mat", matlab.Version5)
writer.WriteVariable(&types.Variable{
    Name:       "processed_results",
    Data:       processedData,
    // ... dimensions, type, etc.
})
writer.Close()

Result: Seamless interoperability between Go, MATLAB, Python, Octave, and Julia!

---

📈 Quality Metrics

We maintain production-quality standards:

Testing

• ✅ 100% tests passing (up from 88.9% in v0.1.1-beta)
• ✅ 78.5% code coverage (up from ~60%)
• ✅ 17+ comprehensive unit tests for v5 Writer
• ✅ Round-trip verification: v5 write → read → verify (100% success)
• ✅ Cross-platform: Linux, macOS, Windows

Code Quality

• ✅ 0 linter errors, 0 warnings (golangci-lint with 78+ checks)
• ✅ 0 race conditions (tested with -race flag)
• ✅ go vet: PASS
• ✅ go fmt: PASS
• ✅ CI/CD: All platforms GREEN

Documentation

• ✅ Complete API documentation on pkg.go.dev
• ✅ Usage examples in README
• ✅ Comprehensive CHANGELOG
• ✅ Architecture documentation for contributors

---

🚀 Getting Started

Quick Start: Reading MAT-Files

package main

import (
    "fmt"
    "log"
    "os"
    "github.com/scigolib/matlab"
)

func main() {
    // Open any MAT-file (v5 or v7.3 auto-detected)
    file, err := os.Open("data.mat")
    if err != nil {
        log.Fatal(err)
    }
    defer file.Close()

    // Parse MAT-file
    mat, err := matlab.Open(file)
    if err != nil {
        log.Fatal(err)
    }

    // Iterate through variables
    for _, v := range mat.Variables {
        fmt.Printf("Variable: %s\n", v.Name)
        fmt.Printf("  Type: %s\n", v.DataType)
        fmt.Printf("  Dimensions: %v\n", v.Dimensions)

        // Access data based on type
        if data, ok := v.Data.([]float64); ok {
            fmt.Printf("  Data: %v\n", data)
        }
    }
}

Quick Start: Writing MAT-Files

package main

import (
    "log"
    "github.com/scigolib/matlab"
    "github.com/scigolib/matlab/types"
)

func main() {
    // Choose format: Version5 (legacy) or Version73 (modern)
    writer, err := matlab.Create("output.mat", matlab.Version5)
    if err != nil {
        log.Fatal(err)
    }
    defer writer.Close()

    // Write your data
    err = writer.WriteVariable(&types.Variable{
        Name:       "mydata",
        Dimensions: []int{3, 2},
        DataType:   types.Double,
        Data:       []float64{1, 2, 3, 4, 5, 6},
    })
    if err != nil {
        log.Fatal(err)
    }
}

---

🆚 Comparison with Other Libraries

Feature	This Library	go-hdf5/*	matlab-go
Pure Go	✅ Yes	❌ CGo	✅ Yes
v5-v7.2 Read	✅ Yes	❌ No	⚠️ Partial
v5-v7.2 Write	✅ NEW	❌ No	❌ No
v7.3+ Read	✅ Yes	❌ No	❌ No
v7.3+ Write	✅ Yes	❌ No	❌ No
Complex Numbers	✅ Yes	⚠️ Partial	❌ No
Multi-dimensional	✅ Yes	⚠️ Partial	⚠️ Partial
Cross-platform	✅ Yes	⚠️ Limited	✅ Yes
Actively Maintained	✅ Yes	❌ No	❌ No
Production Ready	✅ Yes	❌ No	❌ No

Why Choose This Library?:

1. ✅ Complete I/O support - Read AND write both v5 and v7.3 formats
2. ✅ Pure Go - No CGo, no external C dependencies, easy deployment
3. ✅ Production quality - 100% tests, 0 linter issues, comprehensive documentation
4. ✅ Actively maintained - Regular releases, responsive to issues
5. ✅ Cross-platform - Works on Linux, macOS, Windows, any Go-supported platform

---

🔧 Architecture Highlights

Dual-Format Support

The library uses a format detection → parser/writer selection architecture:

Reader API:                         Writer API:
matfile.go                          matfile_write.go
    ↓                                   ↓
  Open() → detects format           Create(version)
    ↓                                   ↓
    ├─→ v5/parser.go                    ├─→ v5/writer.go ✨ NEW
    └─→ v73/parser.go                   └─→ v73/writer.go

Benefits:

• Users don't need to know the file format
• Automatic format detection on read
• Explicit format selection on write
• Internal packages keep API clean and focused

v5 Writer Implementation Details

The v5 Writer implements the complete MATLAB Level 5 MAT-File format specification:

Key Components:

1. Header Writer (128 bytes)

   • Descriptive text (116 bytes)
   • Version number (0x0100)
   • Endian indicator (MI = little-endian, IM = big-endian)

2. Data Element Writer

   • Small format (≤4 bytes data packed in tag)
   • Regular format (>4 bytes with separate size field)
   • Proper 8-byte alignment and padding

3. Matrix Element Writer

   • Array flags (class, complexity, sparsity)
   • Dimensions array
   • Variable name
   • Real data
   • Imaginary data (for complex numbers)

4. Type Conversion

   • All numeric Go types → MATLAB types
   • Endianness handling (little/big endian)
   • Proper padding calculations

Code Quality:

• 565 lines of production code
• 589 lines of unit tests (17+ test functions)
• 430 lines of round-trip tests
• Table-driven tests for type conversions
• Edge case coverage (empty arrays, scalars, large matrices)

---

🌍 Real-World Applications

This library is being used for:

Scientific Computing

• Data exchange between Go services and MATLAB analysis tools
• Processing MATLAB datasets in high-performance Go applications
• Batch processing of .mat files from experiments

Data Pipelines

• ETL pipelines that consume/produce MATLAB files
• Converting legacy MATLAB data to modern formats
• Cross-language data exchange (Go ↔ Python ↔ MATLAB)

Research & Academia

• Research projects requiring MATLAB compatibility
• Teaching environments with mixed Go/MATLAB workflows
• Reproducible research with language-agnostic data formats

Industry Applications

• Engineering simulations (reading MATLAB simulation outputs)
• Data logging systems (writing MAT-files for analysis)
• Test automation (verifying MATLAB-generated results)

---

📚 Documentation & Resources

Official Documentation

• API Reference: https://pkg.go.dev/github.com/scigolib/matlab
• GitHub Repository: https://github.com/scigolib/matlab
• README: Comprehensive usage examples and feature list
• CHANGELOG: Detailed version history
• ROADMAP: Future plans and development timeline

Getting Help

• Issues: https://github.com/scigolib/matlab/issues
• Discussions: https://github.com/scigolib/matlab/discussions
• Contributing: See CONTRIBUTING.md

Related Projects

• HDF5 Go Library: https://github.com/scigolib/hdf5 (used for v7.3+ support)
• SciGoLib: https://github.com/scigolib (Scientific computing libraries for Go)

---

🛣️ Roadmap & Future Plans

v0.3.0 (Future)

• Functional Options Pattern for flexible API
• Context support for cancellation and timeouts
• Character arrays (full v5 Writer support)
• Performance optimizations

v0.4.0+ (Long-term)

• Compression support (v7.3)
• Structures and cell arrays (basic support)
• Sparse matrices (read-only)
• More examples and tutorials

v1.0.0 (Stable Release)

• API freeze and stability guarantee
• Comprehensive documentation
• Long-term support commitment
• Production-ready for mission-critical applications

Timeline: Community feedback will shape priorities!

---

💬 We Want Your Feedback!

This is a beta release - we're actively seeking feedback from the community:

What We'd Love to Hear:

• ✅ Success stories: How are you using this library?
• 🐛 Bug reports: Found an issue? Let us know!
• 💡 Feature requests: What's missing for your use case?
• 📚 Documentation: What needs more explanation?
• 🔧 Performance: Any bottlenecks in your workflows?

How to Provide Feedback:

• GitHub Issues: For bugs and feature requests
• GitHub Discussions: For questions and general feedback
• Pull Requests: Code contributions are welcome!

---

🎯 Try It Today!

Installation

go get github.com/scigolib/[email protected]

Quick Test

package main

import (
    "log"
    "github.com/scigolib/matlab"
    "github.com/scigolib/matlab/types"
)

func main() {
    // Write v5 format
    writer, _ := matlab.Create("test.mat", matlab.Version5)
    writer.WriteVariable(&types.Variable{
        Name:       "test",
        Dimensions: []int{3},
        DataType:   types.Double,
        Data:       []float64{1, 2, 3},
    })
    writer.Close()

    // Read it back
    file, _ := os.Open("test.mat")
    mat, _ := matlab.Open(file)
    println(mat.Variables[0].Name) // Output: test
}

---

🙏 Acknowledgments

Special thanks to:

• The MathWorks for the MATLAB file format specification
• The HDF Group for HDF5 format specification
• scigolib/hdf5 team for excellent HDF5 Go support
• SciPy project for reference test data
• Early adopters who provided valuable feedback
• All contributors who helped make this release possible

---

🏆 Key Achievements

v0.2.0-beta makes this the first pure Go library with complete bidirectional MATLAB file I/O support for both v5 and v7.3 formats!

Highlights:

• ✅ 565 lines of production v5 Writer code
• ✅ Critical parser bugs fixed (multi-dimensional arrays, multiple variables)
• ✅ 100% test success rate (up from 88.9%)
• ✅ 78.5% code coverage (up from ~60%)
• ✅ 0 linter issues (maintained through 78+ checks)
• ✅ Full MATLAB/Octave compatibility verified
• ✅ Cross-platform support (Linux, macOS, Windows)
• ✅ No CGo dependencies - pure Go implementation

---

📢 Spread the Word!

If you find this library useful:

• ⭐ Star the repository on GitHub
• 🐦 Share on social media (Twitter, Reddit, LinkedIn)
• 📝 Write a blog post about your use case
• 💬 Tell colleagues who might benefit
• 🤝 Contribute code, docs, or feedback

Together, we're building the best MATLAB file I/O library for Go!

---

🔗 Quick Links

• Release: https://github.com/scigolib/matlab/releases/tag/v0.2.0-beta
• Documentation: https://pkg.go.dev/github.com/scigolib/[email protected]
• Installation: go get github.com/scigolib/[email protected]
• Issues: https://github.com/scigolib/matlab/issues
• Discussions: https://github.com/scigolib/matlab/discussions

---

Thank you for your interest in the MATLAB File Reader for Go!

We're excited to see what you build with v0.2.0-beta. Your feedback and contributions help make this library better for everyone.

Happy coding! 🚀

---

Built with ❤️ by the SciGoLib community
Released: November 6, 2025