Skip to content

Add manual mapping for non-species label types during migration #1911

@tphakala

Description

@tphakala

Summary

During V2 schema migration, ResolveBatch in labels/resolver.go currently forces all labels to use speciesLabelTypeID. However, BirdNET can produce non-species labels that should be properly categorized.

Current State (PR #1912)

The V2 normalized schema implementation currently:

  • Initializes all labels as "species" type - This is a deliberate simplification for the initial implementation
  • Seeds lookup tables during schema initialization - label_types (species, noise, environment, device) and taxonomic_classes (Aves, Chiroptera) are seeded in manager.go
  • Self-initializes lookup IDs - Both v2only.Datastore and migration Worker automatically look up required IDs from the database

Problem

The ParseRawLabel function in labels/parser.go can identify labels as different types:

  • species - bird species
  • noise - background noise
  • environment - environmental sounds
  • device - device-related sounds

Currently, all labels are saved with label_type_id pointing to "species" regardless of what ParseRawLabel identifies.

Proposed Solution

Add manual mapping for known non-species classes so we can populate the label_type_id correctly during migration:

  1. Create a mapping table or constants for known non-species BirdNET outputs
  2. Update ResolveBatch to use the parsed label type when creating labels
  3. Ensure BatchGetOrCreate can handle mixed label types or group by label type

Files Affected

  • internal/datastore/v2/labels/resolver.go - ResolveBatch function
  • internal/datastore/v2/labels/parser.go - ParseRawLabel function
  • internal/datastore/v2/repository/label_impl.go - BatchGetOrCreate function

Static Maps Already in Place

The following are already defined in internal/datastore/v2/entities/:

Label Types (label_type.go):

func DefaultLabelTypes() []LabelType {
    return []LabelType{
        {Name: "species"},
        {Name: "noise"},
        {Name: "environment"},
        {Name: "device"},
    }
}

Taxonomic Classes (taxonomic_class.go):

func DefaultTaxonomicClasses() []TaxonomicClass {
    return []TaxonomicClass{
        {Name: "Aves"},       // Birds
        {Name: "Chiroptera"}, // Bats
    }
}

Context

Found during code review of feature/v2-normalized-schema branch (PR #1912).

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions