Skip to content

Use timelapse template for image data field validation#2

Merged
ateucher merged 39 commits intomainfrom
Use-timelapse-tdb
Mar 18, 2026
Merged

Use timelapse template for image data field validation#2
ateucher merged 39 commits intomainfrom
Use-timelapse-tdb

Conversation

@ateucher
Copy link
Collaborator

@ateucher ateucher commented Feb 14, 2024

This is to increase the flexibility of importing different csvs. It will allow:

  • Allow extra columns that are not in a template (will just be carried along for the ride)
  • Specifying a template via a .tdb file from TimeLapse
  • Specifying a template by choosing from a list of pre-packaged templates

This will now allow importing data from the wallow/migration timelapse templates.

This reads in migration and wallow data structured with different approaches (with/without a static image accompanying the start of a video episode). It imports, standardizes names and column types, and allows for summary() and the start of qa_image_data() on these new types of datasets

@ateucher ateucher changed the title Add names to tl template Use timelapse template for image data field validation Feb 14, 2024
@ateucher ateucher marked this pull request as ready for review March 17, 2026 23:07
@ateucher ateucher requested a review from Copilot March 17, 2026 23:08
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates read_image_data() to derive CSV column specifications from TimeLapse .tdb templates (bundled or user-supplied), enabling more flexible CSV imports (extra columns, multiple template types including migration/wallow) and aligning downstream summaries/QA with the new schema variability.

Changes:

  • Add .tdb parsing and mapping to readr column specs; support selecting templates via filename detection, interactive menu, or explicit template= path.
  • Update image-data import/cleaning to better handle template-specific columns and to add total_count_episode when missing.
  • Refresh documentation and snapshots; bundle additional TimeLapse templates in inst/extdata.

Reviewed changes

Copilot reviewed 18 out of 32 changed files in this pull request and generated 9 comments.

Show a summary per file
File Description
tests/testthat/test-image-data.R Adds tests for extra columns, unnamed empty columns, and template selection paths.
tests/testthat/_snaps/summary.md Updates summary snapshot outputs reflecting starred counts.
tests/testthat/_snaps/sessions.md Updates snapshots for changed column types (e.g., integer vs double).
tests/testthat/_snaps/rai.md Updates RAI snapshots for changed total_count type.
tests/testthat/_snaps/image-data.md Updates snapshots for new warnings/errors and type changes.
man/read_image_data.Rd Documents new template argument and expanded behavior.
inst/scratch-templates.R Adds an example script for trying migration/wallow imports.
inst/extdata/timelapse-templates/TimelapseTemplate_Ungulate_General_v2.tdb Bundles a new TimeLapse template DB.
inst/extdata/timelapse-templates/TimelapseTemplate_Elk_Migration_v1.tdb Bundles a new TimeLapse template DB.
inst/extdata/timelapse-templates/RISC_WCR_MasterTemplateFieldPicklist_20250109.tdb Bundles master picklist template DB used for type overrides.
inst/extdata/timelapse-templates/RISC_WCR_ImageLabelling_Template_v20230518.2.tdb Bundles a new/updated labeling template DB.
inst/extdata/timelapse-templates/RISC_WCR_ImageLabelling_Template_v20230518.1.tdb Bundles a labeling template DB.
data-raw/field_info.R Removes the prior approach for generating template field info from a .tdb.
data-raw/data-raw.R Stops sourcing/saving the removed tl_template_* dataset.
README.md Updates user docs/examples for selecting templates and reflects type changes.
README.Rmd Same as README.md but in source form.
R/tdb-to-colspec.R Introduces .tdb parsing and mapping to readr::cols().
R/sysdata.rda Updates internal package data (binary).
R/summary.R Makes summary more robust to missing columns across templates.
R/qa-images.R Adjusts QA column dropping logic and count validation to handle missing cols.
R/image-data.R Integrates template selection, .tdb colspecs, extra-column handling, and new cleaning steps.
R/classes.R Stores template name as an attribute on image_data.
DESCRIPTION Bumps minimum R version and moves DBI/RSQLite/jsonlite to Imports.
.gitignore Adds ignores for .claude and Quarto artifacts.
.Rbuildignore Excludes .claude from package builds.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@ateucher ateucher merged commit 4390946 into main Mar 18, 2026
10 checks passed
@ateucher ateucher deleted the Use-timelapse-tdb branch March 18, 2026 00:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants