New Feature: Food Search 2.0 for Loop!

[taylorpatterson-T1D]

Introducing a new AI-enabled food search capability for Loop. Loop has helped me become a better diabetic and I would love to give back to the community by contributing this new feature for Loop, which I believe will benefit all Loop users.

PROBLEM WE'RE SOLVING: Diabetics often lack nutrition knowledge when estimating carb intake. And we need that knowledge when we’re making dosing decisions, especially when eating out at restaurants - this knowledge is the single most important tool we have to improve our Time in Range and A1C. Loop's Food Search feature uses barcodes and AI analysis to provide accurate nutrition information and advanced diabetes management recommendations.

NEW FEATURE'S IMPACT: The Food Search system is a comprehensive food analysis and nutrition tracing solution integrated into Loop for improved diabetes management. It provides multiple search options including text, barcode scanning, and AI-powered image analysis that produces comprehensive nutrition data and dosing suggestions via diabetic notes. With Food Search, we can enter a meal bolus with MUCH greater accuracy which should improve Time in Range and A1C. The AI-analysis engine can even read menus (in multiple languages) and provide guidance for dosing.

Short video demo: https://youtu.be/t7JxvkGcMfU

Review the docs in “/Loop/Documentation/FoodSearch 2.0 Docs/” for comprehensive documentation for Loop's Food Search functionality, including AI-powered nutrition analysis and advanced diabetes management recommendations. API setup guides are provided so the user can enter their personal AI API keys to enable AI food analysis.

This new PR (Food-Search 2.0) fixes compatibility bugs from the initial commit and adds Advanced AI Analysis including FPU analysis, and Dynamic Dosing recommendations.

Requesting a review from the maintainers? @ps2, @marionbarker, @loudnate

Feature is hidden behind a Feature Flag in Loop > Settings > Enable Food Search:

New Search for Food section on the Add Carb Entry view:

Simple text search:

Barcode scan search for high accuracy of packaged foods:

AI-enabled food image analysis engine in action:

AI-Analysis results with comprehensive analysis details including an item by item breakdown of the meal:

Even more detail about how Portions & Servings were calculated and Diabetic Notes to advise during dosing:

Advanced AI Analysis Mode for those who want to geek out on FPU and Advanced Dosing Recommendations:

Food Search Setup screen. Pick the AI service you prefer, enter your own API keys:

[marionbarker]

Code Review

I made some modifications to Taylor's code, starting at 477ca57, as documented in this branch:

See https://github.com/loopandlearn/Loop/commits/modify_for_pr2329

• 542a304 revert unintended changes to .gitignore file
• 2cce0ef restore xcworkspace and xcshareddata files, this change not related to this PR
• 68a738d Restore auto-signing capability, remove unneeded Libre references
• c5d4eae Restore Main.storyboard from dev

Additional changes will be required, but the functionality can be tested.

• The additions (new files) use print statements instead of the log.default or log.debug
• I think the strings may require modifications before they can be localized, but I'll look into this.

More changes are found in a different branch that is focused on the minimum changes to support adding this as a customization: https://github.com/loopandlearn/Loop/commits/prepare_for_customization_pr2329

• Because automated tests for Loop are broken with the files added to LoopTests
  • I removed the LoopTests added in the PR (82dc481c)
  • I removed those file references in pbjproj file and commented out 2 test functions from LoopTests/ViewModels/BolusEntryViewModelTests.swift (ac3f87d6)
    • testCarbEntryDateAndAbsorptionTimeString
    • testCarbEntryDateAndAbsorptionTimeString2
  • With those changes, Tests can now be run

Test

Test with commit 542a304.

This builds and appears to work. More testing will be needed.

[marionbarker]

Update with new commits

When Taylor updates his branch, then I need to also update my two branches:

In a local copy of Taylor's branch:

git pull
git switch modify_for_pr2329
git merge <local-name-for-taylor's-branch>

git switch prepare_for_customization_pr2329
git merge modify_for_pr2329

This keeps the 3 version in alignment.