feat: Offline AI Plant and Wildlife Identification

[Copilot]

• Add enableWildlifeIdentification feature flag to apps/expo/config.ts
• Create apps/expo/features/wildlife/types.ts with type definitions
• Create apps/expo/features/wildlife/constants.ts — shared constants
• Create feature hooks, components, screens, routes
• Update dashboard and translations
• CodeRabbit fixes (commit 495a88d)
• Copilot reviewer inline comments (this commit):
  • WildlifeIdentificationScreen:135 — accessibilityLabel="wildlife.viewHistory" + accessibilityRole="button" on history icon button
  • WildlifeHistoryScreen:84 — accessibilityLabel="wildlife.clearHistoryAction" + accessibilityRole="button" on clear-history icon button
  • en.json — added identifiedCountSingular key; WildlifeTile now uses it when count === 1
  • EdibilityBadge — hardcoded label strings replaced with t('wildlife.edibilityEdible/Poisonous/Medicinal') i18n keys
  • useWildlifeIdentification.ts — both launchCameraAsync and launchImageLibraryAsync now use ImagePicker.MediaTypeOptions.Images
  • WildlifeIdentificationScreen:120 — error in onError callback narrowed via err instanceof Error ? err.message : String(err)
  • en.json — added clearHistoryAction, viewHistory, edibilityEdible/Poisonous/Medicinal keys

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Feature: Offline AI Plant and Wildlife Identification</issue_title>
<issue_description>## Summary

Implement on-device AI for identifying plants and wildlife from photos, even without internet connectivity.

Why

• 42% of hikers lose signal in trails
• On-device identification provides privacy (data never leaves device)
• Eliminates API costs for image recognition
• Works in remote wilderness areas

Technical Approach

On-Device ML Framework

• Use ExecuTorch or TensorFlow Lite for on-device inference
• Pre-downloaded species database organized by hiking region
• Models cached locally after first download

Features

• Camera integration for photo capture
• Real-time species identification
• Confidence scores for results
• Species information cached locally
• History of identified species

Offline First

• All processing happens on-device
• Results cached for offline access
• Background sync when connectivity returns

Research Reference

Based on Barnacle's research on offline AI models for React Native, including ExecuTorch and TensorFlow Lite approaches.

Deliverable

MVP that demonstrates:

• Camera capture for plant/wildlife photos
• On-device species identification
• Results display with confidence scores
• Offline capability</issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes Feature: Offline AI Plant and Wildlife Identification #1809

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

[andrew-bierman]

@coderabbitai Please review this PR for code quality, best practices, and potential issues.

[coderabbitai]

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# List all files changed in the PR
git diff --name-only HEAD~1 HEAD 2>/dev/null || git diff --name-only origin/main HEAD 2>/dev/null || find . -newer package.json -name "*.ts" -o -newer package.json -name "*.tsx" | head -50

# Also try to find the wildlife feature files
echo "=== Wildlife feature files ==="
find . -path "*/wildlife*" -type f | grep -v node_modules | grep -v ".git"

echo "=== App wildlife routes ==="
find . -path "*app*wildlife*" -type f | grep -v node_modules

Length of output: 2292

---

🏁 Script executed:

#!/bin/bash
echo "=== features/wildlife directory ==="
find . -path "*/features/wildlife*" -type f | grep -v node_modules | sort

echo "=== app wildlife routes ==="
find . -path "*app*wildlife*" -type f | grep -v node_modules | sort

Length of output: 1348

---

🏁 Script executed:

#!/bin/bash
echo "=== types.ts ==="
cat apps/expo/features/wildlife/types.ts

echo ""
echo "=== useWildlifeIdentification.ts ==="
cat apps/expo/features/wildlife/hooks/useWildlifeIdentification.ts

echo ""
echo "=== useWildlifeHistory.ts ==="
cat apps/expo/features/wildlife/hooks/useWildlifeHistory.ts

echo ""
echo "=== WildlifeTile.tsx ==="
cat apps/expo/features/wildlife/components/WildlifeTile.tsx

echo ""
echo "=== WildlifeIdentificationScreen.tsx ==="
cat apps/expo/features/wildlife/screens/WildlifeIdentificationScreen.tsx

echo ""
echo "=== WildlifeHistoryScreen.tsx ==="
cat apps/expo/features/wildlife/screens/WildlifeHistoryScreen.tsx

Length of output: 26976

---

🏁 Script executed:

#!/bin/bash
echo "=== _layout.tsx ==="
cat "apps/expo/app/(app)/wildlife/_layout.tsx"

echo ""
echo "=== wildlife/index.tsx ==="
cat "apps/expo/app/(app)/wildlife/index.tsx"

echo ""
echo "=== wildlife/history.tsx ==="
cat "apps/expo/app/(app)/wildlife/history.tsx"

echo ""
echo "=== config.ts (relevant portion) ==="
cat apps/expo/config.ts

echo ""
echo "=== home/index.tsx ==="
cat "apps/expo/app/(app)/(tabs)/(home)/index.tsx"

echo ""
echo "=== en.json wildlife section ==="
python3 -c "import json; data=json.load(open('apps/expo/lib/i18n/locales/en.json')); print(json.dumps(data.get('wildlife', {}), indent=2))"

Length of output: 13507

---

🏁 Script executed:

#!/bin/bash
echo "=== Check all references to HISTORY_STORAGE_KEY ==="
rg "HISTORY_STORAGE_KEY" --type ts --type tsx -n

echo ""
echo "=== Check identifyAsync usage ==="
rg "identifyAsync" -n

echo ""
echo "=== Check if Stack is configured anywhere else for reference ==="
rg "Stack.Screen" "apps/expo/app/(app)" -n --include="*.tsx" --include="*.ts" | head -20

echo ""
echo "=== Check LargeTitleHeader usage for headerShown pattern ==="
rg "headerShown" "apps/expo/app" -n --include="*.tsx" | head -20

echo ""
echo "=== Check en.json for common.delete and common.cancel keys ==="
python3 -c "import json; data=json.load(open('apps/expo/lib/i18n/locales/en.json')); print(json.dumps(data.get('common', {}), indent=2))"

Length of output: 1916

---

🏁 Script executed:

#!/bin/bash
echo "=== Check all references to HISTORY_STORAGE_KEY ==="
rg "HISTORY_STORAGE_KEY" -n

echo ""
echo "=== Check identifyAsync usage across all files ==="
rg "identifyAsync" -n

echo ""
echo "=== Check Stack.Screen usage in app directory ==="
rg "Stack\.Screen" apps/expo/app -n

echo ""
echo "=== Check headerShown in app directory ==="
rg "headerShown" apps/expo/app -n

echo ""
echo "=== Check other layout files for pattern comparison ==="
find apps/expo/app -name "_layout.tsx" | head -10 | xargs -I{} sh -c 'echo "=== {} ===" && cat "{}"'

Length of output: 26472

---

@andrew-bierman: Here is a thorough review of the PR.

---

Review: Offline AI Plant and Wildlife Identification

Overall the feature is well-structured and the abstraction layer for the ML engine is a good design decision. Here are the issues found, ordered by severity.

---

🔴 Critical

1. wildlife route is unregistered in the parent layout — double header on all screens

Every other feature route under app/(app)/ that uses LargeTitleHeader is explicitly registered in apps/expo/app/(app)/_layout.tsx with headerShown: false. For example:

<Stack.Screen name="weather" options={{ headerShown: false }} />

The wildlife route has no such entry. Without it, Expo Router will render the native Stack header on top of the LargeTitleHeader, producing a double-header on both the identification and history screens. Add this to apps/expo/app/(app)/_layout.tsx:

<Stack.Screen name="wildlife" options={{ headerShown: false }} />

2. JSON.parse without try/catch in persistIdentificationToHistory

In useWildlifeIdentification.ts (Line 150), if AsyncStorage contains malformed data, this throws an unhandled exception and crashes the identification flow:

// ❌ No error handling
const existing: SpeciesIdentification[] = stored ? JSON.parse(stored) : [];

Compare to loadHistory in useWildlifeHistory.ts which correctly wraps the parse in a try/catch and clears corrupted data. Apply the same pattern:

let existing: SpeciesIdentification[] = [];
if (stored) {
  try {
    const parsed = JSON.parse(stored);
    existing = Array.isArray(parsed) ? parsed : [];
  } catch {
    await AsyncStorage.removeItem(HISTORY_STORAGE_KEY);
  }
}

---

🟠 High

3. HISTORY_STORAGE_KEY is duplicated across two files

The same string 'wildlife_identification_history' is hard-coded independently in both useWildlifeIdentification.ts (Line 7) and useWildlifeHistory.ts (Line 5). If one is ever changed, the two hooks silently diverge and data is lost with no runtime error.

Extract to a shared location:

// features/wildlife/constants.ts
export const WILDLIFE_HISTORY_STORAGE_KEY = 'wildlife_identification_history';

Then import it in both hooks.

4. No safety disclaimer for edibility information

The EdibilityBadge can render "✓ Edible" or "✗ Poisonous" based on what is currently mock-random output. Even with a real ML model, species identification confidence scores don't meet the bar for foraging safety decisions. A user acting on a false "Edible" result could be seriously harmed.

A prominent disclaimer should be shown near all edibility badges, e.g.:

<View className="rounded-xl bg-yellow-500/10 px-3 py-2 mb-2">
  <Text variant="caption1" className="text-yellow-700 dark:text-yellow-400">
    ⚠️ {t('wildlife.edibilityDisclaimer')}
  </Text>
</View>

With a corresponding i18n key: "edibilityDisclaimer": "For reference only. Never eat a plant or mushroom based solely on app identification.".

5. Cache invalidation is caller-dependent — a latent coupling bug

persistIdentificationToHistory writes directly to AsyncStorage but doesn't touch the React Query cache. The responsibility to call invalidate() falls entirely on the screen's onSuccess callback in handleIdentify. If identify is ever called from another context without that callback, the history tile and history screen will show stale data silently.

The hook should own this responsibility itself. The cleanest fix is to inject the queryClient into useWildlifeIdentification and call invalidateQueries inside mutationFn after the persist:

import { useQueryClient } from '@tanstack/react-query';
// inside useWildlifeIdentification:
const queryClient = useQueryClient();
// ...after persistIdentificationToHistory(...):
await queryClient.invalidateQueries({ queryKey: ['wildlife', 'history'] });

The HISTORY_QUERY_KEY should then be exported from useWildlifeHistory.ts (or the shared constants file) to avoid yet another duplication.

---

🟡 Medium

6. Translated strings used as React key for tips list

In WildlifeIdentificationScreen.tsx:

{[t('wildlife.tip1'), t('wildlife.tip2'), t('wildlife.tip3')].map((tip) => (
  <View key={tip} ...>

Using translated content as a key is fragile — if two tips share an identical translation in any locale, React will warn about duplicate keys and silently drop one item. Use stable keys:

{(['tip1', 'tip2', 'tip3'] as const).map((key) => (
  <View key={key} className="mb-2 flex-row items-start gap-2">
    ...
    <Text ...>{t(`wildlife.${key}`)}</Text>
  </View>
))}

---

🔵 Low / Cleanup

7. identifyAsync is exported but never consumed

identifyAsync: captureAndIdentifyMutation.mutateAsync is returned from useWildlifeIdentification but there is no usage anywhere in the codebase. Either remove it or leave a comment explaining the intended use case.

8. No TODO comment on the mock inference engine

runOnDeviceInference uses Math.random() and setTimeout. Without an in-code marker, this is a production-incident waiting to happen if the PR is merged and the placeholder is forgotten. Add:

// TODO: Replace with real ExecuTorch/TFLite model call before production release.
// Swap the body of this function; the return type contract must be preserved.
async function runOnDeviceInference(imageUri: string): Promise<IdentificationResult> {

9. Misplaced import in home/index.tsx

The useTranslation import appears approximately halfway through the file after several component declarations. It should be grouped with the other imports at the top of the file.

[andrew-bierman]

@copilot CodeRabbit has reviewed this PR and provided feedback. Please address the review comments and push updates. Once you've made the changes, the monitor will ask CodeRabbit to re-review.

[andrew-bierman]

@copilot

[Copilot]

Pull request overview

Adds a new Expo feature module for fully offline plant/wildlife identification, including capture → on-device (mock) inference → results display, plus an AsyncStorage-backed history and dashboard integration behind a feature flag.

Changes:

• Introduces features/wildlife/ with types, hooks (identify + history), tile, and two screens (identify + history).
• Adds Expo Router routes for /wildlife and /wildlife/history.
• Integrates the feature into the dashboard layout (tile) and adds wildlife i18n strings + feature flag.

Reviewed changes

Copilot reviewed 16 out of 16 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
apps/expo/lib/i18n/locales/en.json	Adds wildlife translation namespace for the new UI.
apps/expo/features/wildlife/types.ts	Defines core types for identifications and history entries.
apps/expo/features/wildlife/screens/index.ts	Barrel export for wildlife screens.
apps/expo/features/wildlife/screens/WildlifeIdentificationScreen.tsx	Capture/identify flow UI + results rendering.
apps/expo/features/wildlife/screens/WildlifeHistoryScreen.tsx	History list UI + clear-all action.
apps/expo/features/wildlife/index.ts	Barrel export for wildlife module (hooks/types).
apps/expo/features/wildlife/hooks/useWildlifeIdentification.ts	Image picking + mock on-device inference + persistence to AsyncStorage.
apps/expo/features/wildlife/hooks/useWildlifeHistory.ts	React Query wrapper around AsyncStorage history with invalidation + clear.
apps/expo/features/wildlife/hooks/index.ts	Barrel export for hooks.
apps/expo/features/wildlife/components/index.ts	Barrel export for wildlife components.
apps/expo/features/wildlife/components/WildlifeTile.tsx	Dashboard tile showing offline capability and count.
apps/expo/config.ts	Adds enableWildlifeIdentification feature flag.
apps/expo/app/(app)/wildlife/index.tsx	Registers /wildlife route.
apps/expo/app/(app)/wildlife/history.tsx	Registers /wildlife/history route.
apps/expo/app/(app)/wildlife/_layout.tsx	Adds Stack layout for wildlife routes.
apps/expo/app/(app)/(tabs)/(home)/index.tsx	Adds wildlife tile metadata and conditional dashboard layout entry.

---

You can also share your feedback on Copilot code review. Take the survey.

[Copilot]

The history icon button is an icon-only Pressable without an accessibilityLabel/accessibilityRole. Please add an explicit label (and role) so screen readers can announce what this button does.

[Copilot]

The clear-history control is an icon-only Pressable without an accessibilityLabel/accessibilityRole. Please add a label like “Clear identification history” so assistive tech can identify the action.

[Copilot]

identifiedCount will read awkwardly for a single identification ("1 species identified"). The repo already uses separate singular keys in places (e.g. catalog.totalItemsSingular), so consider adding a singular form (or adopting the existing pluralization approach) and using it when count === 1.

Suggested change

	"offline": "Offline",
	"offlineCapable": "All processing happens on-device — works without internet",
	"capturePhoto": "Take or choose a photo to identify species",
	"identifiedCountSingular": "{{count}} species identified",
	"offline": "Offline",
	"offlineCapable": "All processing happens on-device — works without internet",

[Copilot]

EdibilityBadge hardcodes user-visible strings (and symbols) like "✓ Edible" / "✗ Poisonous". Since the rest of the screen is localized via t('wildlife.*'), these labels should come from i18n keys as well (and keep the badge logic keyed on EdibilityStatus).

Suggested change

	if (!edibility || edibility === 'unknown') return null;
	const config: Record<string, { bg: string; label: string }> = {
	edible: { bg: 'bg-green-500/20', label: '✓ Edible' },
	poisonous: { bg: 'bg-red-500/20', label: '✗ Poisonous' },
	medicinal: { bg: 'bg-blue-500/20', label: '⚕ Medicinal' },
	};
	const c = config[edibility];
	if (!c) return null;
	return (
	<View className={`rounded-full px-3 py-1 ${c.bg}`}>
	<Text variant="caption1" className="font-medium">
	{c.label}
	const { t } = useTranslation();
	if (!edibility || edibility === 'unknown') return null;
	const config: Record<string, { bg: string; i18nKey: string }> = {
	edible: { bg: 'bg-green-500/20', i18nKey: 'wildlife.edibility.edible' },
	poisonous: { bg: 'bg-red-500/20', i18nKey: 'wildlife.edibility.poisonous' },
	medicinal: { bg: 'bg-blue-500/20', i18nKey: 'wildlife.edibility.medicinal' },
	};
	const c = config[edibility];
	if (!c) return null;
	return (
	<View className={`rounded-full px-3 py-1 ${c.bg}`}>
	<Text variant="caption1" className="font-medium">
	{t(c.i18nKey)}

[Copilot]

mediaTypes is being passed as the string 'images', but elsewhere in the app expo-image-picker is configured with ImagePicker.MediaTypeOptions.Images. Using the enum avoids type/runtime mismatches across SDK versions.

[Copilot]

Same as the camera path: mediaTypes: 'images' should use ImagePicker.MediaTypeOptions.Images to match the rest of the codebase and the expo-image-picker typings.

[Copilot]

persistIdentificationToHistory does an unguarded JSON.parse on the stored value. If AsyncStorage is corrupted, this will throw and break identification; consider wrapping parse in a try/catch (similar to loadHistory) and clearing/resetting the key on failure.

Suggested change

	const existing: SpeciesIdentification[] = stored ? JSON.parse(stored) : [];
	let existing: SpeciesIdentification[] = [];
	if (stored) {
	try {
	const parsed = JSON.parse(stored);
	if (Array.isArray(parsed)) {
	existing = parsed as SpeciesIdentification[];
	} else {
	// Stored value is not in the expected format; clear it.
	await AsyncStorage.removeItem(HISTORY_STORAGE_KEY);
	}
	} catch {
	// Corrupted JSON; clear the key and start fresh.
	await AsyncStorage.removeItem(HISTORY_STORAGE_KEY);
	}
	}

[Copilot]

In React Query, the onError callback receives unknown by default; accessing err.message will fail type-checking (and may not be safe at runtime). Please narrow the error (e.g., err instanceof Error) or stringify it before showing the alert.

Suggested change

	Alert.alert(t('wildlife.identificationFailed'), err.message);
	const message = err instanceof Error ? err.message : String(err);
	Alert.alert(t('wildlife.identificationFailed'), message);

[Copilot]

@copilot CodeRabbit has reviewed this PR and provided feedback. Please address the review comments and push updates. Once you've made the changes, the monitor will ask CodeRabbit to re-review.

All 9 CodeRabbit review items have been addressed in commit 495a88d:

1. Double header — registered wildlife route with headerShown: false in app/(app)/_layout.tsx
2. JSON.parse without try/catch — persistIdentificationToHistory now wraps the parse in try/catch, mirroring the pattern in loadHistory
3. Duplicated HISTORY_STORAGE_KEY — extracted to features/wildlife/constants.ts as WILDLIFE_HISTORY_STORAGE_KEY (and WILDLIFE_HISTORY_QUERY_KEY); both hooks now import from there
4. Edibility safety disclaimer — added a yellow warning banner inside SpeciesResultCard whenever edibility is edible, poisonous, or medicinal, with the wildlife.edibilityDisclaimer i18n key
5. Cache invalidation ownership — moved queryClient.invalidateQueries into the mutationFn itself; the screen no longer needs to call invalidate() on success
6. Translated strings as React keys — tips now use stable 'tip1' | 'tip2' | 'tip3' keys
7. Unused identifyAsync — removed from the returned object
8. TODO on mock inference — added // TODO: Replace with real ExecuTorch/TFLite model call before production release. above runOnDeviceInference
9. Misplaced import — useTranslation import moved to the top-level import block in home/index.tsx

[andrew-bierman]

@copilot Please address the CodeRabbit review feedback on this PR (8 inline comments).

[andrew-bierman]

@coderabbitai review

[coderabbitai]

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[coderabbitai]

Warning

Rate limit exceeded

@andrew-bierman has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 5 minutes and 44 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 04f060a2-2f88-4b0c-92ce-25cc907b5ca9

📥 Commits

Reviewing files that changed from the base of the PR and between 3f94763 and a7d2f27.

📒 Files selected for processing (18)

• apps/expo/app/(app)/(tabs)/(home)/index.tsx
• apps/expo/app/(app)/_layout.tsx
• apps/expo/app/(app)/wildlife/_layout.tsx
• apps/expo/app/(app)/wildlife/history.tsx
• apps/expo/app/(app)/wildlife/index.tsx
• apps/expo/config.ts
• apps/expo/features/wildlife/components/WildlifeTile.tsx
• apps/expo/features/wildlife/components/index.ts
• apps/expo/features/wildlife/constants.ts
• apps/expo/features/wildlife/hooks/index.ts
• apps/expo/features/wildlife/hooks/useWildlifeHistory.ts
• apps/expo/features/wildlife/hooks/useWildlifeIdentification.ts
• apps/expo/features/wildlife/index.ts
• apps/expo/features/wildlife/screens/WildlifeHistoryScreen.tsx
• apps/expo/features/wildlife/screens/WildlifeIdentificationScreen.tsx
• apps/expo/features/wildlife/screens/index.ts
• apps/expo/features/wildlife/types.ts
• apps/expo/lib/i18n/locales/en.json

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch copilot/implement-offline-plant-identification

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}