Skip to content

feat: Standardized Time-Series Emotion Output Schema#10

Open
cc-fuyu wants to merge 1 commit intoruxailab:mainfrom
cc-fuyu:feat/standardized-output-schema
Open

feat: Standardized Time-Series Emotion Output Schema#10
cc-fuyu wants to merge 1 commit intoruxailab:mainfrom
cc-fuyu:feat/standardized-output-schema

Conversation

@cc-fuyu
Copy link
Copy Markdown

@cc-fuyu cc-fuyu commented Feb 27, 2026

This PR introduces a standardized, time-aware output format for the facial emotion analysis API. It directly addresses the "Standard Output Schemas" key feature described in the GSoC 2026 project "Sentiment and Emotion Output Standardization for Usability Reports".

Problem

The current /process_video endpoint returns only a flat dictionary of aggregated emotion percentages. While useful, this format lacks:

  • Temporal context — no information about when emotions occurred in the video.
  • Confidence scores — no indication of how certain the model was about each prediction.
  • Metadata — no record of analysis parameters (frames processed, faces detected, video duration).

Changes

New Schema (schemas/standard_output_schema.py)

Defines a structured Pydantic model StandardizedEmotionOutput with three sections:

Section Description
metadata Video name, ISO 8601 analysis timestamp, total frames processed, faces detected, video duration
timeline Chronologically ordered list of EmotionEvent objects, each containing timestamp_sec, emotion label, and confidence score (0.0–1.0)
summary Aggregated emotion percentages (backward-compatible with the original output)

Updated Analysis Service

  • Added get_standardized_output() method to EmotionsAnalysisImp that captures per-frame timestamps and model confidence during video processing.
  • Updated the abstract base class EmotionsAnalysisService with the new method signature.
  • The original get_emotion_percentages() method remains unchanged for backward compatibility.

New API Endpoint

  • POST /process_video_standardized — accepts the same request body as /process_video but returns the full standardized output.

Example Response

{
  "metadata": {
    "video_name": "session_42.webm",
    "analysis_timestamp": "2026-02-27T11:30:00.000Z",
    "total_frames_processed": 120,
    "total_faces_detected": 115,
    "video_duration_sec": 60.0
  },
  "timeline": [
    {"timestamp_sec": 0.5, "emotion": "Neutral", "confidence": 0.87},
    {"timestamp_sec": 1.0, "emotion": "Happy", "confidence": 0.93}
  ],
  "summary": {
    "Angry": 2.5, "Disgusted": 0.0, "Fearful": 1.2,
    "Happy": 35.0, "Neutral": 55.3, "Sad": 4.0, "Surprised": 2.0
  }
}

Backward Compatibility

The original /process_video endpoint and GetEmotionPercentagesResponse schema are not modified. Existing consumers will continue to work without any changes.

Backward Compatibility

The original /process_video endpoint and GetEmotionPercentagesResponse schema are not modified. Existing consumers will continue to work without any changes.

Introduce a new StandardizedEmotionOutput schema that includes:
- AnalysisMetadata: video name, analysis timestamp, frame/face counts, duration
- EmotionEvent timeline: per-frame emotion label, timestamp, and confidence score
- EmotionSummary: aggregated emotion percentages (backward-compatible)

Add get_standardized_output() method to EmotionsAnalysisImp and a new
/process_video_standardized endpoint that returns the structured output.

The original /process_video endpoint remains unchanged for backward
compatibility.

This addresses the 'Standard Output Schemas' key feature of the GSoC
'Sentiment and Emotion Output Standardization' project.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant