Hyperion: User Guidance for Problem Statement Generation & Refinement

[HawKhiem]

Status: Draft
Related: #11940 (Problem statement refinement), #11594 (Problem statement generation)

Summary

Introduce a progressive, contextual user guidance system for the new AI-powered problem statement generation and refinement features, helping first-time users understand the feature's value within 30–60 seconds, teaching at the moment of intent, and fading out once the user has successfully used the feature.

Motivation (Why)

• Feature discoverability gap: The problem statement generation and refinement features exist, but new users don't clearly understand what they do or why they're useful until they stumble upon them.
• No in-product education: There's currently no guidance in the interface explaining how to use the AI assistance effectively, leading to underutilization.
• First-use friction: Users see a textarea and buttons but don't immediately grasp the "input → AI transformation → review → accept" workflow.
• Inline refinement invisibility: The text selection-based refinement pattern is non-obvious and easily missed.

Proposal (Approach)

Implement a layered guidance strategy with four levels:

1. Discoverability (Passive) - Visual markers and subtle animations that make users aware the feature exists
2. Contextual Explanation (On Hover/Focus) - Just-in-time tooltips when users show interest
3. Guided First Success (Interactive) - Empty states and coachmarks that walk users through their first action
4. On-demand Help (Persistent) - Help icons and keyboard shortcuts for users who want to learn more

All guidance elements are:

• ✅ Skippable and non-blocking
• ✅ Contextual and inline (no modals or forced tours)
• ✅ Self-dismissing after successful feature use
• ✅ Stored in localStorage to persist across sessions

Architecture

flowchart TB
    subgraph Client["Client (Angular)"]
        subgraph Components
            ProblemComponent[ProgrammingExerciseProblemComponent]
            EditableInstructions[EditableInstructionComponent]
            InlineButton[InlineRefinementButtonComponent]
            DiffEditor[MonacoDiffEditorComponent]
        end
        
        subgraph Guidance["Guidance Layer"]
            GuidanceService[FeatureGuidanceService]
            EmptyState[Empty State Component]
            Tooltips[Enhanced Tooltips]
            Coachmarks[Coachmark Component]
            Toast[Success Toast]
        end
        
        GuidanceService -->|"tracks state"| LocalStorage[(localStorage)]
        
        ProblemComponent --> EmptyState
        ProblemComponent --> Tooltips
        EditableInstructions --> Coachmarks
        InlineButton --> Tooltips
        DiffEditor --> Toast
    end

    subgraph Storage["Persistence"]
        LocalStorage -->|"stores"| Keys["artemis.guidance.*"]
    end

Loading

UX Flow

First-Time User Journey

sequenceDiagram
    participant I as Instructor
    participant UI as Problem Editor
    participant G as Guidance System
    participant LS as localStorage

    I->>UI: Opens Problem Statement step
    UI->>G: Check first visit
    G->>LS: artemis.guidance.visited?
    LS-->>G: false
    G-->>UI: Show empty state + pulse animation
    
    I->>UI: Hovers "Generate Draft" button
    UI->>I: Show enhanced tooltip (300ms delay)
    
    I->>UI: Types requirements, clicks Generate
    UI->>API: POST /generate
    API-->>UI: Generated problem statement
    UI->>G: Mark generation complete
    G->>LS: Set artemis.guidance.psGenerated = true
    UI->>I: Show success toast + diff view
    
    I->>UI: Selects text in editor
    UI->>I: Show inline refinement button
    G->>LS: artemis.guidance.inlineRefinementUsed?
    LS-->>G: false
    UI->>I: Show first-use explanation on button
    
    I->>UI: Uses inline refinement
    G->>LS: Set artemis.guidance.inlineRefinementUsed = true
    Note over G,LS: Guidance no longer shown

Loading

Guidance Persistence Logic

flowchart TD
    A[User Action] --> B{Which action?}
    
    B -->|"First page visit"| C[Show pulse animation]
    C --> C1[Dismiss after 2s]
    C1 --> C2[Store in sessionStorage]
    
    B -->|"First generation"| D[Show success toast]
    D --> D1[Store psGenerated in localStorage]
    D1 --> D2[Hide textarea hint forever]
    
    B -->|"First inline refinement"| E[Show refinement success]
    E --> E1[Store inlineRefinementUsed]
    E1 --> E2[Hide inline explanation forever]
    
    B -->|"Coachmark dismissed"| F[Store coachmarkDismissed]
    F --> F1[Never show coachmarks again]

Loading

UI Patterns & Copy

Empty State (When editor is empty)

┌──────────────────────────────────────────────────────────────┐
│                         📝                                   │
│                                                              │
│          Create Your Problem Statement                       │
│                                                              │
│   Describe your exercise idea in the text field above,       │
│   then click "Generate Draft Problem Statement" to create    │
│   a structured exercise brief with AI assistance.            │
│                                                              │
│   ▸ See example input                                        │
│                                                              │
│   ─────────────────── or ───────────────────                 │
│                                                              │
│   Start typing in the editor to write manually.              │
│                                                              │
└──────────────────────────────────────────────────────────────┘

Enhanced Tooltips

Element	Tooltip Content
Generate Draft button	"Transform your exercise idea into a complete, structured problem statement with tasks and test placeholders."
Refine button	"Improve your existing problem statement. Describe what changes you want to make."
Inline refinement button	"Refine just your selected text. Try 'make clearer' or 'add more detail'."

First-Use Diff View Guidance

┌────────────────────────────────────────────────────────────────┐
│ 📖 Review Mode                                                  │
│                                                                 │
│ Green lines (+) will be added. Red lines (-) will be removed.  │
│ You can edit the right side before accepting changes.          │
│                                                                 │
│                                              [Got it]           │
└────────────────────────────────────────────────────────────────┘

Implementation Plan

Phase 1: Quick Wins (2-3 days)

Task	Effort	Files to Modify
Enhanced tooltips on Generate/Refine buttons	1h	programming-exercise-problem.component.html
Empty state guidance in editor area	2h	programming-exercise-problem.component.html/ts
Success toast after first generation	1h	programming-exercise-problem.component.ts
Tooltip on inline refinement button	0.5h	inline-refinement-button.component.html
i18n keys for all guidance copy	1h	programmingExercise.json (en/de)

Phase 2: Enhanced Guidance (1 week)

Task	Effort	Files to Create/Modify
FeatureGuidanceService for state tracking	4h	New: feature-guidance.service.ts
Dismissible coachmark component	4h	New: coachmark.component.ts/html/scss
First-use textarea focus hint	2h	programming-exercise-problem.component.ts
Diff view first-use guidance	2h	programming-exercise-editable-instruction.component.ts
Guidance reset in user settings	2h	settings.component.ts

Success Metrics

Metric	Definition	Target
Feature Discovery Rate	% of users who interact with AI features	>60%
First Generation Completion	% who complete their first generation	>80%
Refinement Adoption	% who use refinement after generation	>30%
Time to First Success	Time from page load to first generation	<90 seconds
Guidance Dismissal Rate	% who dismiss without completing action	<20%

Tracking Implementation

// Events to track
analytics.track('guidance.empty_state.viewed');
analytics.track('guidance.tooltip.shown', { element: 'generate_button' });
analytics.track('guidance.coachmark.dismissed', { completed: boolean });
analytics.track('feature.generation.completed', { isFirst: boolean });
analytics.track('feature.refinement.completed', { type: 'global' | 'inline' });

Technical Details

Storage Keys

const GUIDANCE_KEYS = {
  PS_GENERATED: 'artemis.guidance.psGenerated',
  PS_REFINED: 'artemis.guidance.psRefined',
  INLINE_REFINEMENT_USED: 'artemis.guidance.inlineRefinementUsed',
  COACHMARK_DISMISSED: 'artemis.guidance.coachmarkDismissed',
  DIFF_VIEW_EXPLAINED: 'artemis.guidance.diffViewExplained',
};

Service Interface

@Injectable({ providedIn: 'root' })
export class FeatureGuidanceService {
  private readonly storage = inject(LocalStorageService);

  // Check guidance state
  hasCompletedGeneration(): boolean;
  hasCompletedRefinement(): boolean;
  hasUsedInlineRefinement(): boolean;
  shouldShowCoachmark(key: string): boolean;
  
  // Mark actions complete
  markGenerationComplete(): void;
  markRefinementComplete(): void;
  markInlineRefinementUsed(): void;
  dismissCoachmark(key: string): void;
  
  // Reset for support/testing
  resetAllGuidance(): void;
}

i18n Additions

{
  "problemStatement": {
    "emptyState": {
      "title": "Create Your Problem Statement",
      "description": "Describe your exercise idea above, then click 'Generate Draft' to create a structured problem statement.",
      "seeExample": "See example input",
      "orWriteManually": "Start typing in the editor to write manually."
    },
    "tooltips": {
      "generateButton": "Transform your exercise idea into a complete, structured problem statement with tasks and test placeholders.",
      "refineButton": "Improve your existing problem statement. Describe what changes you want to make.",
      "inlineRefinement": "Refine just your selected text. Try 'make clearer' or 'add more detail'."
    },
    "firstUse": {
      "generationSuccess": "Problem statement generated! Review the changes below.",
      "refinementNudge": "Select any text to refine specific sections with AI."
    },
    "diffGuidance": {
      "title": "Review Mode",
      "description": "Green lines (+) will be added. Red lines (-) will be removed. You can edit the right side before accepting changes.",
      "gotIt": "Got it"
    }
  }
}

Out of Scope (Future)

• Video tutorials or documentation links
• Gamification/progress tracking across features
• A/B testing framework integration
• Cross-feature onboarding (e.g., code generation guidance)

Open Questions

1. Analytics integration: Which analytics system should track guidance events (Langfuse, custom, etc.)?
2. Auto-proceed vs manual: Should some tooltips auto-dismiss after a delay, or always require user action?
3. German translations: Should Phase 1 include German i18n, or defer to Phase 2?
4. Accessibility: Do coachmarks need full ARIA support and screen reader announcements?
5. Reset location: Should "Reset tutorials" be in user settings, or a developer-only debug panel?

---

Feedback welcome. Particularly interested in:

• Whether the layered approach provides enough guidance without being intrusive
• Phase 1 vs Phase 2 priority adjustments
• Any additional UI patterns we should consider

[IcanCUthere]

In my opinion the overall idea sounds good and would make it way more usable especially for the new AI features.

I have a question for the layered approach: Is this a conceptual design for how it could be implemented? Like a GuidanceService that you can turn on or off if you need it or not. I did not quite understand that.

The Phase priorities seem fine to me.

An additional UI pattern we need to consider is how we make the AI features stand out in the UI that you can identify them at first glance and know that they are a "nice to have" but not necessary. E.g. with a special color or something. I think this would make the user experience also a lot better.