Skip to content

Social Loss Vignette

Jump to bottom
Mack edited this page May 31, 2023 · 2 revisions

Introduction

This vignette illustrates the process for creating precision feedback messages about a recipient's performance dropping below a peer comparator, such as a top performer benchmark or peer average. These messages use the social loss causal pathway, which specifies feedback messages that are capable of motivating providers through the delivery of information about performance worsening. Motivation from these messages can arise from the recognition of a loss of social status as a top or above-average performer. Example messages that use social loss are "You are no longer a top performer" and "Your performance has dropped below the peer average".

This vignette also contains examples of data features and other entities that a precision feedback system uses to evaluate the potential success of a precision feedback message. An example of such an entity is a peer average comparator (BP), which is defined in the Performance Summary Display Ontology (GH) as an average representing the mean performance of a peer group.

Performance Data

Each month, MPOG receives data about operative case quality and outcomes from approximately 60 healthcare institutions. MPOG calculates performance for each provider individually, for approximately 35 performance measures of quality and outcomes. One example of these measures is PONV-05: Post-operative Nausea and Vomiting Prophylaxis: Adults (MPOG).

Benchmark Comparators

MPOG calculates performance benchmarks and averages for each institution. One comparator is a peer 90th percentile benchmark (BP), which represents the 90th percentile for performance among providers at an institution, for each measure. Another is the peer average comparator (BP), which is the mean performance for all providers at an institution, for each measure.

Healthcare Professional Performance

Alice, an attending anesthesiologist at Midwest Medicine, has the following performance data over the last 6 months for PONV-05:

Month Performance Level Top 10% Benchmark Peer Average
Oct 91% 95 90
Nov 91% 95 90
Dec 91% 95 90
Jan 91% 95 90
Feb 91% 95 90
Mar 85% 95 90

Bob, a CRNA at Danville Hospital, has the following performance data over the last 6 months for PONV-05:

Month Performance Level Top 10% Benchmark Peer Average
Oct 88% 96 87
Nov 88% 96 87
Dec 88% 96 87
Jan 88% 96 87
Feb 88% 96 87
Mar 82% 96 87

Preference data

Preferences for precision feedback are elicited through a preference survey that providers can take. The preference survey produces a preference model for each provider that, with the provider's permission, is shared with MPOG to maintain. MPOG analyses preferences data that is shared to identify population-level preference segments. This generates preference profiles that can serve as a default preference model for an organization, or which can be selected by providers who do not take the preference survey, but who identify preferences that are close enough to their own in the settings menu for the precision feedback system.

Alice's preference data:

Motivating information Utility value Description
Social gain 12.97730 Performance was previously low, but it has improved to reach the peer benchmark.
Social stayed better 6.164377 Performance is consistently high (no recent change).
Worsening -12.23023 Performance is worsening.
Improving 3.325883 Performance is improving.
Social loss 9.956127 Performance was previously high, but it has dropped below the peer average.
Social stayed worse -7.710484 Performance has remained below average (no recent change).
Social better -1.61124 Performance is high this month.
Social worse -14.86794 Performance is low this month.
Social approach 13.99621 Performance is improving, getting closer to the peer benchmark.

Bob's preference data:

Motivating information Utility value Description
Social gain -2.92114 Performance was previously low, but it has improved to reach the peer benchmark.
Social stayed better -12.76936 Performance is consistently high (no recent change).
Worsening 1.06977 Performance is worsening.
Improving -0.26266 Performance is improving.
Social loss 2.72075 Performance was previously high, but it has dropped below the peer average.
Social stayed worse 9.97743 Performance has remained below average (no recent change).
Social better -5.97766 Performance is high this month.
Social worse -0.05277 Performance is low this month.
Social approach -0.24384 Performance is improving, getting closer to the peer benchmark.

Precision feedback message generation

To generate precision feedback, MPOG sends de-identified performance and preference data to a precision feedback system that processes each provider's data with their peer comparator performance data. The precision feedback system produces and evaluates candidate messages using metadata from message templates, then selects an optimal precision feedback message to return to MPOG. The precision feedback system is a knowledge-based system that is comprised of the following parts: 1) A knowledge base, 2) A software pipeline, and 3) A web service.

Precision Feedback Knowledge Base

The knowledge base contains the following components: Causal pathways, message templates, and performance measures.

Social Loss Causal Pathway

This pathway describes the influence of feedback interventions that show the message recipient that their performance has become worse than that of a social comparator, such as a top performer benchmark or peer average. The causal pathway model is made up of preconditions, moderators, mechanisms, and outcomes.

Preconditions

Preconditions for social loss are factors that are necessary for the success of the feedback intervention using this pathway. The social loss pathway has the following preconditions:

Information content preconditions:

  1. Social comparator content (BP)
  2. Negative performance gap content (BP)
  3. Negative performance trend content (BP)
  4. Loss content (BP)

Message preconditions:

  1. Social comparator element (BP)
  2. Negative performance gap set (BP)
  3. Negative performance trend set (BP)
  4. Loss set (BP)

Moderators

Moderators are factors that inhibit or promote the influence of the feedback intervention on the recipient. The social loss causal pathway has the following moderators:

  1. Habituation - How many times has the recipient previously received this message in the last year?
  2. Regulatory fit - To what extent is this message aligned with characteristics of the behavior/task, context, and recipient personality, with regard to motivation to avoid negative outcomes / problem (prevention focus), or motivation to achieve positive outcomes / develop and learn (promotion focus)?
  3. Gap size - How large is the gap between the recipient's performance level and that of the social comparator?
  4. Slope of trend
  5. Time since last achievement

Mechanisms

Mechanisms are factors that the intervention operates through to influence the feedback recipient. The social loss pathway has the following mechanisms:

  1. Awareness (knowledge): The message may change the recipient's awareness of their high performance, relative to peers.
  2. Subjective norms: The message may influence the recipient by creating or reinforcing their perception of their own top-performer status within their peer group.
  3. Motivation: The message may motivate the recipient to work to maintain their status as a top performer.

Outcomes

The expected outcome of a successful social loss pathway precision feedback intervention is clinical process performance improvement.

Message templates

Message templates represent a possible motivational message that a precision feedback system can send. There are two message templates that the precision feedback system can access for this vignette:

Drop Below Peer Average (GH) message:

Your performance dropped below the peer average for the measure [measure name].

This message template is about (OntoBee) the following data features:

  1. Social comparator element (BP)
  2. Negative performance gap set (BP)
  3. Negative performance trend set (BP)
  4. Loss set (BP)
  5. Peer average comparator (BP)
  6. Display format compatability: Line Graph (BP), Bar Chart (BP)

No Longer Top Performer (GH) message:

You are no longer a top performer for the measure [measure name].

This message template is about (OntoBee) the following data features:

  1. Social comparator element (BP)
  2. Negative performance gap set (BP)
  3. Negative performance trend set (BP)
  4. Loss set (BP)
  5. Peer 90th percentile comparator (BP)
  6. Display format compatability: Line Graph (BP), Bar Chart (BP)

Software Pipeline (Precision Feedback Pipeline)

Recipient annotations (Bitstomach)

The first stage of the pipeline analyzes performance to identify features of performance, such as comparisons and trends that are related to motivation. The analysis from this stage results in the following annotations:

Alice's annotations indicate the presence of the following in her performance data this month:

  1. Social comparator content (BP)
  2. Negative performance trend content (BP)
  3. Negative performance gap content (BP) regarding comparator: peer 90th percentile comparator (BP)
  4. Negative performance gap content (BP) regarding comparator: peer average comparator (BP)
  5. Loss content (BP) regarding comparator: peer average comparator (BP)

Bob's annotations indicate the presence of the following in his performance data this month:

  1. Social comparator content (BP)
  2. Negative performance trend content (BP)
  3. Negative performance gap content (BP) regarding comparator: peer 90th percentile comparator (BP)
  4. Positive performance gap content (BP) regarding comparator: peer average comparator (BP)
  5. Loss content (BP) regarding comparator: peer 90th percentile comparator (BP)

Candidate Message Generation (Candidate Smasher)

The second stage of the pipeline creates possible messages by associating the annotations for Alice and Bob with each message template, so that two candidate messages are created for each person.

Alice's candidate messages (A & B):

Candidate Message A has the following annotations:

The Drop Below Peer Average (GH) message template is about (OntoBee) the following data features:

  1. Social comparator element (BP)
  2. Negative performance gap set (BP)
  3. Negative performance trend set (BP)
  4. Loss set (BP)
  5. Peer average comparator (BP)
  6. Display format compatability: Line Graph (BP), Bar Chart (BP)

Alice's annotations from this month:

  1. Social comparator content (BP)
  2. Negative performance trend content (BP)
  3. Negative performance gap content (BP) regarding comparator: peer 90th percentile comparator (BP)
  4. Positive performance gap content (BP) regarding comparator: peer average comparator (BP)
  5. Loss content (BP) regarding comparator: peer average comparator (BP)

Candidate Message B has the following annotations:

The No Longer Top Performer (GH) message template is about (OntoBee) the following data features:

  1. Social comparator element (BP)
  2. Negative performance gap set (BP)
  3. Negative performance trend set (BP)
  4. Loss set (BP)
  5. Peer 90th percentile comparator (BP)
  6. Display format compatability: Line Graph (BP), Bar Chart (BP)

Alice's annotations from this month:

  1. Social comparator content (BP)
  2. Negative performance trend content (BP)
  3. Negative performance gap content (BP) regarding comparator: peer 90th percentile comparator (BP)
  4. Positive performance gap content (BP) regarding comparator: peer average comparator (BP)
  5. Loss content (BP) regarding comparator: peer average comparator (BP)

Bob's candidate messages (A & B):

Candidate Message A has the following annotations:

The Drop Below Peer Average (GH) message template is about (OntoBee) the following data features:

  1. Social comparator element (BP)
  2. Negative performance gap set (BP)
  3. Negative performance trend set (BP)
  4. Loss set (BP)
  5. Peer average comparator (BP)
  6. Display format compatability: Line Graph (BP), Bar Chart (BP)

Bob's performance is about (OntoBee):

  1. Social comparator content (BP)
  2. Negative performance trend content (BP)
  3. Negative performance gap content (BP) regarding comparator: peer 90th percentile comparator (BP)
  4. Positive performance gap content (BP) regarding comparator: peer average comparator (BP)
  5. Loss content (BP) regarding comparator: peer 90th percentile comparator (BP)

Candidate Message B has the following annotations:

The No Longer Top Performer (GH) message template is about (OntoBee) the following data features:

  1. Social comparator element (BP)
  2. Negative performance gap set (BP)
  3. Negative performance trend set (BP)
  4. Loss set (BP)
  5. peer 90th percentile comparator (BP)
  6. Display format compatability: Line Graph (BP), Bar Chart (BP)

Bob's performance is about (OntoBee):

  1. Social comparator content (BP)
  2. Negative performance trend content (BP)
  3. Negative performance gap content (BP) regarding comparator: peer 90th percentile comparator (BP)
  4. Positive performance gap content (BP) regarding comparator: peer average comparator (BP)
  5. Loss content (BP) regarding comparator: peer 90th percentile comparator (BP)

Candidate Message Preconditions Evaluation (Think Pudding)

For Alice, Candidate A is acceptable by social loss. For Bob, Candidate B is acceptable by social loss.

Candidate Message Moderator Evaluation and Selection (Esteemer)

TODO

Message Generation and Delivery (Pictoralist)

TODO

Causal pathway models Message templates Prioritization algorithms

Clone this wiki locally