API completions frequently hallucinate?

[joeyfromspace]

Frequently, we receive garbled random text or long sequences of nonsense characters with our completions. At first I thought this was exclusive to sonar-deep-research but we are now seeing it in our completions with sonar-pro as well. We haven't changed any of our API call parameters before, and sonar-pro had been working great for us for some weeks now.

I've tried tweaking various things like not sending up a system prompt, adjusting settings like temperature and frequency_penalty, but nothing I've tried seems to have any effect on the outcome.

What can I do to improve the quality of our results? Or is this an issue with Perplexity itself?

For example, here is one request for deep research:

{
  "model": "sonar-deep-research",
  "temperature": 0.2,
  "frequency_penalty": 1.0,
  "top_p": 0.9,
  "messages": [
    {
      "content": "Prepare a report detailing Oprah Winfrey's different media businesses and how they compare in terms of revenue.",
      "role": "user"
    }
  ],
  "response_format": {
    "type": "json_schema",
    "schema": {
      "type": "object",
      "properties": {
        "text": {
          "type": "string"
        },
        "citations": {
          "type": "array",
          "items": {
            "type": "object",
            "properties": {
              "url": {
                "type": "string"
              },
              "title": {
                "type": "string"
              }
            },
            "required": [
              "url",
              "title"
            ]
          }
        }
      },
      "required": [
        "text",
        "citations"
      ]
    }
  }
}

And this is the text response I received (after parsing out the CoT and the text from the json schema):

# Analysis of Oprah Winfrey’s Media Business Portfolio

Oprah Winfrey’s media empire spans television networks, production companies publishing platforms,and strategic investments.This report examines five core components:OWN:TheOprahWinfreyNetworkHarpoProductions,O:TheOprahmagazine/digitalplatformsWeightWatchers(WWInternational),andothermediaventures.Eachsectionanalyzeshistoricalperformanceoperationalstructureandrevenuegenerationwithcomparativeinsightsacrossbusinessunits.

##OWN:TheOpraahWnfeyNetwork###FoundingandOwnershipStructureLaunchedinJanuary2011asajointventurebetweenHarProductionsandDiscoveryCommunicationsOWbeganwitha50/50ownershipsplit.Discoveryacquiredadditionalstakesovertimeholding95%byDecember2020whileHarporetained25.%minorityinterest(source: [https://en.wikipedia.org/wiki/Oprah_Winfrey_Network](https://en.wikipedia.org/wiki/Oprah_Winfrey_Network))(source: [https://www.glory.media/business-of-celebrity-oprah-winfrey-net-worth-ventures/](https://www.glory.media/business-of-celebrity-oprah-winfrey-net-worth-ventures/)).HeadquarteredinLosAngelesCaliforniaOWNreaches85millionhomesofferingoriginalprogrammingdocumentariesandsyndicatedcontentfromHaroStudioslibrary.###RevenuePerformanceSearchresultsindicatdivergentfigures:RocketReachreport$14billionannualrevenueforOWiNn024employig1040staffmembers(source: [https://rocketreach.co/own-the-oprah-winfrey-network-profile_b5c1b4a2f42e08d6](https://rocketreach.co/own-the-oprah-winfrey-network-profile_b5c1b4a2f42e08d6)).Industryanalysissuggeststhenetworkgenerates“hundredsofmilions”annuallythroughadvertisigpartnershipsandcontentlicensingagreementswithpeakviewershipreachig300000primetimeviewersduringitsmostsuccessfulperiods.Thediscrepancybetweensourceslikelyreflectsdifferentreportingmethodologies(RocketReachsourcedatavsindustryestimates).

##HarProductions###HistoricalContextFoundedin1986asamultimediaproductioncompanyHarPobecamethedrivingforcebehindTheOprhWifeyShowsyndicationrightsandspin-offprogramslikDrPhilndRachealRay.Atitslate199speakHarpogenerated$150millionannuallyemploying200staffmembersChicago-basedoperationsclosediMarch205tofocusonLAprojectsrelatedtONdevelopment###CurrentFinancialMetricsAccordingtoRocketReachdata,HaroProductiosgenerated$958miloinrevenueduring025withemployeecountdroppingto08personelThisrepresentssignificantdownscalingfrompeakyearsshowcasingshifttowardstelevisionnetworprioritiesoverindependentproductionactivities[]

##OThOPRAHMAGAZINEANDDIGITALTRANSITION###PrintEraSuccessLaunchediinApril200Owithmonthlycirculationpeakingat27millioocopiesbyJune004.Magazinenewsstandsalesgrew58%duringsecondhalfof009despiteindustry-widedeclinesearningtitle“mostsuccessfulstart-up”inpublishinghistory59][14.HearstCommunicationspartneredwthWinftoproducepublicationuntilprintcessationDecember020.###DigitalRebrandingPost-printphasebrandevolvedintoOpahDailyfeaturigquarterlyspeciaissues($50annualmembership)andexpandeddigitalcontentfocus.Oraginalwebsitetrafictripledpost-rebrandwithsocialmediaengagementdrivingnewrevenuestreamsthroughaffiliatemarketingnadsponsoredcontentpartnerships101112]

##WEIGHTWATCHERSINVESTMENT###StrategicStakeAcquisitionIn205,Wifeyinvested$43millionfor10%stakeiWeightWatcherswhichrebrandedaWWInternatioal.Herinvolvementtriggereda400%stockpriceincreasegeneratingover$00millioinpersonalreturnsbysellingportionofshareswhileretainingboardpositionandinfluencoversstrategicdirection.CurrentvaluationimpactcontributessignificantlytOWveralnetworthestimatedt~$.7bilion613]

##OtherMediaVentures###FilmBookProductionBeyondcorebusinessesHaProFilmsproducedcriticallyacclaimedworkslikeSelma(214)ndGreenleaf(16-20),thoughspecificfinancialdataremainsprivate.Bookclubselectionscontinuetodrivepublishigindustrysalesexemplifiedby55milicopiesoldthroughhercuratedlistsoverdecadescontributingroyaltystreamstoherportfolio45]
#ComparativeRevenueAnalysis||BusinessUnit|PeakRevenue|CurrentEstimate|KeyFactors|
|---|---|---|---|---|
oWN|$14bilion(024)|Undisclosed(mid-hundedsmilions)|Advertisingsubscriptionfeesyndication|
aProducions|15omillion(late199os)|958milion(025)|Syndicationrightstelevisionproductio|
magazine/digital|Notdisclosed(~20m/yearcirculationera)|Membership/advertising(~30m/year)|Subscriptionaffiliateincome|
eightWatchersStakeN/A~70millionannualdividendsStockappreciationreturns||
isanalysisrevealsOWNastoprevnucontributorfollowebyHistoricHaProdutionoutput.Modern-dyprofitabilityshiftstowarddigitalplatformsandstrategicinvestmentslikeWWwhichofferhigheragilitymarketvolatilitycomparedtraditionalmediabusinessmodels.

[james-pplx]

Hey @joeyfromspace ! Thanks for bringing this to our attention. A bug has been filed and I'm personally looking into this. Will let you know as soon as we have some updates.

Also in the future if you come across issues like please feel free to file a bug. This will help streamline the process!