Add models for Primo Search API integration

Why these changes are being introduced:

The first iteration of USE UI will include the
ability to toggle between Ex Libris CDI and
TIMDEX results.

Relevant ticket(s):

* [USE-30](https://mitlibraries.atlassian.net/browse/USE-30)

How this addresses that need:

This adds a Primo Search model that calls the
Primo Search API, and a Normalize Primo model that
parses the each result and returns a normalized
record.

Side effects of this change:

* For better or worse, much of the normalization
code has been repurposed from Bento. This felt
acceptable as we do not anticipate this to be a
long-term solution.
* While working on this, I noticed that most of
the TIMDEX normalization logic happens in the
views. It would be better to move this to
a normalization model, as we've done here with
Primo. I opened [USE-73](https://mitlibraries.atlassian.net/browse/USE-73)
to address this.
* It's not technically a side effect of this
changeset, but it's worth noting that we still
periodically see some GeoData test failures on
random test runs. I've documented this in [USE-72](https://mitlibraries.atlassian.net/browse/USE-72).

Author: jazairi

.env.test

@@ -1,4 +1,10 @@
-TIMDEX_HOST=FAKE_TIMDEX_HOST
-TIMDEX_GRAPHQL=https://FAKE_TIMDEX_HOST/graphql
-TIMDEX_INDEX=FAKE_TIMDEX_INDEX
 GDT=false
+MIT_PRIMO_URL=https://mit.primo.exlibrisgroup.com
+PRIMO_API_KEY=FAKE_PRIMO_API_KEY
+PRIMO_API_URL=https://api-na.hosted.exlibrisgroup.com/primo/v1
+PRIMO_SCOPE=cdi
+PRIMO_TAB=all
+PRIMO_VID=01MIT_INST:MIT
+TIMDEX_GRAPHQL=https://FAKE_TIMDEX_HOST/graphql
+TIMDEX_HOST=FAKE_TIMDEX_HOST
+TIMDEX_INDEX=FAKE_TIMDEX_INDEX

README.md

@@ -91,6 +91,12 @@ See `Optional Environment Variables` for more information.
 
 ### Required Environment Variables
 
+- `MIT_PRIMO_URL`: The base URL for MIT Libraries' Primo instance (used to generate record links).
+- `PRIMO_API_KEY`: The Primo Search API key.
+- `PRIMO_API_URL`: The Primo Search API base URL.
+- `PRIMO_SCOPE`: The Primo Search API `scope` param (set to `cdi` for CDI-scoped results).
+- `PRIMO_TAB`: The Primo Search API `tab` param (typically `all`).
+- `PRIMO_VID`: The Primo Search API `vid` (or 'view ID`) param.
 - `TIMDEX_GRAPHQL`: Set this to the URL of the GraphQL endpoint. There is no default value in the application.
 
 ### Optional Environment Variables
@@ -121,6 +127,7 @@ may have unexpected consequences if applied to other TIMDEX UI apps.
 - `GLOBAL_ALERT`: The main functionality for this comes from our theme gem, but when set the value will be rendered as
   safe html above the main header of the site.
 - `PLATFORM_NAME`: The value set is added to the header after the MIT Libraries logo. The logic and CSS for this comes from our theme gem.
+- `PRIMO_TIMEOUT`: The number of seconds before a Primo request times out (default 6).
 - `REQUESTS_PER_PERIOD` - number of requests that can be made for general throttles per `REQUEST_PERIOD`
 - `REQUEST_PERIOD` - time in minutes used along with `REQUESTS_PER_PERIOD`
 - `REDIRECT_REQUESTS_PER_PERIOD`- number of requests that can be made that the query string starts with our legacy redirect parameter to throttle per `REQUEST_PERIOD`

app/models/normalize_primo.rb

@@ -0,0 +1,249 @@
+# Transforms results from Primo Search API into normalized records
+class NormalizePrimo
+  def initialize(record)
+    @record = record
+  end
+
+  def normalize
+    {
+      'title' => title,
+      'creators' => creators,
+      'source' => source,
+      'year' => year,
+      'format' => format,
+      'links' => links,
+      'citation' => citation,
+      'container' => container_title,
+      'identifier' => record_id,
+      'summary' => summary,
+      'numbering' => numbering,
+      'chapter_numbering' => chapter_numbering
+    }
+  end
+
+  private
+
+  def title
+    if @record['pnx']['display']['title'].present?
+      @record['pnx']['display']['title'].join
+    else
+      'unknown title'
+    end
+  end
+
+    def creators
+    return [] unless @record['pnx']['display']['creator'] || @record['pnx']['display']['contributor']
+
+    author_list = []
+
+    if @record['pnx']['display']['creator']
+      creators = sanitize_authors(@record['pnx']['display']['creator'])
+      creators.each do |creator|
+        author_list << { value: creator, link: author_link(creator) }
+      end
+    end
+
+    if @record['pnx']['display']['contributor']
+      contributors = sanitize_authors(@record['pnx']['display']['contributor'])
+      contributors.each do |contributor|
+        author_list << { value: contributor, link: author_link(contributor) }
+      end
+    end
+
+    author_list.uniq
+  end
+
+  def source
+    'Primo'
+  end
+
+  def year
+    if @record['pnx']['display']['creationdate'].present?
+      @record['pnx']['display']['creationdate'].join
+    else
+      return unless @record['pnx']['search'] && @record['pnx']['search']['creationdate']
+
+      @record['pnx']['search']['creationdate'].join
+    end
+  end
+
+  def format
+    return unless @record['pnx']['display']['type']
+
+    normalize_type(@record['pnx']['display']['type'].join)
+  end
+
+  # While the links object in the Primo response often contains more than the Alma openurl, that is
+  # the one that is most predictably useful to us. The record_link is constructed.
+  def links
+    links = []
+    
+    # Add direct record link as the first link
+    if record_link.present?
+      links << { 'url' => record_link, 'kind' => 'full record' }
+    end
+
+    # Add openurl if available
+    if openurl.present?
+      links << { 'url' => openurl, 'kind' => 'openurl' }
+    end
+
+    # Return links if we found any
+    links.any? ? links : []
+  end
+
+  def citation
+    return unless @record['pnx']['addata']
+
+    if @record['pnx']['addata']['volume'].present?
+      if @record['pnx']['addata']['issue'].present?
+        "volume #{@record['pnx']['addata']['volume'].join} issue #{@record['pnx']['addata']['issue'].join}"
+      else
+        "volume #{@record['pnx']['addata']['volume'].join}"
+      end
+    elsif @record['pnx']['addata']['date'].present? && @record['pnx']['addata']['pages'].present?
+      "#{@record['pnx']['addata']['date'].join}, pp. #{@record['pnx']['addata']['pages'].join}"
+    end
+  end
+
+  def container_title
+    return unless @record['pnx']['addata']
+
+    if @record['pnx']['addata']['jtitle'].present?
+      @record['pnx']['addata']['jtitle'].join
+    elsif @record['pnx']['addata']['btitle'].present?
+      @record['pnx']['addata']['btitle'].join
+    end
+  end
+
+  def record_id
+    return unless @record['pnx']['control']['recordid']
+
+    @record['pnx']['control']['recordid'].join
+  end
+
+  def summary
+    return unless @record['pnx']['display']['description']
+
+    @record['pnx']['display']['description'].join(' ')
+  end
+
+  # This constructs a link to the record in Primo.
+  #
+  # We've altered this method slightly to address bugs introduced in the Primo VE November 2021
+  # release. The search_scope param is now required for CDI fulldisplay links, and the context param
+  # is now required for local (catalog) fulldisplay links.
+  #
+  # In order to avoid more surprises, we're adding all of the params included in the fulldisplay
+  # example links provided here, even though not all of them are actually required at present:
+  # https://developers.exlibrisgroup.com/primo/apis/deep-links-new-ui/
+  #
+  # We should keep an eye on this over subsequent Primo reeleases and revert it to something more
+  # minimalist/sensible when Ex Libris fixes this issue.
+  def record_link
+    return unless @record['pnx']['control']['recordid']
+    return unless @record['context']
+
+    record_id = @record['pnx']['control']['recordid'].join
+    base = [ENV.fetch('MIT_PRIMO_URL'), '/discovery/fulldisplay?'].join
+    query = {
+      docid: record_id,
+      vid: ENV.fetch('PRIMO_VID'),
+      context: @record['context'],
+      search_scope: 'all',
+      lang: 'en',
+      tab: ENV.fetch('PRIMO_TAB')
+    }.to_query
+    [base, query].join
+  end
+
+  def numbering
+    return unless @record['pnx'] && @record['pnx']['addata']
+    return unless @record['pnx']['addata']['volume']
+
+    if @record['pnx']['addata']['issue'].present?
+      "volume #{@record['pnx']['addata']['volume'].join} issue #{@record['pnx']['addata']['issue'].join}"
+    else
+      "volume #{@record['pnx']['addata']['volume'].join}"
+    end
+  end
+
+  def chapter_numbering
+    return unless @record['pnx'] && @record['pnx']['addata']
+    return unless @record['pnx']['addata']['btitle']
+    return unless @record['pnx']['addata']['date'] && @record['pnx']['addata']['pages']
+
+    "#{@record['pnx']['addata']['date'].join}, pp. #{@record['pnx']['addata']['pages'].join}"
+  end
+
+  def openurl
+    return unless @record['delivery'] && @record['delivery']['almaOpenurl']
+
+    @record['delivery']['almaOpenurl'].is_a?(Array) ? @record['delivery']['almaOpenurl'].join : @record['delivery']['almaOpenurl']
+  end
+
+  def sanitize_authors(authors)
+    authors.map! { |author| author.split(';') }.flatten! if authors.any? { |author| author.include?(';') }
+    authors.map { |author| author.strip.gsub(/\$\$Q.*$/, '') }
+  end
+
+  def author_link(author)
+    [ENV.fetch('MIT_PRIMO_URL'),
+    '/discovery/search?query=creator,exact,',
+     encode_author(author),
+     '&tab=', ENV.fetch('PRIMO_TAB'),
+     '&search_scope=all&vid=',
+     ENV.fetch('PRIMO_VID')].join
+  end
+
+  def encode_author(author)
+    URI.encode_uri_component(author)
+  end
+
+  def normalize_type(type)
+    r_types = {
+      'BKSE' => 'eBook',
+      'reference_entry' => 'Reference Entry',
+      'Book_chapter' => 'Book Chapter'
+    }
+    r_types[type] || type.capitalize
+  end
+
+  # It's possible we'll encounter records that use a different server,
+  # so we want to test against our expected server to guard against
+  # malformed URLs. This assumes all URL strings begin with https://.
+  def openurl
+    return unless @record['delivery'] && @record['delivery']['almaOpenurl']
+
+    # Check server match
+    openurl_server = ENV['ALMA_OPENURL'][8, 4]
+    record_openurl_server = @record['delivery']['almaOpenurl'][8, 4]
+    if openurl_server == record_openurl_server
+      construct_primo_openurl
+    else
+      Rails.logger.warn "Alma openurl server mismatch. Expected #{openurl_server}, but received #{record_openurl_server}. (record ID: #{record_id})"
+      @record['delivery']['almaOpenurl']
+    end
+  end
+
+  def construct_primo_openurl
+    return unless @record['delivery']['almaOpenurl']
+
+    # Here we are converting the Alma link resolver URL provided by the Primo
+    # Search API to redirect to the Primo UI. This is done for UX purposes,
+    # as the regular Alma link resolver URLs redirect to a plaintext
+    # disambiguation page.
+    primo_openurl_base = [ENV['MIT_PRIMO_URL'],
+                          '/discovery/openurl?institution=',
+                          ENV['EXL_INST_ID'],
+                          '&vid=',
+                          ENV['PRIMO_VID'],
+                          '&'].join
+    primo_openurl = @record['delivery']['almaOpenurl'].gsub(ENV['ALMA_OPENURL'], primo_openurl_base)
+
+    # The ctx params appear to break Primo openurls, so we need to remove them.
+    params = Rack::Utils.parse_nested_query(primo_openurl)
+    filtered = params.delete_if { |key, _value| key.starts_with?('ctx') }
+    URI::DEFAULT_PARSER.unescape(filtered.to_param)
+  end
+end

app/models/primo_search.rb

@@ -0,0 +1,68 @@
+# Searches Primo Search API and formats results
+#
+class PrimoSearch
+  PRIMO_API_URL = ENV.fetch('PRIMO_API_URL', nil).freeze
+  PRIMO_API_KEY = ENV.fetch('PRIMO_API_KEY', nil)
+  PRIMO_SCOPE = ENV.fetch('PRIMO_SCOPE', nil)
+  PRIMO_TAB = ENV.fetch('PRIMO_TAB', nil)
+  PRIMO_VID = ENV.fetch('PRIMO_VID', nil)
+
+  def initialize
+    @primo_http = HTTP.persistent(PRIMO_API_URL)
+    @results = {}
+  end
+
+  def search(term, per_page)
+    url = search_url(term, per_page)
+    Rails.logger.info "Primo Search URL: #{url}"
+    
+    result = @primo_http.timeout(http_timeout)
+                      .headers(
+                        accept: 'application/json',
+                        Authorization: "apikey #{PRIMO_API_KEY}"
+                      )
+                      .get(url)
+
+    Rails.logger.info "Primo Response Status: #{result.status}"
+    Rails.logger.info "Primo Response Headers: #{result.headers.to_h}"
+    
+    raise "Primo Error Detected: #{result.status}" unless result.status == 200
+
+    JSON.parse(result)
+  end
+
+  private
+
+  # Initial search term sanitization
+  def clean_term(term)
+    term.strip.tr(' :,', '+').gsub(/\++/, '+')
+  end
+
+  # Constructs the search URL with required parameters for Primo API
+  def search_url(term, per_page)
+    [
+      PRIMO_API_URL,
+      '/search?q=any,contains,',
+      clean_term(term),
+      '&vid=',
+      PRIMO_VID,
+      '&tab=',
+      PRIMO_TAB,
+      '&scope=',
+      PRIMO_SCOPE,
+      '&limit=',
+      per_page,
+      '&apikey=',
+      PRIMO_API_KEY
+    ].join
+  end
+
+  # Timeout configuration for HTTP requests
+  def http_timeout
+    if ENV.fetch('PRIMO_TIMEOUT', nil).present?
+      ENV['PRIMO_TIMEOUT'].to_f
+    else
+      6
+    end
+  end
+end

test/fixtures/primo/full_record.json

@@ -0,0 +1,37 @@
+{
+  "pnx": {
+    "display": {
+      "title": ["Testing the Limits of Knowledge"],
+      "creator": ["Smith, John A.", "Jones, Mary B."],
+      "contributor": ["Brown, Robert C."],
+      "creationdate": ["2023"],
+      "type": ["book"],
+      "description": ["A comprehensive study of testing methodologies"],
+      "subject": ["Computer Science", "Software Testing"]
+    },
+    "addata": {
+      "btitle": ["Complete Guide to Testing"],
+      "date": ["2023"],
+      "volume": ["2"],
+      "issue": ["3"],
+      "pages": ["123-145"],
+      "jtitle": ["Journal of Testing"]
+    },
+    "search": {
+      "creationdate": ["2023"]
+    },
+    "control": {
+      "recordid": ["MIT01000000001"]
+    }
+  },
+  "context": "contextual",
+  "delivery": {
+    "bestlocation": {
+      "mainLocation": "Hayden Library",
+      "subLocation": "Stacks",
+      "callNumber": "QA76.73.R83 2023"
+    },
+    "link": [],
+    "almaOpenurl": "https://example.com/openurl?param=value"
+  }
+}