DEV: Backfill embeddings concurrently.

[romanrizzi]

We are adding a new method for generating and storing embeddings in bulk, which relies on Concurrent::Promises::Future. Generating an embedding consists of three steps:

• Prepare text
• HTTP call to retrieve the vector
• Save to DB.

Second one is independently executed on whatever thread the pool gives us.

We are bringing a custom thread pool instead of the global executor since we want control over how many threads we spawn to limit concurrency. We also avoid firing thousands of HTTP requests when working with large batches.

Ran benchmarks locally with the following script:

def benchmark_embeddings
  ActiveRecord::Base.logger.level = 1 # or Logger::INFO
  truncation = DiscourseAi::Embeddings::Strategies::Truncation.new
  vector_rep = DiscourseAi::Embeddings::VectorRepresentations::Base.current_representation(truncation)

  Benchmark.bmbm do |x|
    x.report("With concurrent (90 topics)") do 
      DB.exec("DELETE FROM ai_topic_embeddings")
      vector_rep.gen_bulk_reprensentations(Topic.includes(:tags, :posts).all)
    end

    x.report("Without concurrent (90 topics)") do
      DB.exec("DELETE FROM ai_topic_embeddings")
      Topic.includes(:tags, :posts).all.each { |t| vector_rep.generate_representation_from(t) }
    end
  end
end

Results:

                                     user     system      total        real
With concurrent (90 topics)      2.418655   0.084879   2.503534 (  3.716947)
Without concurrent (90 topics)   5.097356   0.268728   5.366084 ( 63.132123)