Skip to content
This repository was archived by the owner on Jul 22, 2025. It is now read-only.

Commit 6827147

Browse files
authored
DEV: Add topic and post id when using completions for traceability to AiApiAuditLog (#1414)
The AiApiAuditLog per translation event doesn't trace back easily to a post or topic. This commit adds support to that, and also switches the translators to named arguments rather than positional arguments.
1 parent 8a3a247 commit 6827147

11 files changed

+98
-72
lines changed

lib/translation/base_translator.rb

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,16 +3,20 @@
33
module DiscourseAi
44
module Translation
55
class BaseTranslator
6-
def initialize(text, target_language)
6+
def initialize(text:, target_locale:, topic_id: nil, post_id: nil)
77
@text = text
8-
@target_language = target_language
8+
@target_locale = target_locale
9+
@topic_id = topic_id
10+
@post_id = post_id
911
end
1012

1113
def translate
1214
prompt =
1315
DiscourseAi::Completions::Prompt.new(
1416
prompt_template,
1517
messages: [{ type: :user, content: formatted_content, id: "user" }],
18+
topic_id: @topic_id,
19+
post_id: @post_id,
1620
)
1721

1822
structured_output =
@@ -27,7 +31,7 @@ def translate
2731
end
2832

2933
def formatted_content
30-
{ content: @text, target_language: @target_language }.to_json
34+
{ content: @text, target_locale: @target_locale }.to_json
3135
end
3236

3337
def response_format

lib/translation/category_localizer.rb

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -6,20 +6,20 @@ class CategoryLocalizer
66
def self.localize(category, target_locale = I18n.locale)
77
return if category.blank? || target_locale.blank?
88

9-
target_locale_sym = target_locale.to_s.sub("-", "_").to_sym
9+
target_locale = target_locale.to_s.sub("-", "_")
1010

11-
translated_name = ShortTextTranslator.new(category.name, target_locale_sym).translate
11+
translated_name = ShortTextTranslator.new(text: category.name, target_locale:).translate
1212
translated_description =
1313
if category.description.present?
14-
PostRawTranslator.new(category.description, target_locale_sym).translate
14+
PostRawTranslator.new(text: category.description, target_locale:).translate
1515
else
1616
""
1717
end
1818

1919
localization =
2020
CategoryLocalization.find_or_initialize_by(
2121
category_id: category.id,
22-
locale: target_locale_sym.to_s,
22+
locale: target_locale,
2323
)
2424

2525
localization.name = translated_name

lib/translation/post_localizer.rb

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5,16 +5,23 @@ module Translation
55
class PostLocalizer
66
def self.localize(post, target_locale = I18n.locale)
77
return if post.blank? || target_locale.blank? || post.locale == target_locale.to_s
8-
target_locale_sym = target_locale.to_s.sub("-", "_").to_sym
8+
target_locale = target_locale.to_s.sub("-", "_")
99

1010
translated_raw =
1111
ContentSplitter
1212
.split(post.raw)
13-
.map { |chunk| PostRawTranslator.new(chunk, target_locale_sym).translate }
13+
.map do |text|
14+
PostRawTranslator.new(
15+
text:,
16+
target_locale:,
17+
topic_id: post.topic_id,
18+
post_id: post.id,
19+
).translate
20+
end
1421
.join("")
1522

1623
localization =
17-
PostLocalization.find_or_initialize_by(post_id: post.id, locale: target_locale_sym.to_s)
24+
PostLocalization.find_or_initialize_by(post_id: post.id, locale: target_locale)
1825

1926
localization.raw = translated_raw
2027
localization.cooked = PrettyText.cook(translated_raw)

lib/translation/post_raw_translator.rb

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -21,17 +21,17 @@ class PostRawTranslator < BaseTranslator
2121
8. Ensure the translation only contains the original language and the target language.
2222
2323
Output your translation in the following JSON format:
24-
{"translation": "Your TARGET_LANGUAGE translation here"}
24+
{"translation": "Your TARGET_LOCALE translation here"}
2525
2626
Here are three examples of correct translations:
2727
28-
Original: {"content":"New Update for Minecraft Adds Underwater Temples", "target_language":"Spanish"}
28+
Original: {"content":"New Update for Minecraft Adds Underwater Temples", "target_locale":"Spanish"}
2929
Correct translation: {"translation": "Nueva actualización para Minecraft añade templos submarinos"}
3030
31-
Original: {"content": "# Machine Learning 101\n\nMachine Learning (ML) is a subset of Artificial Intelligence (AI) that focuses on the development of algorithms and statistical models that enable computer systems to improve their performance on a specific task through experience.\n\n## Key Concepts\n\n1. **Supervised Learning**: The algorithm learns from labeled training data.\n2. **Unsupervised Learning**: The algorithm finds patterns in unlabeled data.\n3. **Reinforcement Learning**: The algorithm learns through interaction with an environment.\n\n```python\n# Simple example of a machine learning model\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.linear_model import LogisticRegression\n\n# Assuming X and y are your features and target variables\nX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n\nmodel = LogisticRegression()\nmodel.fit(X_train, y_train)\n\n# Evaluate the model\naccuracy = model.score(X_test, y_test)\nprint(f'Model accuracy: {accuracy}')\n```\n\nFor more information, visit [Machine Learning on Wikipedia](https://en.wikipedia.org/wiki/Machine_learning).", "target_language":"French"}
31+
Original: {"content": "# Machine Learning 101\n\nMachine Learning (ML) is a subset of Artificial Intelligence (AI) that focuses on the development of algorithms and statistical models that enable computer systems to improve their performance on a specific task through experience.\n\n## Key Concepts\n\n1. **Supervised Learning**: The algorithm learns from labeled training data.\n2. **Unsupervised Learning**: The algorithm finds patterns in unlabeled data.\n3. **Reinforcement Learning**: The algorithm learns through interaction with an environment.\n\n```python\n# Simple example of a machine learning model\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.linear_model import LogisticRegression\n\n# Assuming X and y are your features and target variables\nX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n\nmodel = LogisticRegression()\nmodel.fit(X_train, y_train)\n\n# Evaluate the model\naccuracy = model.score(X_test, y_test)\nprint(f'Model accuracy: {accuracy}')\n```\n\nFor more information, visit [Machine Learning on Wikipedia](https://en.wikipedia.org/wiki/Machine_learning).", "target_locale":"French"}
3232
Correct translation: {"translation": "# Machine Learning 101\n\nLe Machine Learning (ML) est un sous-ensemble de l'Intelligence Artificielle (IA) qui se concentre sur le développement d'algorithmes et de modèles statistiques permettant aux systèmes informatiques d'améliorer leurs performances sur une tâche spécifique grâce à l'expérience.\n\n## Concepts clés\n\n1. **Apprentissage supervisé** : L'algorithme apprend à partir de données d'entraînement étiquetées.\n2. **Apprentissage non supervisé** : L'algorithme trouve des motifs dans des données non étiquetées.\n3. **Apprentissage par renforcement** : L'algorithme apprend à travers l'interaction avec un environnement.\n\n```python\n# Exemple simple d'un modèle de machine learning\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.linear_model import LogisticRegression\n\n# En supposant que X et y sont vos variables de caractéristiques et cibles\nX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n\nmodel = LogisticRegression()\nmodel.fit(X_train, y_train)\n\n# Évaluer le modèle\naccuracy = model.score(X_test, y_test)\nprint(f'Model accuracy: {accuracy}')\n```\n\nPour plus d'informations, visitez [Machine Learning sur Wikipedia](https://en.wikipedia.org/wiki/Machine_learning)."}
3333
34-
Original: {"content": "**Heathrow fechado**: paralisação de voos deve continuar nos próximos dias, diz gestora do aeroporto de *Londres*", "target_language": "English"}
34+
Original: {"content": "**Heathrow fechado**: paralisação de voos deve continuar nos próximos dias, diz gestora do aeroporto de *Londres*", "target_locale": "English"}
3535
Correct translation: {"translation": "**Heathrow closed**: flight disruption expected to continue in coming days, says *London* airport management"}
3636
3737
Remember, you are being consumed via an API. Only return the translated text in the specified JSON format. Do not include any additional information or explanations in your response.

lib/translation/short_text_translator.rb

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -15,18 +15,18 @@ class ShortTextTranslator < BaseTranslator
1515
Provide your translation in the following JSON format:
1616
1717
<output>
18-
{"translation": "target_language translation here"}
18+
{"translation": "target_locale translation here"}
1919
</output>
2020
2121
Here are three examples of correct translation
2222
23-
Original: {"content":"Japan", "target_language":"Spanish"}
23+
Original: {"content":"Japan", "target_locale":"es"}
2424
Correct translation: {"translation": "Japón"}
2525
26-
Original: {"name":"Cats and Dogs", "target_language":"Chinese"}
26+
Original: {"name":"Cats and Dogs", "target_locale":"zh_CN"}
2727
Correct translation: {"translation": "猫和狗"}
2828
29-
Original: {"name": "Q&A", "target_language": "Portuguese"}
29+
Original: {"name": "Q&A", "target_locale": "pt"}
3030
Correct translation: {"translation": "Perguntas e Respostas"}
3131
3232
Remember to keep proper nouns like "Minecraft" and "Toyota" in their original form. Translate the text now and provide your answer in the specified JSON format.

lib/translation/topic_localizer.rb

Lines changed: 6 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -6,16 +6,15 @@ class TopicLocalizer
66
def self.localize(topic, target_locale = I18n.locale)
77
return if topic.blank? || target_locale.blank? || topic.locale == target_locale.to_s
88

9-
target_locale_sym = target_locale.to_s.sub("-", "_").to_sym
9+
target_locale = target_locale.to_s.sub("-", "_")
1010

11-
translated_title = TopicTitleTranslator.new(topic.title, target_locale_sym).translate
12-
translated_excerpt = ShortTextTranslator.new(topic.excerpt, target_locale_sym).translate
11+
translated_title =
12+
TopicTitleTranslator.new(text: topic.title, target_locale:, topic_id: topic.id).translate
13+
translated_excerpt =
14+
ShortTextTranslator.new(text: topic.excerpt, target_locale:, topic_id: topic.id).translate
1315

1416
localization =
15-
TopicLocalization.find_or_initialize_by(
16-
topic_id: topic.id,
17-
locale: target_locale_sym.to_s,
18-
)
17+
TopicLocalization.find_or_initialize_by(topic_id: topic.id, locale: target_locale)
1918

2019
localization.title = translated_title
2120
localization.fancy_title = Topic.fancy_title(translated_title)

lib/translation/topic_title_translator.rb

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -4,9 +4,9 @@ module DiscourseAi
44
module Translation
55
class TopicTitleTranslator < BaseTranslator
66
PROMPT_TEMPLATE = <<~TEXT.freeze
7-
You are a translation service specializing in translating forum post titles from English to the asked target_language. Your task is to provide accurate and contextually appropriate translations while adhering to the following guidelines:
7+
You are a translation service specializing in translating forum post titles from English to the asked target_locale. Your task is to provide accurate and contextually appropriate translations while adhering to the following guidelines:
88
9-
1. Translate the given title from English to target_language asked.
9+
1. Translate the given title from English to target_locale asked.
1010
2. Keep proper nouns and technical terms in their original language.
1111
3. Attempt to keep the translated title length close to the original when possible.
1212
4. Ensure the translation maintains the original meaning and tone.
@@ -15,25 +15,25 @@ class TopicTitleTranslator < BaseTranslator
1515
1616
1. Read and understand the title carefully.
1717
2. Identify any proper nouns or technical terms that should remain untranslated.
18-
3. Translate the remaining words and phrases into the target_language, ensuring the meaning is preserved.
18+
3. Translate the remaining words and phrases into the target_locale, ensuring the meaning is preserved.
1919
4. Adjust the translation if necessary to keep the length similar to the original title.
20-
5. Review your translation for accuracy and naturalness in the target_language.
20+
5. Review your translation for accuracy and naturalness in the target_locale.
2121
2222
Provide your translation in the following JSON format:
2323
2424
<output>
25-
{"translation": "Your target_language translation here"}
25+
{"translation": "Your target_locale translation here"}
2626
</output>
2727
2828
Here are three examples of correct translation
2929
30-
Original: {"title":"New Update for Minecraft Adds Underwater Temples", "target_language":"Spanish"}
30+
Original: {"title":"New Update for Minecraft Adds Underwater Temples", "target_locale":"es"}
3131
Correct translation: {"translation": "Nueva actualización para Minecraft añade templos submarinos"}
3232
33-
Original: {"title":"Toyota announces revolutionary battery technology", "target_language":"French"}
33+
Original: {"title":"Toyota announces revolutionary battery technology", "target_locale":"fr"}
3434
Correct translation: {"translation": "Toyota annonce une technologie de batteries révolutionnaire"}
3535
36-
Original: {"title": "Heathrow fechado: paralisação de voos deve continuar nos próximos dias, diz gestora do aeroporto de Londres", "target_language": "English"}
36+
Original: {"title": "Heathrow fechado: paralisação de voos deve continuar nos próximos dias, diz gestora do aeroporto de Londres", "target_locale": "en"}
3737
Correct translation: {"translation": "Heathrow closed: flight disruption expected to continue in coming days, says London airport management"}
3838
3939
Remember to keep proper nouns like "Minecraft" and "Toyota" in their original form. Translate the title now and provide your answer in the specified JSON format.

spec/lib/translation/base_translator_spec.rb

Lines changed: 7 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -10,16 +10,18 @@
1010
end
1111

1212
describe ".translate" do
13-
let(:text_to_translate) { "cats are great" }
14-
let(:target_language) { "de" }
13+
let(:text) { "cats are great" }
14+
let(:target_locale) { "de" }
1515
let(:llm_response) { "hur dur hur dur!" }
1616

1717
it "creates the correct prompt" do
1818
post_translator =
19-
DiscourseAi::Translation::PostRawTranslator.new(text_to_translate, target_language)
19+
DiscourseAi::Translation::PostRawTranslator.new(text:, target_locale:, topic_id: 1)
2020
allow(DiscourseAi::Completions::Prompt).to receive(:new).with(
2121
DiscourseAi::Translation::PostRawTranslator::PROMPT_TEMPLATE,
2222
messages: [{ type: :user, content: post_translator.formatted_content, id: "user" }],
23+
topic_id: 1,
24+
post_id: nil,
2325
).and_call_original
2426

2527
DiscourseAi::Completions::Llm.with_prepared_responses([llm_response]) do
@@ -30,8 +32,7 @@
3032
it "sends the translation prompt to the selected ai helper model" do
3133
mock_prompt = instance_double(DiscourseAi::Completions::Prompt)
3234
mock_llm = instance_double(DiscourseAi::Completions::Llm)
33-
post_translator =
34-
DiscourseAi::Translation::PostRawTranslator.new(text_to_translate, target_language)
35+
post_translator = DiscourseAi::Translation::PostRawTranslator.new(text:, target_locale:)
3536

3637
structured_output =
3738
DiscourseAi::Completions::StructuredOutput.new({ translation: { type: "string" } })
@@ -54,10 +55,7 @@
5455
it "returns the translation from the llm's response" do
5556
DiscourseAi::Completions::Llm.with_prepared_responses([llm_response]) do
5657
expect(
57-
DiscourseAi::Translation::PostRawTranslator.new(
58-
text_to_translate,
59-
target_language,
60-
).translate,
58+
DiscourseAi::Translation::PostRawTranslator.new(text:, target_locale:).translate,
6159
).to eq "hur dur hur dur!"
6260
end
6361
end

spec/lib/translation/category_localizer_spec.rb

Lines changed: 16 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -12,17 +12,17 @@
1212
def post_raw_translator_stub(opts)
1313
mock = instance_double(DiscourseAi::Translation::PostRawTranslator)
1414
allow(DiscourseAi::Translation::PostRawTranslator).to receive(:new).with(
15-
opts[:value],
16-
opts[:locale],
15+
text: opts[:text],
16+
target_locale: opts[:target_locale],
1717
).and_return(mock)
1818
allow(mock).to receive(:translate).and_return(opts[:translated])
1919
end
2020

2121
def short_text_translator_stub(opts)
2222
mock = instance_double(DiscourseAi::Translation::ShortTextTranslator)
2323
allow(DiscourseAi::Translation::ShortTextTranslator).to receive(:new).with(
24-
opts[:value],
25-
opts[:locale],
24+
text: opts[:text],
25+
target_locale: opts[:target_locale],
2626
).and_return(mock)
2727
allow(mock).to receive(:translate).and_return(opts[:translated])
2828
end
@@ -32,16 +32,20 @@ def short_text_translator_stub(opts)
3232
end
3333

3434
describe ".localize" do
35-
let(:target_locale) { :fr }
35+
let(:target_locale) { "fr" }
3636

3737
it "translates the category name and description" do
3838
translated_cat_desc = "C'est une catégorie de test"
3939
translated_cat_name = "Catégorie de Test"
4040
short_text_translator_stub(
41-
{ value: category.name, locale: target_locale, translated: translated_cat_name },
41+
{ text: category.name, target_locale: target_locale, translated: translated_cat_name },
4242
)
4343
post_raw_translator_stub(
44-
{ value: category.description, locale: target_locale, translated: translated_cat_desc },
44+
{
45+
text: category.description,
46+
target_locale: target_locale,
47+
translated: translated_cat_desc,
48+
},
4549
)
4650

4751
res = localizer.localize(category, target_locale)
@@ -54,13 +58,13 @@ def short_text_translator_stub(opts)
5458
translated_cat_desc = "C'est une catégorie de test"
5559
translated_cat_name = "Catégorie de Test"
5660
short_text_translator_stub(
57-
{ value: category.name, locale: :fr, translated: translated_cat_name },
61+
{ text: category.name, target_locale:, translated: translated_cat_name },
5862
)
5963
post_raw_translator_stub(
60-
{ value: category.description, locale: :fr, translated: translated_cat_desc },
64+
{ text: category.description, target_locale:, translated: translated_cat_desc },
6165
)
6266

63-
res = localizer.localize(category, "fr")
67+
res = localizer.localize(category, target_locale)
6468

6569
expect(res.name).to eq(translated_cat_name)
6670
expect(res.description).to eq(translated_cat_desc)
@@ -79,10 +83,10 @@ def short_text_translator_stub(opts)
7983
translated_cat_desc = "C'est une catégorie de test"
8084
translated_cat_name = "Esta es una categoría de prueba"
8185
short_text_translator_stub(
82-
{ value: category.name, locale: :es, translated: translated_cat_name },
86+
{ text: category.name, target_locale: "es", translated: translated_cat_name },
8387
)
8488
post_raw_translator_stub(
85-
{ value: category.description, locale: :es, translated: translated_cat_desc },
89+
{ text: category.description, target_locale: "es", translated: translated_cat_desc },
8690
)
8791

8892
res = localizer.localize(category)

0 commit comments

Comments
 (0)