Search in browse by title does not work when the title starts with "A", "As", "O", "Os" and the dc.title contains certain values in text_lang etc...

[IgorBaptist4]

Describe the bug

This bug can be reproduced in versions 7.x, 9.x, and probably 8.x as well (I have not tested it in 8.x).

If a title starts with "A", "As", "O", or "Os" (for example: "O papel...", "A birth...") and the dc.title field contains a text_lang value like "pt_BR", "por", or "en_US", then when browsing by title and searching for the item's title, the item is not found.
However, if text_lang is changed to "pt-BR", "en" or left empty, the item can be found when browsing by title and searching for the title.

To Reproduce

Steps to reproduce the behavior:

1. Log in and deposit an item with a title like "O papel...", "A birth...", etc. (Or just find an item that fits the requirements and proceed).
2. Edit the item, adding a value to the Lang field of dc.title, such as "pt_BR", "por", etc.
3. Go to "Browse by Title" and search for the title of the deposited item.

Expected behavior

If the Lang value of dc.title contains a form like "pt_BR", "por", or similar, and the title starts with "A", "E", "I", "O", "U", or "As", the item will not show up when using the search of "Browse by Title".

Proofs

Item created in sandbox to test it: https://sandbox.dspace.org/items/bdaf0451-b966-4008-8259-f55155224f95
An item already in the sandbox that matches the bug: https://sandbox.dspace.org/entities/publication/bb399a20-8fb5-4f19-9bdc-d9bc23cbd04f/full

When lang of dc.title is "por" (ISO 639-2 for portuguese)

When lang of dc.title is "pt_BR"

When lang of dc.title is "pt"

When lang of dc.title is "en" (works as expected)

[tdonohue]

@IgorBaptist4 : Thanks for reporting this. I can reproduce it on the Sandbox site.

Here's an Item with a title that starts with "A" which works: "A New Methodology for Assessing..."

If you search for "A New Methodology" in the Browse by Title, it will show up: https://sandbox.dspace.org/browse/title?bbm.page=1&startsWith=A%20New%20Methodology

However, as you noted, here's a title that starts with "A" which DOESN'T work: "A birth cohort study of viral infections.."

If you search for "A birth cohort" in the Browse by Title, it will NOT show up: https://sandbox.dspace.org/browse/title?bbm.page=1&startsWith=A%20birth%20cohort

However, if you remove the "A" and search "birth cohort", then it DOES show up: https://sandbox.dspace.org/browse/title?bbm.page=1&startsWith=birth%20cohort

I'm not sure of the exact cause, but it definitely seems to have something to do with the language being specified. I wonder if it's because "a" (and "as" and "the") is a stopword (ignored word) in Solr when using English? I'm not sure.

Regardless, this needs a volunteer to investigate further and see if we can find a way to fix this behavior.

[jesielviana]

Based on my research into how stopwords work in Lucene/Solr, it's possible to configure a different set of stopwords for each language. However, in DSpace, we currently have only one stopword list configured, which is for English. You can find it here:
👉 https://github.com/DSpace/DSpace/blob/main/dspace/solr/search/conf/stopwords.txt

This means the same stopword list is applied to all languages. In the tests I performed, when the dc.title field has a valid language code (e.g., pt, en, or es), the words included in the stopwords list are ignored. However, if the language is left empty, the stopwords are not applied, and titles starting with stopwords can be found correctly.

As in the example shared by @tdonohue:

Searching for "A New Methodology" returns the item:
https://sandbox.dspace.org/browse/title?bbm.page=1&startsWith=A%20New%20Methodology
(This item has an empty dc.title language)

Searching for "A birth cohort" does not return the item https://sandbox.dspace.org/entities/publication/bb399a20-8fb5-4f19-9bdc-d9bc23cbd04f (This item has dc.title language set to "en")
search: https://sandbox.dspace.org/browse/title?bbm.page=1&startsWith=A%20birth%20cohort

So the issue seems to be related to how stopwords are applied, regardless of the dc.title language field.

@IgorBaptist4 If you're interested, I'm available for a quick chat to better understand the problem together.