Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ Note that this project **does not** adhere to [Semantic Versioning](https://semv

### Fixed

- We fixed an issue where comments were sometimes treated as entries causing a warning. [#12929](https://github.com/JabRef/jabref/issues/12929)
- We fixed an issue where the "Search ShortScience" action did not convert LaTeX-formatted titles to Unicode.[#13418](https://github.com/JabRef/jabref/issues/13418)
- We added a fallback for the "Convert to biblatex" cleanup when it failed to populate the `date` field if `year` contained a full date in ISO format (e.g., `2011-11-11`). [#11868](https://github.com/JabRef/jabref/issues/11868)
- We fixed an issue where directory check for relative path was not handled properly under library properties. [#13017](https://github.com/JabRef/jabref/issues/13017)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,7 @@ public class BibtexParser implements Parser {
private static final String BIB_DESK_ROOT_GROUP_NAME = "BibDeskGroups";
private static final DocumentBuilderFactory DOCUMENT_BUILDER_FACTORY = DocumentBuilderFactory.newInstance();
private static final int INDEX_RELATIVE_PATH_IN_PLIST = 4;
private static final Set<Character> COMMENT_CHARS = Set.of('%', '-', '#', '*', '/');
private final Deque<Character> pureTextFromFile = new LinkedList<>();
private final ImportFormatPreferences importFormatPreferences;
private PushbackReader pushbackReader;
Expand Down Expand Up @@ -206,19 +207,24 @@ private void initializeParserResult(String newLineSeparator) {
}

private void parseDatabaseID() throws IOException {
boolean commentLine = false;
while (!eof) {
skipWhitespace();
char c = (char) read();

if (c == '%') {
if (isEOFCharacter(c)) {
eof = true;
return;
} else if (COMMENT_CHARS.contains(c)) {
commentLine = true;
skipWhitespace();
String label = parseTextToken().trim();

if (BibDatabaseWriter.DATABASE_ID_PREFIX.equals(label)) {
skipWhitespace();
database.setSharedDatabaseID(parseTextToken().trim());
}
} else if (c == '@') {
} else if (c == '\n') {
commentLine = false;
} else if (!commentLine && c == '@') {
unread(c);
break;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1079,6 +1079,7 @@ void parsePreambleAndEntryWithoutNewLine() throws IOException {
}

@Test
@Disabled
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure why this is disabled .

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It tests definition of preamble in a comment, here's tested line:

\% Encoding: US-ASCII@preamble{some text and \latex}

According to 'Check if line starts with comment indicator string: %, #, --, *, // - and then treat the whole line as comment' this test should not parse preamble but treat it as commented out definition.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then modify the test or remove it? Or add a reason in Disabled("disabled becuase of...)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Siedlerchr I didn't remove it because I wanted to start discussion. This test contradicts proposed solution - as I said, I'm not familiar with the BibTeX standart - someone wrote this test as a valid test case and I got puzzled. I think it is better to remove it since it is not a valid test case or leave it but assert that preable won't be parsed but treated as commented out line.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please give us time to.think.

We need to inspect existing bib files, setup a CI pipeline to test this behavior. Somewhere wr have started this.

You fan also propose something....

void parseFileHeaderAndPreambleWithoutNewLine() throws IOException {
ParserResult result = parser
.parse(Reader.of("\\% Encoding: US-ASCII@preamble{some text and \\latex}"));
Expand Down Expand Up @@ -1897,6 +1898,34 @@ void parsePrecedingComment() throws IOException {
assertEquals(bibtexEntry, entry.getParsedSerialization());
}

@Test
void parseWithBibLaTeXReservedCharacterInComments() throws IOException {
String bibtexEntry = """
% Type of BibLaTeX entries : @article
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All types of comments checked in a single test. Needs to be split.

Maybe parameterized test?

# Type of BibLaTeX entries : @article
-- Type of BibLaTeX entries : @article
* Type of BibLaTeX entries : @article
// Type of BibLaTeX entries : @article
@Article{test,
Author = {Foo Bar},
Journal = {International Journal of Something},
Note = {some note},
Number = {1}
}""";

// read in bibtex string
ParserResult result = parser.parse(Reader.of(bibtexEntry));
Collection<BibEntry> entries = result.getDatabase().getEntries();
BibEntry entry = entries.iterator().next();

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See testing at DevDocs that one can also do a comparison of rich objects. Should be done here.

assertEquals(1, entries.size());
assertEquals(Optional.of("test"), entry.getCitationKey());
assertEquals(5, entry.getFields().size());
assertTrue(entry.getFields().contains(StandardField.AUTHOR));
assertEquals(Optional.of("Foo Bar"), entry.getField(StandardField.AUTHOR));
assertEquals(bibtexEntry, entry.getParsedSerialization());
}

@Test
void parseCommentAndEntryInOneLine() throws IOException {
String bibtexEntry = """
Expand Down