#429: Added a peek function to the factory that decides whether .xml files are ReqM2 files (#469)

#429: Added a peek function to the factory that decides whether .xml files are ReqM2 files

Co-authored-by: Christoph Pirkl <[email protected]>

Author: redcatbear

.github/workflows/broken_links_checker.yml

@@ -15,7 +15,15 @@ jobs:
       - name: Configure broken links checker
         run: |
           mkdir -p ./target
-          echo '{ "aliveStatusCodes": [429, 200] }' > ./target/broken_links_checker.json
+          echo '{
+            "aliveStatusCodes": [429, 200],
+            "ignorePatterns": [
+              {
+                # The Autosar TLS certificate cannot be validated from GitHub Actions.
+                "pattern": "^https://www.autosar.org/"
+              }
+            ]
+          }' > ./target/broken_links_checker.json
       - uses: tcort/github-action-markdown-link-check@v1
         with:
           use-quiet-mode: "yes"

doc/changes/changes.md

@@ -1,5 +1,6 @@
 # Changes
 
++ [4.2.1](changes_4.2.1.md)
 * [4.2.0](changes_4.2.0.md)
 * [4.1.0](changes_4.1.0.md)
 * [4.0.2](changes_4.0.2.md)

doc/changes/changes_4.2.1.md

@@ -0,0 +1,11 @@
+# OpenFastTrace 4.2.1, released 2025-09-14
+
+Code name: Peek before you guess
+
+## Summary
+
+This is a bugfix release. It addresses the problem that ReqM2 uses a generic `.xml` file suffix which led OpenFastTrace to treat any XML file as a ReqM2 file based solely on the extension. To avoid false detections, this version adds a peek function to the file type detection so that we no longer rely only on the file type/extension (#429).
+
+## Bugfixes
+
+* #429: Add content peek to file type detection so `.xml` no longer implies ReqM2; prevents misclassification of arbitrary XML files

doc/spec/design.md

@@ -7,7 +7,7 @@
 
 ## Acknowledgments
 
-This documents structure is derived from the "[arc42][bib.arc42]" architectural template by Dr. Gernot Starke, Dr. Peter Hruschka.
+This document's structure is derived from the "[arc42][bib.arc42]" architectural template by Dr. Gernot Starke, Dr. Peter Hruschka.
 
 If you build your own modifications based on this document, please keep the attrbiutions.
 
@@ -174,9 +174,25 @@ Needs: impl, itest
 
 ## Import
 
-Depending on the source format a variety of [importers](#importers) takes care of reading the input [specification items](#specification-item). Each importer emits events which an [import event listener](#import-event-listener) consumes.
+Depending on the source format, a variety of [importers](#importers) takes care of reading the input [specification items](#specification-item). Each importer emits events which an [import event listener](#import-event-listener) consumes.
 
-Common parts of the import like filtering out unnecessary items or attributes are handled by the listener.
+The listener handles Common parts of the import like filtering out unnecessary items or attributes.
+
+A factory for importers decides which importer to use. Usually, by file extension.
+
+### ReqM2 File Detection
+`dsn~import.reqm2-file-detection~1`
+
+The `SpecobjectImporterFactory` detects ReqM2 files either
+
+1. via the file extension `.oreqm` or
+2. via the file extension `.xml` and the presence of the string `<specdocument` within the first 4096 bytes of the file.
+
+Covers:
+
+* `req~import.reqm2-file-detection~1`
+
+Needs: impl, utest
 
 ### Selective Artifact Type Import
 

doc/spec/system_requirements.md

@@ -255,7 +255,7 @@ Needs: dsn
 
 #### Common Requirements for Lightweight Markup Import
 
-Typical OFT specification are written in a lightweight markup language like [Markdown](#markdown-import) or [ReStructured Text](#restructured-text-rst-import). Before we go into the specifics, this section discusses the common requirements.
+Typical OFT specifications are written in a lightweight markup language like [Markdown](#markdown-import) or [ReStructured Text](#restructured-text-rst-import). Before we go into the specifics, this section discusses the common requirements.
 
 ##### Disabling OFT Parsing for Parts of a Markup File
 `req~disabling-oft-parsing-for-parts-of-a-markup-file~1`
@@ -295,7 +295,7 @@ Markdown focuses on content over formatting by giving the document structure lik
 
 OFT defines a Markdown format that we call "Requirement-Enhanced Markdown" which is a superset of the regular Markdown. Any Markdown renderer can render this format without understanding it. The additional structural definitions tell OFT which part of the text is a specification item.
 
-For backward compatibility OFT supports a variant of this format that was introduced at Elektrobit. This format is a little bit closer to ReqM2, the predecessor that sparked the OFT idea. We recommend using standard OFT Markdown format in new documents though since this format is cleaner.
+For backward compatibility OFT supports a variant of this format that was introduced at Elektrobit. This format is a little bit closer to ReqM2, the predecessor that sparked the OFT idea. We recommend using the standard OFT Markdown format in new documents, though, since this format is cleaner.
 
 ##### Markdown Standard Syntax
 `req~markdown-standard-syntax~1`
@@ -319,7 +319,7 @@ The Markdown outline -- a table of contents created from the heading structure b
 
 Rationale:
 
-In long specification document the outline is the primary means of navigating the document. Only if the outline can be read easily, it is useful for authoring specification documents.
+In long specification documents the outline is the primary means of navigating the document. Only if the outline can be read easily, it is useful for authoring specification documents.
 
 Covers:
 
@@ -368,6 +368,24 @@ Covers:
 
 Needs: dsn
 
+#### ReqM2 Format
+
+ReqM2 is a markup format developed at Elektrobit. ReqM2 files traditionally use the `.xml` file extension, a better way is to use `.oreqm` instead since that identifies the files uniquely.
+
+##### ReqM2 File Detection
+`req~import.reqm2-file-detection~1`
+
+OFT considers a file to be a ReqM2 file if it either
+
+1. has a the `.oreqm` extension or
+2. has a the `.xml` extension and the header indicates that it is a ReqM2 file.
+
+Covers:
+
+* [feat~reqm2-import~1](#reqm2-import)
+
+Needs: dsn
+
 ### Tracing
 
 #### Outgoing Coverage Link Status
@@ -467,13 +485,13 @@ Usually the responsibility of document authors or coders when it comes to tracin
 
 If the users try to run a regular trace without feeding in the artifacts all the way to the bottom level of the tracing chain, the coverage check will always report errors because of missing lower level coverage.
 
-To mitigate the situation OFT allows users to ignore required coverage for selected artifact types.
+To mitigate the situation, OFT allows users to ignore required coverage for selected artifact types.
 
 Example:
 
-Kim is a software architect and it is her job to cover the system requirements coming from Steve in her software architecture. Kim wants to make sure she did not forget to cover a system requirement and uses OFT to trace the two documents. The system requirement specification uses the artifact types `feat` and `req` where `req` covers the `feat` artifacts in the same document. Kim's architecture uses the artifact type `sysarch` which covers `req` and requires a detailed design `dsn`.
+Kim is a software architect, and it is her job to cover the system requirements coming from Steve in her software architecture. Kim wants to make sure she did not forget to cover a system requirement and uses OFT to trace the two documents. The system requirement specification uses the artifact types `feat` and `req` where `req` covers the `feat` artifacts in the same document. Kim's architecture uses the artifact type `sysarch` which covers `req` and requires a detailed design `dsn`.
 
-Obviously the detailed design is missing at the point when Kim runs the trace. To mitigate this situation Kim configures OFT to ignore all artifacts of type `dsn`, including the needed coverage. This allows Kim to validate coverage towards the system requirement without needing the detailed design document.
+Obviously, the detailed design is missing at the point when Kim runs the trace. To mitigate this situation, Kim configures OFT to ignore all artifacts of type `dsn`, including the needed coverage. This allows Kim to validate coverage towards the system requirement without needing the detailed design document.
 
 #### Include Only Artifact Types
 `req~include-only-artifact-types~1`

importer/specobject/src/main/java/org/itsallcode/openfasttrace/importer/specobject/SpecobjectImporterFactory.java

@@ -1,25 +1,71 @@
 package org.itsallcode.openfasttrace.importer.specobject;
 
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.util.Locale;
+import java.util.logging.Logger;
+
 import org.itsallcode.openfasttrace.api.importer.*;
 import org.itsallcode.openfasttrace.api.importer.input.InputFile;
 import org.itsallcode.openfasttrace.importer.xmlparser.XmlParserFactory;
 
 /**
- * An {@link ImporterFactory} for XML specobject files.
+ * An {@link ImporterFactory} for ReqM2/SpecObject XML files.
  */
-public class SpecobjectImporterFactory extends RegexMatchingImporterFactory
+public class SpecobjectImporterFactory extends ImporterFactory
 {
+    private static final Logger LOG = Logger.getLogger(SpecobjectImporterFactory.class.getName());
+    private static final int PEEK_CHARS = 4096;
+
     private final XmlParserFactory xmlParserFactory;
 
     /**
      * Create a new instance.
      */
     public SpecobjectImporterFactory()
     {
-        super("(?i).*\\.(xml|oreqm)");
         this.xmlParserFactory = new XmlParserFactory();
     }
 
+    // [impl -> dsn~import.reqm2-file-detection~1]
+    @Override
+    public boolean supportsFile(final InputFile file)
+    {
+        final String path = file.getPath();
+        final String lower = path.toLowerCase(Locale.ROOT);
+        if (lower.endsWith(".oreqm"))
+        {
+            return true;
+        }
+        else if (lower.endsWith(".xml"))
+        {
+            return doesFileContainOreqmHeader(file, path);
+        } else {
+            return false;
+        }
+    }
+
+    private static boolean doesFileContainOreqmHeader(final InputFile file, final String path) {
+        try (BufferedReader reader = file.createReader())
+        {
+            final char[] buf = new char[PEEK_CHARS];
+            final int read = reader.read(buf);
+            if (read <= 0)
+            {
+                return false;
+            }
+            else {
+                final String header = new String(buf, 0, read);
+                return header.contains("<specdocument");
+            }
+        }
+        catch (final IOException exception)
+        {
+            LOG.fine(() -> "Unable to peek XML file '" + path + "' trying to determine if it contains ReqM2 format: " + exception.getMessage());
+            return false;
+        }
+    }
+
     @Override
     public Importer createImporter(final InputFile file, final ImportEventListener listener)
     {

importer/specobject/src/test/java/org/itsallcode/openfasttrace/importer/specobject/TestSpecobjectImporterFactory.java

@@ -1,10 +1,21 @@
 package org.itsallcode.openfasttrace.importer.specobject;
 
 import static java.util.Arrays.asList;
+import static org.hamcrest.MatcherAssert.assertThat;
+import static org.hamcrest.Matchers.equalTo;
+import static org.mockito.Mockito.when;
 
+import java.io.IOException;
+import java.nio.file.Files;
+import java.nio.file.Path;
 import java.util.List;
 
+import org.itsallcode.openfasttrace.api.importer.input.InputFile;
+import org.itsallcode.openfasttrace.api.importer.input.RealFileInput;
 import org.itsallcode.openfasttrace.testutil.importer.ImporterFactoryTestBase;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.io.TempDir;
+import org.mockito.Mock;
 
 /**
  * Tests for {@link SpecobjectImporterFactory}
@@ -19,16 +30,73 @@ protected SpecobjectImporterFactory createFactory()
         return new SpecobjectImporterFactory();
     }
 
+    /**
+     * Only the {@code .oreqm} extension is always supported. That is why we
+     * can't simply list {@code .xml} here. Whether {@code .xml} is supported
+     * depends on a successful peek that confirms the file is in ReqM2 format.
+     * 
+     * @return list of files that need to pass the test for supported files
+     */
     @Override
     protected List<String> getSupportedFilenames()
     {
-        return asList("file.xml", "file.XML", "FILE.xml", "FILE.XML", "file.md.xml");
+        return asList("file.oreqm", "file.OREQM");
     }
 
     @Override
     protected List<String> getUnsupportedFilenames()
     {
         return asList("file.md", "file.xm", "file.ml", "file.1xml", "file.xml1", "file.xml.md",
-                "file_xml", "filexml");
+                "file_xml", "filexml", "file_oreqm", "fileoreqm");
+    }
+
+    // [utest ->dsn~import.reqm2-file-detection~1]
+    @Test
+    void supportsXmlIfContentLooksLikeReqM2(@TempDir final Path tempDir) throws IOException
+    {
+        final Path tempFile = tempDir.resolve("reqm2-specdocument.xml");
+        Files.writeString(tempFile, """
+                <?xml version="1.0"?>
+                <specdocument>
+                     <specobjects doctype="REQ"/>
+                </specdocument>
+                """);
+        final boolean supported = createFactory().supportsFile(RealFileInput.forPath(tempFile));
+        assertThat(supported, equalTo(true));
+    }
+
+    // [utest ->dsn~import.reqm2-file-detection~1]
+    @Test
+    void doesNotSupportXmlIfContentIsGeneric(@TempDir final Path tempDir) throws IOException
+    {
+        final Path tempFile = tempDir.resolve("generic.xml");
+        Files.writeString(tempFile, """
+                <?xml version="1.0"?>
+                <root>
+                    <child/>
+                </root>"
+                """);
+        final boolean supported = createFactory().supportsFile(RealFileInput.forPath(tempFile));
+        assertThat(supported, equalTo(false));
+    }
+
+    // [utest ->dsn~import.reqm2-file-detection~1]
+    @Test
+    void givenEmptyFileWhenCheckingReqM2HeaderThenDoesNotClaimSupport(@TempDir final Path tempDir) throws IOException
+    {
+        final Path tempFile = tempDir.resolve("empty.xml");
+        Files.writeString(tempFile, "");
+        final boolean supported = createFactory().supportsFile(RealFileInput.forPath(tempFile));
+        assertThat(supported, equalTo(false));
+    }
+
+    // [utest ->dsn~import.reqm2-file-detection~1]
+    @Test
+    void givenFileWhenIOExceptionOccursThenDoesNotClaimSupport(@Mock InputFile mockInputFile) throws IOException
+    {
+        when(mockInputFile.getPath()).thenReturn("/irrelevant/path/to/file.xml");
+        when(mockInputFile.createReader()).thenThrow(new IOException("This is an expected test exception"));
+        final boolean supported = createFactory().supportsFile(mockInputFile);
+        assertThat(supported, equalTo(false));
     }
 }

parent/pom.xml

@@ -10,7 +10,7 @@
     <description>Free requirement tracking suite</description>
     <url>https://github.com/itsallcode/openfasttrace</url>
     <properties>
-        <revision>4.2.0</revision>
+        <revision>4.2.1</revision>
         <java.version>17</java.version>
         <junit.version>5.12.2</junit.version>
         <maven.surefire.version>3.5.3</maven.surefire.version>