Markdown fix, more coverage

Author: pamelafox

CONTRIBUTING.md

@@ -72,8 +72,8 @@ python -m pytest --snapshot-update
 Once tests are passing, generate a coverage report to make sure your changes are covered:
 
 ```shell
-pytest --cov --cov-report=xml
-diff-cover coverage.xml --format html:coverage_report.html
+pytest --cov --cov-report=xml && \
+diff-cover coverage.xml --format html:coverage_report.html && \
 open coverage_report.html
 ```
 

docs/customization.md

@@ -9,11 +9,9 @@ This guide provides more details for customizing the RAG chat app.
 - [Using your own data](#using-your-own-data)
 - [Customizing the UI](#customizing-the-ui)
 - [Customizing the backend](#customizing-the-backend)
-  - [Chat/Ask tabs](#chatask-tabs)
+  - [Chat/Ask approaches](#chatask-approaches)
     - [Chat approach](#chat-approach)
-      - [Chat with vision](#chat-with-vision)
-    - [Ask tab](#ask-tab)
-      - [Ask with vision](#ask-with-vision)
+    - [Ask approach](#ask-approach)
 - [Improving answer quality](#improving-answer-quality)
   - [Identify the problem point](#identify-the-problem-point)
   - [Improving OpenAI ChatCompletion results](#improving-openai-chatcompletion-results)
@@ -32,7 +30,7 @@ The frontend is built using [React](https://reactjs.org/) and [Fluent UI compone
 
 The backend is built using [Quart](https://quart.palletsprojects.com/), a Python framework for asynchronous web applications. The backend code is stored in the `app/backend` folder. The frontend and backend communicate over HTTP using JSON or streamed NDJSON responses. Learn more in the [HTTP Protocol guide](http_protocol.md).
 
-### Chat/Ask tabs
+### Chat/Ask approaches
 
 Typically, the primary backend code you'll want to customize is the `app/backend/approaches` folder, which contains the classes powering the Chat and Ask tabs. Each class uses a different RAG (Retrieval Augmented Generation) approach, which include system messages that should be changed to match your data
 
@@ -55,7 +53,7 @@ there are several differences in the chat approach:
 2. **Search**: For this step, it also calculates a vector embedding for the user question using [the Azure AI Vision vectorize text API](https://learn.microsoft.com/azure/ai-services/computer-vision/how-to/image-retrieval#call-the-vectorize-text-api), and passes that to the Azure AI Search to compare against the image embedding fields in the indexed documents. For each matching document, it downloads each associated image from Azure Blob Storage and converts it to a base 64 encoding.
 3. **Answering**: When it combines the search results and user question, it includes the base 64 encoded images, and sends along both the text and images to the multimodal LLM. The model generates a response that includes citations to the images, and the UI renders the images when a citation is clicked.
 
-#### Ask tab
+#### Ask approach
 
 The ask tab uses the approach programmed in [retrievethenread.py](https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/app/backend/approaches/retrievethenread.py).
 

tests/conftest.py

@@ -1083,28 +1083,6 @@ def mock_readinto(self, stream: IO[bytes]):
     monkeypatch.setattr(azure.storage.filedatalake.aio.StorageStreamDownloader, "readinto", mock_readinto)
 
 
-# Add a mock token_provider for tests
-@pytest.fixture
-def mock_token_provider():
-    async def dummy_token_provider():
-        return "dummy_token"
-
-    return dummy_token_provider
-
-
-@pytest.fixture(autouse=True)
-def patch_get_bearer_token_provider(monkeypatch, mock_token_provider):
-    """
-    Patch the get_bearer_token_provider function used in app.py to return our mock_token_provider.
-    This is automatically applied to all tests.
-    """
-
-    def mock_get_bearer_token(*args, **kwargs):
-        return mock_token_provider
-
-    monkeypatch.setattr("azure.identity.aio.get_bearer_token_provider", mock_get_bearer_token)
-
-
 @pytest.fixture
 def chat_approach():
     return ChatReadRetrieveReadApproach(

tests/mocks.py

@@ -7,7 +7,6 @@
 import openai.types
 from azure.cognitiveservices.speech import ResultReason
 from azure.core.credentials_async import AsyncTokenCredential
-from azure.core.exceptions import ResourceNotFoundError
 from azure.core.pipeline.transport import (
     AioHttpTransportResponse,
     AsyncHttpTransport,
@@ -92,24 +91,18 @@ def __init__(self, url, body_bytes, headers=None):
 
 class MockTransport(AsyncHttpTransport):
     async def send(self, request: HttpRequest, **kwargs) -> AioHttpTransportResponse:
-        if request.url.endswith("notfound.png"):
-            raise ResourceNotFoundError(MockAiohttpClientResponse404(request.url, b""))
-        else:
-            return AioHttpTransportResponse(
-                request,
-                MockAiohttpClientResponse(
-                    request.url,
-                    b"test content",
-                    {
-                        "Content-Type": "application/octet-stream",
-                        "Content-Range": "bytes 0-27/28",
-                        "Content-Length": "28",
-                    },
-                ),
-            )
-
-    async def __aenter__(self):
-        return self
+        return AioHttpTransportResponse(
+            request,
+            MockAiohttpClientResponse(
+                request.url,
+                b"test content",
+                {
+                    "Content-Type": "application/octet-stream",
+                    "Content-Range": "bytes 0-27/28",
+                    "Content-Length": "28",
+                },
+            ),
+        )
 
     async def __aexit__(self, *args):
         pass

tests/test_pdfparser.py

@@ -21,7 +21,10 @@
 from azure.core.exceptions import HttpResponseError
 from PIL import Image, ImageChops
 
-from prepdocslib.mediadescriber import ContentUnderstandingDescriber
+from prepdocslib.mediadescriber import (
+    ContentUnderstandingDescriber,
+    MultimodalModelDescriber,
+)
 from prepdocslib.page import ImageOnPage
 from prepdocslib.pdfparser import DocumentAnalysisParser, MediaDescriptionStrategy
 
@@ -386,3 +389,78 @@ async def mock_poller_result():
     assert pages[0].page_num == 0
     assert pages[0].offset == 0
     assert pages[0].text == "Page content"
+
+
+@pytest.mark.asyncio
+async def test_parse_doc_with_openai(monkeypatch):
+    mock_poller = MagicMock()
+
+    async def mock_begin_analyze_document(self, model_id, analyze_request, **kwargs):
+        return mock_poller
+
+    async def mock_poller_result():
+        content = open(TEST_DATA_DIR / "Simple Figure_content.txt").read()
+        return AnalyzeResult(
+            content=content,
+            pages=[DocumentPage(page_number=1, spans=[DocumentSpan(offset=0, length=148)])],
+            figures=[
+                DocumentFigure(
+                    id="1.1",
+                    caption=DocumentCaption(content="Figure 1"),
+                    bounding_regions=[
+                        BoundingRegion(
+                            page_number=1, polygon=[0.4295, 1.3072, 1.7071, 1.3076, 1.7067, 2.6088, 0.4291, 2.6085]
+                        )
+                    ],
+                    spans=[DocumentSpan(offset=70, length=22)],
+                )
+            ],
+        )
+
+    monkeypatch.setattr(DocumentIntelligenceClient, "begin_analyze_document", mock_begin_analyze_document)
+    monkeypatch.setattr(mock_poller, "result", mock_poller_result)
+
+    async def mock_describe_image(self, image_bytes):
+        return "Pie chart"
+
+    monkeypatch.setattr(MultimodalModelDescriber, "describe_image", mock_describe_image)
+
+    parser = DocumentAnalysisParser(
+        endpoint="https://example.com",
+        credential=MockAzureCredential(),
+        media_description_strategy=MediaDescriptionStrategy.OPENAI,
+        openai_client=Mock(),
+        openai_model="gpt-4o",
+        openai_deployment="gpt-4o",
+    )
+
+    with open(TEST_DATA_DIR / "Simple Figure.pdf", "rb") as f:
+        content = io.BytesIO(f.read())
+        content.name = "Simple Figure.pdf"
+
+    pages = [page async for page in parser.parse(content)]
+
+    assert len(pages) == 1
+    assert pages[0].page_num == 0
+    assert pages[0].offset == 0
+    assert (
+        pages[0].text
+        == "# Simple Figure\n\nThis text is before the figure and NOT part of it.\n\n\n<figure><figcaption>1.1 Figure 1<br>Pie chart</figcaption></figure>\n\n\nThis is text after the figure that's not part of it."
+    )
+
+
+@pytest.mark.asyncio
+async def test_parse_doc_with_openai_missing_parameters():
+    parser = DocumentAnalysisParser(
+        endpoint="https://example.com",
+        credential=MockAzureCredential(),
+        media_description_strategy=MediaDescriptionStrategy.OPENAI,
+        # Intentionally not providing openai_client and openai_model
+    )
+
+    content = io.BytesIO(b"pdf content bytes")
+    content.name = "test.pdf"
+
+    with pytest.raises(ValueError, match="OpenAI client must be provided when using OpenAI media description strategy"):
+        # Call the first iteration of the generator without using async for
+        await parser.parse(content).__anext__()