Add Base64 encoded screenshot and additional screenshot creation from XPATH element (#194)

* Update .gitignore, add VSCode directory and .venv

* Add base64 logging function to robotlog

* Change screenshot module behaviour to accomodate base64 image conversion

- Add xpath to ValueContainer
- Add additional actions for element and Base64 format
- add automation to init
- Add capture function for base64 using MemoryStream
- Add optional xpath to both capture functions

* Change screenshot keywords to use optional base64 and allow capturing by XPATh

- Adjust 'take screenshot' keyword with optional base64 boolean
- Add keyword 'take screenshot from element' for capturing element with xpath

* Fix multiple build errors

- Fix initialization error due to missing automation parameter
- fix line too long
- Fix build errors, add ignore for ImageFormat
- Remove ScreenshotType
- Remove unused Optional import

* Add screenshot mode, Adjust parameter to retrieved element, execute capture based on mode

* Add AutomationInterfaceContainer, Combine capture into single keyword with optional XPATH, Add set mode keyword

* Remove unused imports

* Change example text

* Add simple testcases for new screenshot keyword behaviour

* Fix init to use container for screenshot keywords, remove container from Screenshot module

* Fix ScreenshotMode not defined (missing self)

* add getter for screenshot mode, Fix linting
errors/warnings, adjust docstrings

* Update changelog

* Fix missing default value for element in _capture_base64

* Improve screenshot testcases for base64 mode and xpath

* Fix argument name typo

* Tiny fix of take_screenshot docstring to explain identifier

* Fix AttributeError due to improper key access

* Add truthy checks in base64 screenshot tests before checking length

* Remove redundant None, explicit msg argument for get_element

* Add return statement to _capture function

* Fix check for None (wrong keyword used)

* Fix element assigned to wrong argument for screenshot action

* Remove redundant screenshot logging in keyword

* Fix filename for screenshot in test

* Fix Screenshot.robot (reorder test cases, deactivate Base64)

* Update Screenshot.robot (Minor format adjustment)

* Add getter keyword for screenshot log mode

* Add teardown mechanism with default state for all screenshot cases

* Add missing PID for reset to properly close applications

* Update .gitignore for custom keen.bat file in development

* Fix pylint errors to get them good grades

* Fix missing trailing blank line

* Fix via robotidy

Author: HackXIt

.gitignore

@@ -1,3 +1,13 @@
+# Custom keen.bat for development
+keen.dev.bat
+
+# Python environment
+.venv
+
+# VSCode IDE files
+.vscode
+tmp/
+
 ## Ignore Visual Studio temporary files, build results, and
 ## files generated by popular Visual Studio add-ons.
 

CHANGELOG.md

@@ -7,6 +7,10 @@ This document follows the conventions laid out in [Keep a CHANGELOG][].
 
 ## [Unreleased][]
 
+### Added
+- [#192](https://github.com/GDATASoftwareAG/robotframework-flaui/issues/192) New Screenshot Logging behaviour using Base64 encoded images
+- [#193](https://github.com/GDATASoftwareAG/robotframework-flaui/issues/193) Changed Screenshot keyword behaviour (Optional XPATH for element-based screenshot)
+
 ## [Release][3.3.0] [3.3.0][3.2.0-3.3.0] - 2024-08-03
 
 ### Added

atests/Screenshot.robot

@@ -8,6 +8,7 @@ Library             String
 Library             OperatingSystem
 Library             StringFormat
 Library             FlaUILibrary    uia=${UIA}    screenshot_on_failure=True
+Resource            util/Common.resource
 Resource            util/Error.resource
 Resource            util/XPath.resource
 
@@ -24,7 +25,7 @@ Take No Screenshot If Module Is Disabled
     Take Screenshots On Failure    False
     Run Keyword And Expect Error    ${EXP_ERR_MSG}    Click    ${XPATH_NOT_EXISTS}
     File Should Not Exist    ${OUTPUT DIR}/${SCREENSHOT_FOLDER}/${FILENAME}
-    Take Screenshots On Failure    True
+    [Teardown]    Reset Screenshot Environment To Default
 
 Take Screenshot If XPath Not Found Multiple Times Default Folder
     FOR    ${_}    IN RANGE    1    3
@@ -33,6 +34,7 @@ Take Screenshot If XPath Not Found Multiple Times Default Folder
         Run Keyword And Expect Error    ${EXP_ERR_MSG}    Click    ${XPATH_NOT_EXISTS}
         Wait Until Created    ${OUTPUT DIR}/${FILENAME}    1s
     END
+    [Teardown]    Reset Screenshot Environment To Default
 
 Take Screenshot If XPath Not Found Multiple Times By Specific Folder
     Set Screenshot Directory    ${SCREENSHOT_FOLDER}
@@ -42,7 +44,7 @@ Take Screenshot If XPath Not Found Multiple Times By Specific Folder
         Run Keyword And Expect Error    ${EXP_ERR_MSG}    Click    ${XPATH_NOT_EXISTS}
         Wait Until Created    ${OUTPUT DIR}/${SCREENSHOT_FOLDER}/${FILENAME}    1s
     END
-    Set Screenshot Directory
+    [Teardown]    Reset Screenshot Environment To Default
 
 Take Manual Screenshot By Keyword
     Set Screenshot Directory    ${SCREENSHOT_FOLDER}
@@ -54,16 +56,15 @@ Take Manual Screenshot By Keyword
     File Should Not Exist    ${OUTPUT DIR}/${SCREENSHOT_FOLDER}/${FILENAME}
     Take Screenshot
     File Should Exist    ${OUTPUT DIR}/${SCREENSHOT_FOLDER}/${FILENAME}
-    Set Screenshot Directory
-    Take Screenshots On Failure    True
+    [Teardown]    Reset Screenshot Environment To Default
 
 Test Case 1234: Something to Test
     Set Screenshot Directory    ${SCREENSHOT_FOLDER}
     ${FILENAME}    Get Expected Filename    Test Case 1234 Something to Test
     File Should Not Exist    ${OUTPUT DIR}/${SCREENSHOT_FOLDER}/${FILENAME}
     Take Screenshot
     File Should Exist    ${OUTPUT DIR}/${SCREENSHOT_FOLDER}/${FILENAME}
-    Set Screenshot Directory
+    [Teardown]    Reset Screenshot Environment To Default
 
 No Screenshots Should Created For No Library Keywords
     Set Screenshot Directory    ${SCREENSHOT_FOLDER}
@@ -72,7 +73,33 @@ No Screenshots Should Created For No Library Keywords
     Run Keyword And Ignore Error    Fail    You Should Not Pass
     Run Keyword And Ignore Error    Wait Until Keyword Succeeds    5x    10ms    Fail    You Should Not Pass
     File Should Not Exist    ${OUTPUT DIR}/${SCREENSHOT_FOLDER}/${FILENAME}
-    Set Screenshot Directory
+    [Teardown]    Reset Screenshot Environment To Default
+
+Take Screenshot Of Window
+    [Setup]    Start Application
+    ${PID}    Attach Application By Name    ${TEST_APP}
+    Set Screenshot Directory    ${SCREENSHOT_FOLDER}
+    ${FILENAME}    Get Expected Filename    ${TEST_NAME}
+    File Should Not Exist    ${OUTPUT DIR}/${SCREENSHOT_FOLDER}/${FILENAME}
+    Take Screenshot    ${MAIN_WINDOW}
+    File Should Exist    ${OUTPUT DIR}/${SCREENSHOT_FOLDER}/${FILENAME}
+    [Teardown]    Reset Screenshot Environment To Default    ${PID}
+
+Take Screenshot As Base64
+    Set Screenshot Log Mode    Base64
+    ${base64}    Take Screenshot
+    Should Not Be Equal    ${base64}    ${None}    Returned base64 image is 'None'
+    Should Not Be Empty    ${base64}    Returned base64 image is empty
+    [Teardown]    Reset Screenshot Environment To Default
+
+Take Screenshot Of Window As Base64
+    [Setup]    Start Application
+    ${PID}    Attach Application By Name    ${TEST_APP}
+    Set Screenshot Log Mode    Base64
+    ${base64}    Take Screenshot    ${MAIN_WINDOW}
+    Should Not Be Equal    ${base64}    ${None}    Returned base64 image is 'None'
+    Should Not Be Empty    ${base64}    Returned base64 image is empty
+    [Teardown]    Reset Screenshot Environment To Default    ${PID}
 
 
 *** Keywords ***
@@ -91,3 +118,11 @@ Get Expected Filename
     ${FILENAME}    Catenate    SEPARATOR=.    ${FILENAME}    jpg
 
     RETURN    ${FILENAME}
+
+Reset Screenshot Environment To Default
+    [Documentation]    Reset screenshot environment to default and initial settings.
+    [Arguments]    ${pid}=${None}
+    Set Screenshot Log Mode    File
+    Take Screenshots On Failure    True
+    Set Screenshot Directory
+    Run Keyword And Ignore Error    Stop Application    ${pid}

src/FlaUILibrary/__init__.py

@@ -138,7 +138,7 @@ def __init__(self, uia='UIA3', screenshot_on_failure='True', screenshot_dir=None
             FlaUILibrary.KeywordModules.GRID: GridKeywords(self.container),
             FlaUILibrary.KeywordModules.MOUSE: MouseKeywords(self.container),
             FlaUILibrary.KeywordModules.KEYBOARD: KeyboardKeywords(self.container),
-            FlaUILibrary.KeywordModules.SCREENSHOT: ScreenshotKeywords(self.screenshots),
+            FlaUILibrary.KeywordModules.SCREENSHOT: ScreenshotKeywords(self.screenshots, self.container),
             FlaUILibrary.KeywordModules.TEXTBOX: TextBoxKeywords(self.container),
             FlaUILibrary.KeywordModules.WINDOW: WindowKeywords(self.container),
             FlaUILibrary.KeywordModules.RADIOBUTTON: RadioButtonKeywords(self.container),

src/FlaUILibrary/flaui/module/screenshot.py

@@ -2,13 +2,18 @@
 import platform
 import time
 from enum import Enum
+from typing import Any, Optional
 from FlaUI.Core.Capturing import Capture  # pylint: disable=import-error
 from System import Exception as CSharpException  # pylint: disable=import-error
+from System import Convert as CSharpConvert  # pylint: disable=import-error
+from System.IO import MemoryStream  # pylint: disable=import-error
+from System.Drawing.Imaging import ImageFormat  # pylint: disable=import-error
 from FlaUILibrary.flaui.exception import FlaUiError
 from FlaUILibrary.flaui.interface import (ModuleInterface, ValueContainer)
 from FlaUILibrary.robotframework import robotlog
 
 
+
 # pylint: disable=too-many-instance-attributes
 class Screenshot(ModuleInterface):
     """
@@ -21,12 +26,21 @@ class Container(ValueContainer):
         Value container from screenshot module.
         """
         keywords: list
+        element: Optional[Any]
 
     class Action(Enum):
         """
         Supported actions for execute action implementation.
         """
         CAPTURE = "CAPTURE"
+        CAPTURE_ELEMENT = "CAPTURE_ELEMENT"
+
+    class ScreenshotMode(Enum):
+        """
+        Supported modes for screenshots.
+        """
+        FILE = "File"
+        BASE64 = "Base64"
 
     def __init__(self, directory, is_enabled):
         """
@@ -40,6 +54,7 @@ def __init__(self, directory, is_enabled):
         self._name = ""
         self._hostname = self._clean_invalid_windows_syntax(platform.node().lower())
         self._filename = "test_{}_{}_{}_{}.jpg"
+        self._mode = self.ScreenshotMode.FILE
 
     def set_name(self, name):
         """
@@ -51,8 +66,28 @@ def set_name(self, name):
         """
         self._name = self._clean_invalid_windows_syntax(name.replace(" ", "_").lower())
 
+    def set_mode(self, mode: str):
+        """
+        Set screenshot logging mode. Available modes: File, Base64
+
+        Args:
+            mode (str): Screenshot mode to set.
+        """
+        if mode.upper() == 'FILE':
+            self._mode = self.ScreenshotMode.FILE
+        elif mode.upper() == 'BASE64':
+            self._mode = self.ScreenshotMode.BASE64
+        else:
+            FlaUiError.raise_fla_ui_error(FlaUiError.ActionNotSupported)
+
+    def get_mode(self):
+        """
+        Return the configured screenshot logging mode.
+        """
+        return self._mode
+
     @staticmethod
-    def create_value_container(keywords=None):
+    def create_value_container(keywords=None, element=None):
         """
         Helper to create container object.
 
@@ -62,17 +97,29 @@ def create_value_container(keywords=None):
         if keywords is None:
             keywords = []
 
-        return Screenshot.Container(keywords=keywords)
+        return Screenshot.Container(keywords=keywords, element=element)
 
     def execute_action(self, action: Action, values: ValueContainer):
         # pylint: disable=unnecessary-lambda
         switcher = {
-            self.Action.CAPTURE: lambda: self._capture()
+            self.Action.CAPTURE: lambda: self._capture(),
+            self.Action.CAPTURE_ELEMENT: lambda: self._capture(element=values['element'])
         }
 
         return switcher.get(action, lambda: FlaUiError.raise_fla_ui_error(FlaUiError.ActionNotSupported))()
 
-    def _capture(self):
+    def _capture(self, element=None):
+        """
+        Capture image depending on mode
+        """
+        if self._mode == self.ScreenshotMode.FILE:
+            return self._capture_file(element)
+        if self._mode == self.ScreenshotMode.BASE64:
+            return self._capture_base64(element)
+        return FlaUiError.raise_fla_ui_error("Invalid screenshot mode selected. Available modes: "
+                                             + '\n'.join([str(mode) for mode in self.ScreenshotMode]))
+
+    def _capture_file(self, element=None):
         """
         Capture image from desktop.
         """
@@ -90,7 +137,10 @@ def _capture(self):
                 os.makedirs(directory)
 
             try:
-                image = Capture.Screen()
+                if element:
+                    image = Capture.Element(element)
+                else:
+                    image = Capture.Screen()
                 image.ToFile(filepath)
 
                 # Log screenshot from temp or persist mode
@@ -107,6 +157,33 @@ def _capture(self):
 
         return filepath
 
+    def _capture_base64(self, element=None):
+        image = None
+
+        try:
+            if element:
+                image = Capture.Element(element)
+            else:
+                image = Capture.Screen()
+            stream = MemoryStream()
+            image.Bitmap.Save(stream, ImageFormat.Png)
+            base64 = CSharpConvert.ToBase64String(stream.GetBuffer())
+            stream.Close()
+
+            # Log screenshot from temp or persist mode
+            robotlog.log_screenshot_base64(base64)
+
+        except CSharpException:
+            robotlog.log("Error to save as base64 encoded string: " + element)
+
+        finally:
+            self.img_counter += 1
+            if image is not None:
+                # C# --> class CaptureImage : IDisposable
+                image.Dispose()
+
+        return base64
+
     def _get_path(self):
         """
         Get directory path for logging.

src/FlaUILibrary/keywords/screenshot.py

@@ -1,35 +1,74 @@
 from robot.utils import is_truthy
 from robotlibcore import keyword
 from FlaUILibrary.flaui.module.screenshot import Screenshot
-from FlaUILibrary.robotframework import robotlog
-
+from FlaUILibrary.flaui.util.automationinterfacecontainer import AutomationInterfaceContainer
 
 class ScreenshotKeywords:
     """
     Interface implementation from Robotframework usage for screenshot keywords.
     """
 
-    def __init__(self, screenshots: Screenshot):
+    def __init__(self, screenshots: Screenshot, container: AutomationInterfaceContainer):
         """Creates screenshot keywords module to handle image capturing.
 
-        ``screenshots`` Screenshots module for image capturing.
+        ``screenshots`` Screenshots module for image capturing
+        ``container`` User automation container to handle element interaction
         """
         self._screenshots = screenshots
+        self._container = container
 
     @keyword
-    def take_screenshot(self):
-        """ Takes a screenshot of the whole desktop. Returns path to the screenshot if screenshot was created.
+    def get_screenshot_log_mode(self):
+        """Returns the current logging mode of the screenshot module. Default is 'File'.
+
+        Example:
+        | ${log_mode} = | Get Screenshot Log Mode |
+        """
+        return self._screenshots.get_mode()
+
+    @keyword
+    def set_screenshot_log_mode(self, log_mode: str):
+        """Sets the logging mode of the screenshot module. Default is 'File'.
+        Mode 'File' logs screenshots as files in the screenshot directory.
+        Mode 'Base64' logs screenshots as base64 encoded strings embedded in the test report.
+
+        Arguments:
+        | Argument      | Type   | Description          |
+        | log_mode      | string | File | Base64        |
+
+        Example:
+        | Set Screenshot Log Mode    Base64 |
+        """
+        self._screenshots.set_mode(log_mode)
+
+    @keyword
+    def take_screenshot(self, identifier=None, msg=None):
+        """ Takes a screenshot of the whole desktop or the element, from the optionally provided identifier. 
+        Returns screenshot depending on log mode.
+        Screenshot mode File -> returns filepath
+        Screenshot mode Base64 -> returns encoded base64 string of image
+
+        Arguments:
+        | Argument      | Type   | Description          |
+        | identifier    | string | XPath identifier from element |
+        | msg           | string | Custom error message          |
 
         Example:
         | Take Screenshot |
+        | Take Screenshot   <XPATH> |
+        | Take Screenshot   <XPATH>    "Your custom error message" |
         """
-        filepath = self._screenshots.execute_action(Screenshot.Action.CAPTURE,
+        image_var = None
+        if identifier:
+            module = self._container.create_or_get_module()
+            element = module.get_element(identifier, msg=msg)
+            image_var = self._screenshots.execute_action(Screenshot.Action.CAPTURE_ELEMENT,
+                                                    Screenshot.create_value_container(element=element))
+        else:
+            image_var = self._screenshots.execute_action(Screenshot.Action.CAPTURE,
                                                     Screenshot.create_value_container())
 
-        if filepath:
-            robotlog.log_screenshot(filepath)
-
-        return filepath
+        return image_var
 
     @keyword
     def take_screenshots_on_failure(self, enabled):

src/FlaUILibrary/robotframework/robotlog.py

@@ -37,3 +37,15 @@ def log_screenshot(filepath: str):
         ),
         html=True,
     )
+
+def log_screenshot_base64(image: str):
+    """
+    Append testing log by a screenshot in base64 format.
+
+    ``image`` Image as string in base64 encoding. 
+    """
+    logger.info(
+        '</td></tr><tr><td colspan="3">' +
+        f'<img src="data:image/png;base64,{image}" width="800px"/>',
+        html=True
+    )