Merge branch 'develop'

Author: rjrudin

.gitignore

@@ -5,3 +5,5 @@ __pycache__
 .venv
 venv
 .idea
+
+.DS_Store

CONTRIBUTING.md

@@ -35,7 +35,8 @@ To run an individual test method:
 
 Note that due to the pytest config in the `pyproject.toml` file, all 
 [Python logging](https://docs.python.org/3/howto/logging.html) should appear immediately
-as tests are executed.
+as tests are executed. If you are using VSCode, you can see the logging by selecting 
+"Python Test Log" in the "Output" panel.
 
 ## Testing the client in a Python shell
 
@@ -56,6 +57,16 @@ And you can then start sending requests with the client - for example:
     r = client.get("/v1/search?format=json&pageLength=2")
     r.json()
 
+You can also use the scripts in the `./shell` directory to initialize a client. The following initializes a client
+that connects to this project's test application:
+
+    python -i shell/test_app.py
+
+And this initializes a client that connects to the OOTB App-Services app server, which is used for examples in this 
+project's documentation:
+
+    python -i shell/docs.py
+
 ## Testing the documentation locally
 
 The docs for this project are stored in the `./docs` directory as a set of Markdown files. These are published via

docs/Gemfile.lock

@@ -251,6 +251,7 @@ GEM
 
 PLATFORMS
   arm64-darwin-21
+  arm64-darwin-23
 
 DEPENDENCIES
   github-pages (~> 228)

docs/eval.md

@@ -0,0 +1,119 @@
+---
+layout: default
+title: Executing code
+nav_order: 7
+---
+
+The [MarkLogic REST service extension](https://docs.marklogic.com/REST/client/service-extension) supports the
+execution of custom code, whether via an inline script or an existing module in your application's modules database. 
+The MarkLogic Python client supports execution of custom code by simplifying the submission of custom code
+and converting the multipart response into more useful Python data types.
+
+## Setup
+
+The examples below all depend on the instructions in the [setup guide](example-setup.md) having already been performed.
+
+To try out the examples, start a Python shell and first run the following:
+
+```
+from marklogic import Client
+client = Client('http://localhost:8000', digest=('python-user', 'pyth0n'))
+```
+
+## Executing ad-hoc queries
+
+The [v1/eval REST endpoint](https://docs.marklogic.com/REST/POST/v1/eval) supports the execution of ad-hoc JavaScript 
+and XQuery queries. Each type of query can be easily submitted via the client:
+
+```
+client.eval(javascript="fn.currentDateTime()")
+client.eval(xquery="fn:current-dateTime()")
+```
+
+Variables can optionally be provided via a `dict`:
+
+```
+results = client.eval(javascript='Sequence.from([{"hello": myValue}])', vars={"myValue": "world"})
+assert "world" == results[0]["hello"]
+```
+
+Because the REST endpoint returns a sequence of items, the client will always return a list of values. See the section
+below on how data types are converted to understand how the client will convert each value into an appropriate Python
+data type.
+
+## Invoking modules
+
+The [v1/invoke REST endpoint](https://docs.marklogic.com/REST/POST/v1/invoke) supports the execution of JavaScript
+and XQuery main modules that have been deployed to your application's modules database. 
+
+If you wish to attempt the examples below (instead of invoking your own modules), first run the following to 
+insert a simple runnable module into the out-of-the-box "Modules" database:
+
+```
+from marklogic.documents import Document
+module = Document('/sample.sjs', 'const doc = {"hello": "world"}; doc', permissions={"rest-reader": ["read", "update", "execute"]})
+client.documents.write(module, params={"database": "Modules"})
+```
+
+A module can be invoked via the client in the following manner:
+
+```
+# Set the input to the URI of the module you wish to invoke in your application's 
+# modules database.
+client.invoke("/sample.sjs")
+```
+
+You can provide variables to your module in the same fashion as when evaluating custom 
+code (the variable will not have any impact on the sample module loaded above; this is
+shown purely to demonstrate how to define variables):
+
+```
+client.invoke("/sample.sjs", vars={"my_var1": "value1"})
+```
+
+## Conversion of data types
+
+The REST endpoints for evaluating ad-hoc code and for invoking a module both return a sequence of values, with each 
+value having MarkLogic-specific type information. The client will use this type information to convert each value into
+an appropriate Python data type. For example, each JSON object into the example below is converted into a `dict`:
+
+```
+results = client.eval(javascript='Sequence.from([{"doc": 1}, {"doc": 2}])')
+assert len(results) == 2
+assert results[0]["doc"] == 1
+assert results[1]["doc"] == 2
+```
+
+The following table describes how each MarkLogic type is associated with a Python data type. For any 
+MarkLogic type not listed in the table, such as `hexBinary` and `base64Binary`, the value is not converted and will 
+remain of type `bytes`. 
+
+| MarkLogic type | Python type | 
+| --- | --- |
+| string | str |
+| integer | int |
+| boolean | bool |
+| decimal | [Decimal](https://docs.python.org/3/library/decimal.html) |
+| map | dict |
+| element() | str |
+| array | list |
+| array-node() | list |
+| object-node() | dict or `marklogic.documents.Document` |
+| document-node() | str or `marklogic.documents.Document` |
+| binary() | bytes or `marklogic.documents.Document` | 
+
+For the `object-node()`, `document-node()`, and `binary()` entries in the above table, a 
+`marklogic.documents.Document` instance will be returned if the value is associated with a URI via 
+the multipart `X-URI` header. Otherwise, a value of type `dict`, `str`, or `bytes` is returned respectively.
+
+## Returning the original HTTP response
+
+Each `client.eval` method and `client.invoke` accept a `return_response` argument. When that
+argument is set to `True`, the original response is returned. This can be useful for custom
+processing of the response or debugging requests.
+
+## Referencing a transaction
+
+The `client.eval` and `client.invoke` functions both support referencing a 
+[REST API transaction](https://docs.marklogic.com/REST/client/transaction-management) via the `tx` 
+argument. See [the guide on transactions](transactions.md) for further information.

docs/example-setup.md

@@ -5,17 +5,30 @@ nav_order: 2
 permalink: /setup
 ---
 
-The examples in this documentation depend on a particular MarkLogic username and password. If you 
-would like to try these examples out against your own installation of MarkLogic, you will need to create this 
-MarkLogic user. To do so, please go to the Admin application for your MarkLogic instance - e.g. if you are running MarkLogic locally, this will be at <http://localhost:8001> - and authenticate as your "admin" user. 
-Then perform the following steps to create a new user:
+The examples in this documentation depend on a particular MarkLogic user with a role containing specific privileges. 
+If you would like to try these examples out against your own installation of MarkLogic, you will need to create this 
+MarkLogic user and role. To do so, please go to the Admin application for your MarkLogic instance - e.g. if you are 
+running MarkLogic locally, this will be at <http://localhost:8001> - and authenticate as your "admin" user. 
+Then perform the following steps to create a new role:
+
+1. Click on "Roles" in the "Security" box. 
+2. Click on "Create".
+3. In the form, enter `python-docs-role` for "Role Name".
+4. Scroll down and select the `rest-extension-user`, `rest-reader`, `rest-writer`, and `tde-admin` roles.
+5. Scroll further down and select the `xdbc:eval`, `xdbc:invoke`, and `xdmp:eval-in` privileges.
+6. Scroll to the top or bottom and click on "OK" to create the role.
+
+After creating the role, return to the Admin application home page and perform the following steps:
 
 1. Click on "Users" in the "Security" box.
 2. Click on "Create".
-3. In the form, enter "python-user" for "User Name" and "pyth0n" as the password. 
-4. Scroll down until you see the "Roles" section. Click on the "rest-reader" and "rest-writer" checkboxes. 
+3. In the form, enter `python-user` for "User Name" and `pyth0n` as the password. 
+4. Scroll down until you see the "Roles" section and select the `python-docs-role` role. 
 5. Scroll to the top or bottom and click on "OK" to create the user.
 
+(Note that you could use the `admin` role instead to grant full access to all features in MarkLogic, but this is 
+generally discouraged for security reasons.)
+
 You can verify that you correctly created the user by accessing the REST API for the out-of-the-box REST API 
 server in MarkLogic that listens on port 8000. Go to <http://localhost:8000/v1/search> (changing "localhost" to 
 the correct name of your MarkLogic host if you are not running it locally) in a web browser and enter the 

docs/index.md

@@ -31,6 +31,6 @@ If you wish to try these examples on your own installation of MarkLogic, please
 for instructions on creating this user. 
 
 Otherwise, please see the [guide on creating a client](creating-client.md) for more information on connecting to a 
-MarkLogic REST API instance. The [guide on managing documents](managing-documents/managing-documents.md) provides
-more information on how the client simplifies writing and reading multiple documents in a single request.
+MarkLogic REST API instance. Then choose a guide from the navigation on the left to see how the client can simplify
+your application's interactions with the MarkLogic REST API.
 

docs/managing-documents/managing-documents.md

@@ -1,12 +1,12 @@
 ---
 layout: default
 title: Managing documents
-nav_order: 3
+nav_order: 4
 has_children: true
 permalink: /documents
 ---
 
-The [/v1/documents endpoint](https://docs.marklogic.com/REST/client/management) in the MarkLogic REST API supports
+The [MarkLogic REST documents service](https://docs.marklogic.com/REST/client/management) supports
 operations that involve writing or reading a single document. It also supports operations that involve multiple 
 documents, though those require use of a potentially complex multipart HTTP request or response. The MarkLogic Python
 client simplifies those operations by hiding the details of multipart requests and responses.

docs/managing-documents/reading.md

@@ -114,6 +114,18 @@ print(docs)
 Please see [the application developer's guide](https://docs.marklogic.com/guide/rest-dev/documents#id_80116)
 for more information on reading documents.
 
+## Returning the original HTTP response
+
+Starting in the 1.1.0 release, the `client.documents.search` method accepts a 
+`return_response` argument. When that argument is set to `True`, the original response 
+is returned. This can be useful for custom processing of the response or debugging requests.
+
+## Referencing a transaction
+
+Starting in the 1.1.0 release, you can reference a 
+[REST API transaction](https://docs.marklogic.com/REST/client/transaction-management) via the `tx` 
+argument. See [the guide on transactions](../transactions.md) for further information.
+
 ## Error handling
 
 If the `client.documents.read` method receives an HTTP response with a status code of 200, then the client will return

docs/managing-documents/searching.md

@@ -155,6 +155,18 @@ assert len(docs) == 2
 Please see [the application developer's guide](https://docs.marklogic.com/guide/rest-dev/search#id_49329)
 for more information on searching documents.
 
+## Returning the original HTTP response
+
+Starting in the 1.1.0 release, the `client.documents.search` method accepts a 
+`return_response` argument. When that argument is set to `True`, the original response 
+is returned. This can be useful for custom processing of the response or debugging requests.
+
+## Referencing a transaction
+
+Starting in the 1.1.0 release, you can reference a 
+[REST API transaction](https://docs.marklogic.com/REST/client/transaction-management) via the `tx` 
+argument. See [the guide on transactions](../transactions.md) for further information.
+
 ## Error handling
 
 If the `client.documents.read` method receives an HTTP response with a status code of 200, then the client will return

docs/managing-documents/writing.md

@@ -150,6 +150,13 @@ response = client.documents.write(Document("/doc1.json", {"doc": 1}, permissions
 Please see [the application developer's guide](https://docs.marklogic.com/guide/rest-dev/documents#id_11953) for 
 more information on writing documents.
 
+## Referencing a transaction
+
+Starting in the 1.1.0 release, you can reference a 
+[REST API transaction](https://docs.marklogic.com/REST/client/transaction-management) via the `tx` 
+argument. See [the guide on transactions](../transactions.md) for further information.
+
+
 ## Error handling
 
 Because the `client.documents.write` method returns a `requests Response` object, any error that occurs during