Configuration references

Below is the description of the available configurations of the CosmsoDB Spark Connector. Depends on the scenario, different configurations should be used to optimize the performance and throughput.

Reading data

Many of the below configurations are passed on to the Java SDK when fetching the data from the CosmosDB collection.

• query_maxretryattemptsonthrottledrequests: sets the maximum number of retries in the case where the request fails because the Azure CosmosDB database service has applied rate limiting on the client. If not specified, the default value is 9.
• query_maxretrywaittimeinseconds: sets the maximum retry time in seconds. By default, it is 30 seconds.
• query_maxdegreeofparallelism: sets the number of concurrent operations run client side during parallel query execution in the Azure DocumentDB database service. A positive property value limits the number of concurrent operations to the set value. If it is set to less than 0, the system automatically decides the number of concurrent operations to run. As the Connector maps each collection partition with an executor, this value won't have any effect on the reading operation.
• query_maxbuffereditemcount: sets the maximum number of items that can be buffered client side during parallel query execution in the Azure DocumentDB database service. A positive property value limits the number of buffered items to the set value. If it is set to less than 0, the system automatically decides the number of items to buffer.
• query_enablescan: sets the option to enable scans on the queries which couldn't be served as indexing was opted out on the requested paths in the Azure DocumentDB database service.
• query_disableruperminuteusage: disables Request Units(RUs)/minute capacity to serve the query if regular provisioned RUs/second is exhausted.
• query_emitverbosetraces: Sets the option to allow queries to emit out verbose traces for investigation.