Changes done for v4.1.7.

Author: Senthil Nathan

com.ibm.streamsx.dps/com.ibm.streamsx/store/distributed/native.function/function.xml

@@ -435,40 +435,69 @@ it does safety checks and is therefore slower.</description>
       </function>
 
       <function>
-        <description>This function can be called to get multiple keys present in a given store.
+        <description>This function can be called to get i.e. read/fetch multiple keys present in a given store.
 @param store The handle of the store.
  @param keys User provided mutable list variable. This list must be suitable for storing multiple keys found in a given store and it must be made of a given store's key data type.
- @param keyStartPosition User can indicate a start position from where keys should be fetched and returned. It must be greater than or equal to zero. If not, this API will return back with an empty list of keys.
+ @param keyStartPosition User can indicate a zero index based start position from where keys should be fetched and returned. It must be greater than or equal to zero. If not, this API will return back with an empty list of keys.
  @param numberOfKeysNeeded User can indicate the total number of keys to be returned as available from the given key start position. It must be greater than or equal to 0 and less than or equal to 50000. If it is set to 0, then all the available keys upto a maximum of 50000 keys from the given key start position will be returned.
  @param keyExpression User can provide an expression made of the attributes from the key's data type. This expression will be evaluated in determining which matching keys to be returned. Due to very high logic complexity, this feature is not implemented at this time.
  @param valueExpression User can provide an expression made of the attributes from the value's data type. This expression will be evaluated in determining which matching keys to be returned. Due to very high logic complexity, this feature is not implemented at this time.
- @param err Contains the error code. Will be '0' if no error occurs, and a non-zero value otherwise. 
+@param err User provided mutable uint64 typed variable in which the result of this bulk get keys operation will be returned. It will be 0 if no error occurs and a non-zero error code otherwise.
 </description>
         <prototype>&lt;any T1> public stateful void dpsGetKeys(uint64 store, mutable list&lt;T1&gt; keys, int32 keyStartPosition, int32 numberOfKeysNeeded, rstring keyExpression, rstring valueExpression, mutable uint64 err)</prototype>
       </function>
 
       <function>
-        <description>This function can be called to get values for a given list of multiple keys present in a given store.
+        <description>This function can be called to get i.e. read/fetch values for a given list of multiple keys present in a given store.
 @param store The handle of the store.
-@param keys User provided list variable that contains keys for which values need to be fetched.
-@param values User provided mutabe list variable in which the fetched values will be returned. This list must be suitable for storing multiple values obtained from a given store and it must be made of a given store's value data type. Fetched values will be available in this list at the same index where the key appears in the user provided keys list. Before using the individual values from this list, it is recommended to make sure that a given value fetch worked with no errors.
-@param errors User provided mutable list variable in which the individual success or failure value fetch result codes will be returned. This list must be of type uint64. Each list element will be 0 if no error occurs and a non-zero error code otherwise. Such value fetch result codes will be available in this list at the same index where the key appears in the user provided keys list. If a given result code doesn't indicate a successful value fetch, it is better to skip the corresponding element at the same index in the mutable values list.
-@return It returns true if value fetch worked for all given keys with no errors. Else, it returns false to indicate that value fetch encountered error for one or more keys.
+@param keys User provided list variable that contains keys for which values need to be fetched. It must be made of a given store's key data type.
+@param keyExistsOrNot User provided mutable list variable in which true or false status for every user given key will be returned. This is done to indicate to the user about if that key exists or not in the given store. This list must be made of a boolean data type. Status returned in this list can be used in combination with the items returned in the values list which is explained next.
+@param values User provided mutabe list variable in which the fetched values will be returned. This list must be suitable for storing multiple values obtained from a given store and it must be made of a given store's value data type. Fetched values will be available in this list at the same index where the key appeared in the user provided keys list. If a user given key is not present in the store, there will still be a default value returned in that key's index. It is better to first confirm in the keyExistsOrNot list that will carry true or false status about the existence of all the user provided keys. Depending on the key existence status found in that other list, user can decide to either consider using or ignore a value in a given index of the values list.
+@param err User provided mutable uint64 typed variable in which the result of this bulk get values operation will be returned. It will be 0 if no error occurs and a non-zero error code otherwise.
 </description>
-        <prototype>&lt;any T1, any T2> public stateful boolean dpsGetValues(uint64 store, list&lt;T1&gt; keys, mutable list&lt;T2&gt; values, mutable list&lt;uint64&gt; errors)</prototype>
+        <prototype>&lt;any T1, any T2> public stateful void dpsGetValues(uint64 store, list&lt;T1&gt; keys, mutable list&lt;boolean&gt; keyExistsOrNot, mutable list&lt;T2&gt; values, mutable uint64 err)</prototype>
       </function>
 
       <function>
-        <description>This function can be called to get multiple Key/Value (KV) pairs present in a given store.
+        <description>This function can be called to get i.e. read/fetch multiple Key/Value (KV) pairs present in a given store.
 @param store The handle of the store.
 @param keys User provided mutable list variable in which the keys found in the store will be returned. This list must be suitable for storing multiple keys found in a given store and it must be made of a given store's key data type.
-@param values User provided mutabe list variable in which the fetched values will be returned. This list must be suitable for storing multiple values obtained from a given store and it must be made of a given store's value data type. Fetched values will be available in this list at the same index where the corresponding key appears in the keys list. Before using the individual values from this list, it is recommended to make sure that a given value fetch worked with no errors.
-@param keyStartPosition User can indicate a start position from where the K/V pairs should be fetched and returned. It must be greater than or equal to zero. If not, this API will return back with an empty list of keys.
+@param values User provided mutabe list variable in which the fetched values will be returned. This list must be suitable for storing multiple values obtained from a given store and it must be made of a given store's value data type. Fetched values will be available in this list at the same index where the corresponding key appears in the keys list.
+@param keyStartPosition User can indicate a zero index based start position from where the K/V pairs should be fetched and returned. It must be greater than or equal to zero. If not, this API will return back with an empty list of keys.
 @param numberOfPairsNeeded User can indicate the total number of K/V pairs to be returned as available from the given key start position. It must be greater than or equal to 0 and less than or equal to 50000. If it is set to 0, then all the available K/V pairs upto a maximum of 50000 pairs from the given key start position will be returned.
-@param errors User provided mutable list variable in which the individual success or failure value fetch result codes will be returned. This list must be of type uint64. Each list element will be 0 if no error occurs and a non-zero error code otherwise. Such value fetch result codes will be available in this list at the same index where the key appears in the keys list. If a given result code doesn't indicate a successful value fetch, it is better to skip the corresponding element at the same index in the mutable values list.
-@return It returns true if value fetch worked for all the keys with no errors. Else, it returns false to indicate that value fetch encountered error for one or more keys.
+@param err User provided mutable uint64 typed variable in which the result of this bulk get KV pairs operation will be returned. It will be 0 if no error occurs and a non-zero error code otherwise.
 </description>
-        <prototype>&lt;any T1, any T2> public stateful boolean dpsGetKVPairs(uint64 store, mutable list&lt;T1&gt; keys, mutable list&lt;T2&gt; values, int32 keyStartPosition, int32 numberOfPairsNeeded, mutable list&lt;uint64&gt; errors)</prototype>
+        <prototype>&lt;any T1, any T2> public stateful void dpsGetKVPairs(uint64 store, mutable list&lt;T1&gt; keys, mutable list&lt;T2&gt; values, int32 keyStartPosition, int32 numberOfPairsNeeded, mutable uint64 err)</prototype>
+      </function>
+
+      <function>
+        <description>This function can be called to put i.e. write/save multiple Key/Value (KV) pairs to a given store. Note: You must make sure that the intended store exists before you use this function. If not, you will end up creating bulk K/V pairs in an unmanageable fashion in the backend data store which will make it less useful. 
+@param store The handle of the store.
+@param keys User provided list variable that contains the keys to be written i.e. saved. This list must be made of a given store's key data type.
+@param values User provided list variable that contains the values to be written i.e. saved. This list must be made of a given store's value data type. A KV pair is formed with a key and a value taken from the same index position of the keys and values lists. If the keys and values lists are not of the same size, this function will simply return back without doing any bulk put operation.
+@param err User provided mutable uint64 typed variable in which the result of this bulk put operation will be returned. It will be 0 if no error occurs and a non-zero error code otherwise.
+</description>
+        <prototype>&lt;any T1, any T2> public stateful void dpsPutKVPairs(uint64 store, list&lt;T1&gt; keys, list&lt;T2&gt; values, mutable uint64 err)</prototype>
+      </function>
+
+      <function>
+        <description>This function can be called to check for the existence of a given list of keys in a given store.
+@param store The handle of the store.
+@param keys User provided list variable that contains the keys to be checked for their existence in the given store. This list must be made of a given store's key data type.
+@param results User provided mutable list variable in which the key existence check true or false results will be returned. This list must be made of a boolean data type.
+@param err User provided mutable uint64 typed variable in which the result of this bulk has keys operation will be returned. It will be 0 if no error occurs and a non-zero error code otherwise.
+</description>
+        <prototype>&lt;any T1> public stateful void dpsHasKeys(uint64 store, list&lt;T1&gt; keys, mutable list&lt;boolean&gt; results, mutable uint64 err)</prototype>
+      </function>
+
+      <function>
+        <description>This function can be called to remove a given list of keys from a given store.
+@param store The handle of the store.
+@param keys User provided list variable that contains the keys to be removed from the given store. This list must be made of a given store's key data type.
+@param totalKeysRemoved User provided mutable int32 variable in which the total number of keys removed from the store will be returned.
+@param err User provided mutable uint64 typed variable in which the result of this bulk remove keys operation will be returned. It will be 0 if no error occurs and a non-zero error code otherwise.
+</description>
+        <prototype>&lt;any T1> public stateful void dpsRemoveKeys(uint64 store, list&lt;T1&gt; keys, mutable int32 totalKeysRemoved, mutable uint64 err)</prototype>
       </function>
 
       <function>

com.ibm.streamsx.dps/impl/include/CassandraDBLayer.h

@@ -159,7 +159,10 @@ namespace distributed
     bool removeLock(uint64_t lock, PersistenceError & lkError);
     uint32_t getPidForLock(std::string const & name, PersistenceError & lkError);
     void getKeys(uint64_t store, std::vector<unsigned char *> & keysBuffer, std::vector<uint32_t> & keysSize, int32_t keyStartPosition, int32_t numberOfKeysNeeded, PersistenceError & dbError);
-    void getValue(std::string const & storeIdString, char const * & key, uint32_t const & keySize, unsigned char * & value, uint32_t & valueSize, uint64_t & error);
+    void getValues(uint64_t store, std::vector<char *> const & keyData, std::vector<uint32_t> const & keySize, std::vector<unsigned char *> & valueData, std::vector<uint32_t> & valueSize, PersistenceError & dbError);
+    void putKVPairs(uint64_t store, std::vector<char *> const & keyData, std::vector<uint32_t> const & keySize, std::vector<unsigned char *> const & valueData, std::vector<uint32_t> const & valueSize, PersistenceError & dbError);
+    void hasKeys(uint64_t store, std::vector<char *> const & keyData, std::vector<uint32_t> const & keySize, std::vector<bool> & results, PersistenceError & dbError);
+    void removeKeys(uint64_t store, std::vector<char *> const & keyData, std::vector<uint32_t> const & keySize, int32_t & totalKeysRemoved, PersistenceError & dbError);
 
   };
 } } } } }

com.ibm.streamsx.dps/impl/include/CloudantDBLayer.h

@@ -197,8 +197,11 @@ namespace distributed
     bool acquireLock(uint64_t lock, double leaseTime, double maxWaitTimeToAcquireLock, PersistenceError & lkError);
     bool removeLock(uint64_t lock, PersistenceError & lkError);
     uint32_t getPidForLock(std::string const & name, PersistenceError & lkError);
-   void getKeys(uint64_t store, std::vector<unsigned char *> & keysBuffer, std::vector<uint32_t> & keysSize, int32_t keyStartPosition, int32_t numberOfKeysNeeded, PersistenceError & dbError);
-    void getValue(std::string const & storeIdString, char const * & key, uint32_t const & keySize, unsigned char * & value, uint32_t & valueSize, uint64_t & error);
+    void getKeys(uint64_t store, std::vector<unsigned char *> & keysBuffer, std::vector<uint32_t> & keysSize, int32_t keyStartPosition, int32_t numberOfKeysNeeded, PersistenceError & dbError);
+    void getValues(uint64_t store, std::vector<char *> const & keyData, std::vector<uint32_t> const & keySize, std::vector<unsigned char *> & valueData, std::vector<uint32_t> & valueSize, PersistenceError & dbError);
+    void putKVPairs(uint64_t store, std::vector<char *> const & keyData, std::vector<uint32_t> const & keySize, std::vector<unsigned char *> const & valueData, std::vector<uint32_t> const & valueSize, PersistenceError & dbError);
+    void hasKeys(uint64_t store, std::vector<char *> const & keyData, std::vector<uint32_t> const & keySize, std::vector<bool> & results, PersistenceError & dbError);
+    void removeKeys(uint64_t store, std::vector<char *> const & keyData, std::vector<uint32_t> const & keySize, int32_t & totalKeysRemoved, PersistenceError & dbError);
 
   };
 } } } } }

com.ibm.streamsx.dps/impl/include/CouchbaseDBLayer.h

@@ -203,8 +203,11 @@ namespace distributed
     bool acquireLock(uint64_t lock, double leaseTime, double maxWaitTimeToAcquireLock, PersistenceError & lkError);
     bool removeLock(uint64_t lock, PersistenceError & lkError);
     uint32_t getPidForLock(std::string const & name, PersistenceError & lkError);
-   void getKeys(uint64_t store, std::vector<unsigned char *> & keysBuffer, std::vector<uint32_t> & keysSize, int32_t keyStartPosition, int32_t numberOfKeysNeeded, PersistenceError & dbError);
-    void getValue(std::string const & storeIdString, char const * & key, uint32_t const & keySize, unsigned char * & value, uint32_t & valueSize, uint64_t & error);
+    void getKeys(uint64_t store, std::vector<unsigned char *> & keysBuffer, std::vector<uint32_t> & keysSize, int32_t keyStartPosition, int32_t numberOfKeysNeeded, PersistenceError & dbError);
+    void getValues(uint64_t store, std::vector<char *> const & keyData, std::vector<uint32_t> const & keySize, std::vector<unsigned char *> & valueData, std::vector<uint32_t> & valueSize, PersistenceError & dbError);
+    void putKVPairs(uint64_t store, std::vector<char *> const & keyData, std::vector<uint32_t> const & keySize, std::vector<unsigned char *> const & valueData, std::vector<uint32_t> const & valueSize, PersistenceError & dbError);
+    void hasKeys(uint64_t store, std::vector<char *> const & keyData, std::vector<uint32_t> const & keySize, std::vector<bool> & results, PersistenceError & dbError);
+    void removeKeys(uint64_t store, std::vector<char *> const & keyData, std::vector<uint32_t> const & keySize, int32_t & totalKeysRemoved, PersistenceError & dbError);
 
   };
 } } } } }

com.ibm.streamsx.dps/impl/include/DBLayer.h

@@ -289,9 +289,21 @@ namespace store {
     /// @return a list containing multiple keys of a given data type.
     virtual void getKeys(uint64_t store, std::vector<unsigned char *> & keysBuffer, std::vector<uint32_t> & keysSize, int32_t keyStartPosition, int32_t numberOfKeysNeeded, PersistenceError & dbError) = 0;
 
-    /// Get value for a given key present in a given store.
-    /// @return a value for a given key of a given data type.
-    virtual void getValue(std::string const & storeIdString, char const * & key, uint32_t const & keySize, unsigned char * & value, uint32_t & valueSize, uint64_t & error) = 0;
+    /// Get values for a given list of keys present in a given store.
+    /// @return a list containing multiple values for a given list of keys of a given data type.
+    virtual void getValues(uint64_t store, std::vector<char *> const & keyData, std::vector<uint32_t> const & keySize, std::vector<unsigned char *> & valueData, std::vector<uint32_t> & valueSize, PersistenceError & dbError) = 0;
+
+    /// Put (save) K/V pairs in a given store.
+    /// @return 0 if all put operations succeeded. Else, non-zero if one or more or all put failed.
+    virtual void putKVPairs(uint64_t store, std::vector<char *> const & keyData, std::vector<uint32_t> const & keySize, std::vector<unsigned char *> const & valueData, std::vector<uint32_t> const & valueSize, PersistenceError & dbError) = 0;
+
+    /// Check for the existence of a given list of keys in a given store.
+    /// @return a list containing the key existence check true or false results.
+    virtual void hasKeys(uint64_t store, std::vector<char *> const & keyData, std::vector<uint32_t> const & keySize, std::vector<bool> & results, PersistenceError & dbError) = 0;
+
+    /// Remove a given list of keys from a given store.
+    /// @return an integer value indicating the total number of keys removed.
+    virtual void removeKeys(uint64_t store, std::vector<char *> const & keyData, std::vector<uint32_t> const & keySize, int32_t & totalKeysRemoved, PersistenceError & dbError) = 0;
 
     /// A store iterator
     class Iterator