Add IN OUT bind support for temporary LOBs and add IN OUT bind support for LOBs fetched as String or Buffer with more than 32K data.

Author: cjbj

doc/api.md

@@ -1536,10 +1536,8 @@ the connection is still open.
 Open temporary LOB usage can be monitored using the view
 [`V$TEMPORARY_LOBS`](http://docs.oracle.com/database/122/ADLOB/managing-LOBs.htm#ADLOB45157).
 
-LOBs created with `createLob()` can be bound for IN and for OUT binds,
-but not for IN OUT binds.  This is to stop temporary LOB leaks where
-node-oracledb is unable to free resources held by the initial
-temporary LOB.
+LOBs created with `createLob()` can be bound for IN, IN OUT and OUT
+binds.
 
 See [Working with CLOB and BLOB Data](#lobhandling) and [LOB Bind Parameters](#lobbinds) for more information.
 
@@ -3971,10 +3969,7 @@ If the data is larger than can be handled as a String or Buffer in
 Node.js or node-oracledb, it will need to be streamed to
 a [Lob](#lobclass), as discussed later.
 See [LOB Bind Parameters](#lobbinds) for size considerations regarding
-LOB binds.  Note node-oracledb will internally use a temporary LOB to
-insert String or Buffer data larger than 32 KB.  This is transparent
-to the application but size limits on the JavaScript String or Buffer
-still apply.
+LOB binds.
 
 Given the table:
 
@@ -4167,11 +4162,6 @@ until [`lob.close()`](#lobclose) is called.  Database Administrators
 can track this usage by querying
 [`V$TEMPORARY_LOBS`](http://docs.oracle.com/database/122/ADLOB/managing-LOBs.htm#ADLOB45157).
 
-Temporary Lobs such as those created with `createLob()` can be bound
-as IN and as OUT binds, but not for IN OUT binds.  This is to stop
-temporary LOB leaks where node-oracledb is unable to free resources
-held by the initial temporary LOB.
-
 #### Passing a Lob Into PL/SQL
 
 The following insertion example is based on
@@ -4304,9 +4294,7 @@ If the data is larger than can be handled as a String or Buffer in
 Node.js or node-oracledb, it will need to be explicitly streamed to
 a [Lob](#lobclass), as discussed previously.
 See [LOB Bind Parameters](#lobbinds) for size considerations regarding
-LOB binds.  Note node-oracledb will internally use a temporary LOB to
-fetch String or Buffer data larger than 32 KB.  This is transparent to
-the application but size limits on the String or Buffer still apply.
+LOB binds.
 
 #### Selecting and Fetching Lob Instances
 
@@ -4975,14 +4963,9 @@ tables) or temporary LOBs (such as those created
 with [`createLob()`](#connectioncreatelob) or returned by some SQL and
 PL/SQL operations).
 
-Persistent LOBs can be bound with direction `BIND_IN`, `BIND_OUT` or
+LOBs can be bound with direction `BIND_IN`, `BIND_OUT` or
 `BIND_INOUT`, depending on context.
 
-Temporary LOBs can be bound with direction `BIND_IN` or `BIND_OUT`,
-but not with `BIND_INOUT`.  This is to prevent temporary LOB leaks
-where node-oracledb is unable to free resources held by the initial
-temporary LOB.
-
 Note that any PL/SQL OUT LOB parameter should be initialized in the
 PL/SQL block - even just to NULL - before the PL/SQL code
 completes. Make sure to do this in all PL/SQL code paths including in
@@ -5001,32 +4984,23 @@ size used for these binds in the OUT direction is 200, so set
 See [Working with CLOB and BLOB Data](#lobhandling) for examples and
 more information on binding and working with LOBs.
 
-#### Theoretical Limits (Bytes) for Binding LOBs to Strings and Buffers
-
-DB Type | Bind Type       | Direction   | Oracle Client 12c Limit<sup>*</sup> | Oracle Client 11.2 Limit<sup>*</sup>
---------|-----------------|-------------|-------------------------|------------------------
-CLOB    | oracledb.STRING |BIND_IN      |  1 GB                   | 64 KB
-CLOB    | oracledb.STRING |BIND_OUT     |  1 GB                   | 64 KB
-CLOB    | oracledb.STRING |BIND_INOUT   | 32 KB                   | 32 KB
-BLOB    | oracledb.BUFFER |BIND_IN      |  1 GB                   | 64 KB
-BLOB    | oracledb.BUFFER |BIND_OUT     |  1 GB                   | 64 KB
-BLOB    | oracledb.BUFFER |BIND_INOUT   | 32 KB                   | 32 KB
-
-<sup>*</sup>The largest usable data length is two bytes less than the
-size shown for 12c and one byte less for 11.2.
+#### Size Limits for Binding LOBs to Strings and Buffers
 
-In practice, the limitation on binding IN or OUT is the memory
-available to Node.js and the V8 engine.  For data larger than several
-megabytes, it is recommended to bind as `oracledb.CLOB` or
-`oracledb.BLOB` and use [Lob streaming](#streamsandlobs).  If you try
-to create large Strings or Buffers in Node.js you will see errors like
-*JavaScript heap out of memory*, or other space related messages.
+When CLOBs are bound as `oracledb.STRING`, and BLOBs are bound as
+`oracledb.BUFFER`, the theoretical maxium data length that can be
+bound is 2 bytes less than 1 GB.  When node-oracledb uses Oracle Client
+11.2 the limit is 1 byte less than 64 KB.
 
-Internally, for PL/SQL calls, node-oracledb uses temporary LOBs when
-binding Strings and Buffers larger than 32 KB.  Since temporary LOBs
-cannot be used for IN OUT binds, the data size in this case is
-restricted to 32 KB.  For SQL call no temporary LOBs are used.
+In practice, the limitation on binding is the memory available to
+Node.js and the V8 engine.  For data larger than several megabytes, it
+is recommended to bind as `oracledb.CLOB` or `oracledb.BLOB` and
+use [Lob streaming](#streamsandlobs).  If you try to create large
+Strings or Buffers in Node.js you will see errors like *JavaScript
+heap out of memory*, or other space related messages.
 
+Internally, temporary LOBs are used when binding Strings and Buffers
+larger than 32 KB for PL/SQL calls.  For SQL calls no temporary LOBs
+are used.
 
 ### <a name="plsqlindexbybinds"></a> 13.6 PL/SQL Collection Associative Array (Index-by) Bind Parameters
 

src/njs/src/njsConnection.cpp

@@ -2054,7 +2054,8 @@ void Connection::CLOB2String ( eBaton* executeBaton, unsigned int index )
   unsigned long long byteAmount  = 0;
   // Set the charAmount to (maxSize + 1) to know whether given maxSize good
   // enough to store PLSQL OUT data
-  unsigned long long charAmount  = executeBaton->binds[index]->maxSize + 1;
+  unsigned long long charAmount  =
+                     executeBaton->extBinds[index]->fields.extLob.maxSize + 1;
 
   executeBaton->binds[index]->type = DpiVarChar;
   LOB2StringOrBuffer ( executeBaton, index, byteAmount, charAmount );
@@ -2078,7 +2079,8 @@ void Connection::BLOB2Buffer ( eBaton* executeBaton, unsigned int index )
 {
   // Set the charAmount to (maxSize + 1) to know whether given maxSize good
   // enough to store PLSQL OUT data
-  unsigned long long byteAmount  = executeBaton->binds[index]->maxSize + 1;
+  unsigned long long byteAmount  =
+                     executeBaton->extBinds[index]->fields.extLob.maxSize + 1;
   unsigned long long charAmount  = 0;
 
   executeBaton->binds[index]->type = DpiRaw;
@@ -2141,7 +2143,8 @@ void Connection::LOB2StringOrBuffer ( eBaton* executeBaton, unsigned int index,
    * of charAmount or byteAmount passed to Lob::read()
    * If there is more data in the LOB than the maxSize, set the error
    */
-  if ( byteAmount > ( unsigned long long ) bind->maxSize )
+  if ( byteAmount > ( unsigned long long )
+                    executeBaton->extBinds[index]->fields.extLob.maxSize )
   {
     executeBaton->error = NJSMessages::getErrorMsg(
                                           errInsufficientBufferForBinds);
@@ -2313,7 +2316,9 @@ void Connection::Descr2StringOrBuffer ( eBaton* executeBaton )
         }
         else
         {
-          if ( *bind->ind != -1 )
+          if ( Lob::isTempLob ( executeBaton->dpienv->envHandle(),
+               executeBaton->dpiconn->getErrh (),
+               *( Descriptor** ) bind->value ) )
           {
             Lob::freeTempLob ( executeBaton->dpiconn->getSvch (),
                                executeBaton->dpiconn->getErrh (),
@@ -2355,12 +2360,14 @@ void Connection::ConvertStringOrBuffer2LOB ( eBaton* executeBaton,
 {
   Bind *bind = executeBaton->binds[index];
 
-  // In case of IN bind convert string/buffer to LOB if input data
-  // is not NULL and data size more than 32k
-  if ( !bind->isOut && *bind->ind != -1  &&
-       ( bind->len && *bind->len > NJS_THRESHOLD_SIZE_PLSQL_STRING_ARG ) )
+  // In case of IN or INOUT bind convert string/buffer to LOB if input data
+  // is not NULL and input data size more than 32k for strict IN binds or
+  // maxSize more than 32k for INOUT binds
+  if ( ( ( !bind->isOut || bind->isInOut ) && *bind->ind != -1 ) &&
+       ( ( !bind->isOut && *bind->len > NJS_THRESHOLD_SIZE_PLSQL_STRING_ARG ) ||
+         ( bind->isInOut && bind->maxSize > NJS_THRESHOLD_SIZE_PLSQL_STRING_ARG
+         ) ) )
   {
-    // This block is only for BIND_IN case
     ExtBind* extBind = new ExtBind ();
     if ( !extBind )
     {
@@ -2369,25 +2376,24 @@ void Connection::ConvertStringOrBuffer2LOB ( eBaton* executeBaton,
       goto exitConvertStringOrBuffer2LOB;
     }
     executeBaton->njsconn->InitExtBind ( extBind, NJS_EXTBIND_LOB );
-    // Set a flag to know that type conversion happened
-    // This helps in later steps to clean LOB resources
-    extBind->fields.extLob.isStringBuffer2LOB = true;
-    executeBaton->extBinds[index]             = extBind;
+    extBind->fields.extLob.maxSize = bind->maxSize;
 
     if ( bind->type == DpiVarChar )
     {
-      // Convert String to CLOB
       String2CLOB ( executeBaton, index );
     }
     else
     {
-      // Convert Buffer to BLOB
       Buffer2BLOB ( executeBaton, index );
     }
+    // Set a flag to know that type conversion happened
+    // This helps in later steps to clean LOB resources
+    extBind->fields.extLob.isStringBuffer2LOB = true;
+    executeBaton->extBinds[index]             = extBind;
   }
-  else if ( bind->maxSize > NJS_THRESHOLD_SIZE_PLSQL_STRING_ARG )
+  else if ( ( bind->isOut || bind->isInOut ) &&
+            bind->maxSize > NJS_THRESHOLD_SIZE_PLSQL_STRING_ARG )
   {
-    // This block is only for BIND_OUT case
     ExtBind* extBind = new ExtBind ();
 
     if ( !extBind )
@@ -2397,6 +2403,7 @@ void Connection::ConvertStringOrBuffer2LOB ( eBaton* executeBaton,
       goto exitConvertStringOrBuffer2LOB;
     }
     executeBaton->njsconn->InitExtBind ( extBind, NJS_EXTBIND_LOB );
+    extBind->fields.extLob.maxSize = bind->maxSize;
 
     // Set a flag to know that type conversion happened
     // This helps in later steps for converting LOB to String/Buffer and
@@ -2476,6 +2483,7 @@ void Connection::InitExtBind ( ExtBind *extBind, ExtBindType fieldType )
   {
     extBind->fields.extLob.value              = NULL;
     extBind->fields.extLob.isStringBuffer2LOB = false;
+    extBind->fields.extLob.maxSize            = 0;
   }
 }
 
@@ -2514,9 +2522,8 @@ void Connection::PrepareAndBind (eBaton* executeBaton)
         if ( ( executeBaton->st == DpiStmtBegin ||
                executeBaton->st == DpiStmtDeclare ||
                executeBaton->st == DpiStmtCall ) &&
-             ( !executeBaton->binds[index]->isInOut &&
              ( executeBaton->binds[index]->type == DpiVarChar ||
-               executeBaton->binds[index]->type == DpiRaw ) ) )
+               executeBaton->binds[index]->type == DpiRaw ) )
         {
           ConvertStringOrBuffer2LOB ( executeBaton, index );
         }
@@ -2590,9 +2597,8 @@ void Connection::PrepareAndBind (eBaton* executeBaton)
         if ( ( executeBaton->st == DpiStmtBegin ||
                executeBaton->st == DpiStmtDeclare ||
                executeBaton->st == DpiStmtCall ) &&
-             ( !executeBaton->binds[index]->isInOut &&
              ( executeBaton->binds[index]->type == DpiVarChar ||
-               executeBaton->binds[index]->type == DpiRaw ) ) )
+               executeBaton->binds[index]->type == DpiRaw ) )
         {
           ConvertStringOrBuffer2LOB ( executeBaton, index );
         }

src/njs/src/njsConnection.h

@@ -165,6 +165,8 @@ typedef struct ExtBind
       void *value;             // Stores extra bind reference
       bool isStringBuffer2LOB; // Specifies whether string or buffer Converted
                                // to LOB or not
+      DPI_SZ_TYPE maxSize;     // Size for the OUT or IN OUT bind value
+
     } extLob;
   } fields;
 } ExtBind;

src/njs/src/njsIntLob.cpp

@@ -370,12 +370,7 @@ NJSErrorType ILob::preBind ( Bind *bind )
   {
     bind->type = this->dpiLobType_;
 
-    // Temp LOB INOUT bind not supported, check for it.
-    if ( this->isTempLob_ && bind->isInOut )
-    {
-      errNum = errTempLOBINOUTBind;
-    }
-    else if ( !this->isValid_ || this->state_ == NJS_ACTIVE )
+    if ( !this->isValid_ || this->state_ == NJS_ACTIVE )
     {
       errNum = this->getErrNumber ( true );
     }

src/njs/src/njsMessages.cpp

@@ -85,10 +85,9 @@ static const char *errMsg[] =
   "NJS-046: pool alias \"%s\" already exists in the connection pool cache", // errPoolWithAliasAlreadyExists
   "NJS-047: pool alias \"%s\" not found in the connection pool cache", // errPoolWithAliasNotFound
   "NJS-048: operation not permitted while Lob object is active in a bind operation", // errLOBBindActive
-  "NJS-049: cannot use bind direction IN OUT for temporary LOBs", // errTempLOBINOUTBind
-  "NJS-050: Temporary LOBs were open when the connection was closed", // errBusyConnTEMPLOB
-  "NJS-051: data must be shorter than %d", // errBindValueTooLarge
-  "NJS-052: \"%s\" must be less than %d", // errMaxValueTooLarge
+  "NJS-049: Temporary LOBs were open when the connection was closed", // errBusyConnTEMPLOB
+  "NJS-050: data must be shorter than %d", // errBindValueTooLarge
+  "NJS-051: \"%s\" must be less than %d", // errMaxValueTooLarge
 };
 
 string NJSMessages::getErrorMsg ( NJSErrorType err, ... )

src/njs/src/njsMessages.h

@@ -84,7 +84,6 @@ typedef enum
   errPoolWithAliasAlreadyExists,
   errPoolWithAliasNotFound,
   errLOBBindActive,
-  errTempLOBINOUTBind,
   errBusyConnTEMPLOB,
   errBindValueTooLarge,
   errMaxValueTooLarge,