enable DumpBuffers to work with USM allocations (#201)

Author: bashbaug

docs/controls.md

@@ -481,43 +481,43 @@ If set, the Intercept Layer for OpenCL Applications will dump the contents of a
 
 ##### `DumpBuffersBeforeEnqueue` (bool)
 
-If set to a nonzero value, the Intercept Layer for OpenCL Applications will dump buffers before calls to clEnqueueNDRangeKernel().  Only buffers that are kernel arguments for the kernel being enqueued are dumped.  Buffers are dumped as raw binary data to a "memDumpPreEnqueue" subdirectory of the dump directory.  The filenames will have the form "Enqueue\_\<Enqueue Number\>\_Kernel\_\<Kernel Name\>\_Arg\_\<Argument Number\>\_Buffer\_\<Unique Memory Object Number\>.bin".
+If set to a nonzero value, the Intercept Layer for OpenCL Applications will dump buffer, SVM, and USM kernel arguments before calls to clEnqueueNDRangeKernel().  Only buffers that are kernel arguments for the kernel being enqueued are dumped.  Buffers are dumped as raw binary data to a "memDumpPreEnqueue" subdirectory of the dump directory.  The filenames will have the form "Enqueue\_\<Enqueue Number\>\_Kernel\_\<Kernel Name\>\_Arg\_\<Argument Number\>\_Buffer\_\<Unique Memory Object Number\>.bin".
 
 ##### `DumpBuffersAfterEnqueue` (bool)
 
-If set to a nonzero value, the Intercept Layer for OpenCL Applications will dump buffers after calls to clEnqueueNDRangeKernel().  Only buffers that are kernel arguments for the kernel being enqueued are dumped.  Buffers are dumped as raw binary data to a "memDumpPostEnqueue" subdirectory of the dump directory.  The filenames will have the form "Enqueue\_\<Enqueue Number\>\_Kernel\_\<Kernel Name\>\_Arg\_\<Argument Number\>\_Buffer\_\<Unique Memory Object Number\>.bin".  Note that this is the same naming convention as with DumpBuffersBeforeEnqueue, so the changes resulting from an enqueue can be determined by diff'ing the preEnqueue folder with the postEnqueue folder.
+If set to a nonzero value, the Intercept Layer for OpenCL Applications will dump buffer, SVM, and USM kernel arguments after calls to clEnqueueNDRangeKernel().  Only buffers that are kernel arguments for the kernel being enqueued are dumped.  Buffers are dumped as raw binary data to a "memDumpPostEnqueue" subdirectory of the dump directory.  The filenames will have the form "Enqueue\_\<Enqueue Number\>\_Kernel\_\<Kernel Name\>\_Arg\_\<Argument Number\>\_Buffer\_\<Unique Memory Object Number\>.bin".  Note that this is the same naming convention as with DumpBuffersBeforeEnqueue, so the changes resulting from an enqueue can be determined by diff'ing the preEnqueue folder with the postEnqueue folder.
 
 ##### `DumpBuffersForKernel` (string)
 
-If set, the Intercept Layer for OpenCL Applications will only dump buffers when the specified kernel is enqueued.  This control is ignored unless DumpBuffersBeforeEnqueue or DumpBuffersAfterEnqueue are enabled.
+If set, the Intercept Layer for OpenCL Applications will only dump buffer, SVM, and USM kernel arguments when the specified kernel is enqueued.  This control is ignored unless DumpBuffersBeforeEnqueue or DumpBuffersAfterEnqueue are enabled.
 
 ##### `DumpImagesBeforeEnqueue` (bool)
 
-If set to a nonzero value, the Intercept Layer for OpenCL Applications will dump images before calls to clEnqueueNDRangeKernel().  Only images that are kernel arguments for the kernel being enqueued are dumped.  Images are dumped as raw binary data to a "memDumpPreEnqueue" subdirectory of the dump directory.  The filenames will have the form "Enqueue\_\<Enqueue Number\>\_Kernel\_\<Kernel Name\>\_Arg\_\<Argument Number\>\_Image\_\<Unique Memory Object Number\>\_\<Width\>x\<Height\>x\<Depth\>\_\<Element Size\>bpp.raw".
+If set to a nonzero value, the Intercept Layer for OpenCL Applications will dump image kernel arguments before calls to clEnqueueNDRangeKernel().  Only images that are kernel arguments for the kernel being enqueued are dumped.  Images are dumped as raw binary data to a "memDumpPreEnqueue" subdirectory of the dump directory.  The filenames will have the form "Enqueue\_\<Enqueue Number\>\_Kernel\_\<Kernel Name\>\_Arg\_\<Argument Number\>\_Image\_\<Unique Memory Object Number\>\_\<Width\>x\<Height\>x\<Depth\>\_\<Element Size\>bpp.raw".
 
 ##### `DumpImagesAfterEnqueue` (bool)
 
-If set to a nonzero value, the Intercept Layer for OpenCL Applications will dump images after calls to clEnqueueNDRangeKernel().  Only images that are kernel arguments for the kernel being enqueued are dumped.  Images are dumped as raw binary data to a "memDumpPostEnqueue" subdirectory of the dump directory.  The filenames will have the form "Enqueue\_\<Enqueue Number\>\_Kernel\_\<Kernel Name\>\_Arg\_\<Argument Number\>\_Image\_\<Unique Memory Object Number\>\_\<Width\>x\<Height\>x\<Depth\>\_\<Element Size\>bpp.raw".  Note that this is the same naming convention as with DumpImagesBeforeEnqueue, so the changes resulting from an enqueue can be determined by diff'ing the preEnqueue folder with the postEnqueue folder.
+If set to a nonzero value, the Intercept Layer for OpenCL Applications will dump image kernel arguments after calls to clEnqueueNDRangeKernel().  Only images that are kernel arguments for the kernel being enqueued are dumped.  Images are dumped as raw binary data to a "memDumpPostEnqueue" subdirectory of the dump directory.  The filenames will have the form "Enqueue\_\<Enqueue Number\>\_Kernel\_\<Kernel Name\>\_Arg\_\<Argument Number\>\_Image\_\<Unique Memory Object Number\>\_\<Width\>x\<Height\>x\<Depth\>\_\<Element Size\>bpp.raw".  Note that this is the same naming convention as with DumpImagesBeforeEnqueue, so the changes resulting from an enqueue can be determined by diff'ing the preEnqueue folder with the postEnqueue folder.
 
 ##### `DumpImagesForKernel` (string)
 
-If set, the Intercept Layer for OpenCL Applications will only dump image when the specified kernel is enqueued.  This control is ignored unless DumpImagesBeforeEnqueue or DumpImagesAfterEnqueue are enabled.
+If set, the Intercept Layer for OpenCL Applications will only dump image kernel arguments when the specified kernel is enqueued.  This control is ignored unless DumpImagesBeforeEnqueue or DumpImagesAfterEnqueue are enabled.
 
 ##### `DumpBuffersMinEnqueue` (cl_uint)
 
-The Intercept Layer for OpenCL Applications will only dump buffers when the enqueue counter is greater than this value, inclusive.
+The Intercept Layer for OpenCL Applications will only dump buffer, SVM, and USM kernel arguments when the enqueue counter is greater than this value, inclusive.
 
 ##### `DumpBuffersMaxEnqueue` (cl_uint)
 
-The Intercept Layer for OpenCL Applications will only dump buffers when the enqueue counter is less than this value, inclusive.
+The Intercept Layer for OpenCL Applications will only dump buffer, SVM, and USM kernel arguments when the enqueue counter is less than this value, inclusive.
 
 ##### `DumpImagesMinEnqueue` (cl_uint)
 
-The Intercept Layer for OpenCL Applications will only dump images when the enqueue counter is greater than this value, inclusive.
+The Intercept Layer for OpenCL Applications will only dump image kernel arguments when the enqueue counter is greater than this value, inclusive.
 
 ##### `DumpImagesMaxEnqueue` (cl_uint)
 
-The Intercept Layer for OpenCL Applications will only dump images when the enqueue counter is less than this value, inclusive.
+The Intercept Layer for OpenCL Applications will only dump image kernel arguments when the enqueue counter is less than this value, inclusive.
 
 ### Device Partitioning Controls
 

intercept/src/controls.h

@@ -126,16 +126,16 @@ CLI_CONTROL( bool,          DumpArgumentsOnSet,                     false, "If s
 CLI_CONTROL( bool,          DumpBuffersAfterCreate,                 false, "If set, the Intercept Layer for OpenCL Applications will dump buffers to a file after creation.  This control still honors the enqueue counter limits, even though no enqueues are involved during buffer creation.  Currently only works for cl_mem buffers created from host pointers." )
 CLI_CONTROL( bool,          DumpBuffersAfterMap,                    false, "If set, the Intercept Layer for OpenCL Applications will dump the contents of a buffer to a file after the buffer is mapped.  Only valid if the buffer is NOT mapped with CL_MAP_WRITE_INVALIDATE_REGION.  If the buffer was mapped non-blocking, this may insert a clFinish() into the command queue, which may have functional or performance implications." )
 CLI_CONTROL( bool,          DumpBuffersBeforeUnmap,                 false, "If set, the Intercept Layer for OpenCL Applications will dump the contents of a buffer to a file immediately before the buffer is unmapped.  This is done by inserting a blocking clEnqueueMapBuffer() (and matching clEnqueueUnmapMemObject()) into the command queue, which may have functional or performance implications." )
-CLI_CONTROL( bool,          DumpBuffersBeforeEnqueue,               false, "If set to a nonzero value, the Intercept Layer for OpenCL Applications will dump buffers before calls to clEnqueueNDRangeKernel().  Only buffers that are kernel arguments for the kernel being enqueued are dumped.  Buffers are dumped as raw binary data to a \"memDumpPreEnqueue\" subdirectory of the dump directory.  The filenames will have the form \"Enqueue_<Enqueue Number>_Kernel_<Kernel Name>_Arg_<Argument Number>_Buffer_<Unique Memory Object Number>.bin\"." )
-CLI_CONTROL( bool,          DumpBuffersAfterEnqueue,                false, "If set to a nonzero value, the Intercept Layer for OpenCL Applications will dump buffers after calls to clEnqueueNDRangeKernel().  Only buffers that are kernel arguments for the kernel being enqueued are dumped.  Buffers are dumped as raw binary data to a \"memDumpPostEnqueue\" subdirectory of the dump directory.  The filenames will have the form \"Enqueue_<Enqueue Number>_Kernel_<Kernel Name>_Arg_<Argument Number>_Buffer_<Unique Memory Object Number>.bin\".  Note that this is the same naming convention as with DumpBuffersBeforeEnqueue, so the changes resulting from an enqueue can be determined by diff'ing the preEnqueue folder with the postEnqueue folder." )
-CLI_CONTROL( std::string,   DumpBuffersForKernel,                   "",    "If set, the Intercept Layer for OpenCL Applications will only dump buffers when the specified kernel is enqueued.  This control is ignored unless DumpBuffersBeforeEnqueue or DumpBuffersAfterEnqueue are enabled." )
-CLI_CONTROL( bool,          DumpImagesBeforeEnqueue,                false, "If set to a nonzero value, the Intercept Layer for OpenCL Applications will dump images before calls to clEnqueueNDRangeKernel().  Only images that are kernel arguments for the kernel being enqueued are dumped.  Images are dumped as raw binary data to a \"memDumpPreEnqueue\" subdirectory of the dump directory.  The filenames will have the form \"Enqueue_<Enqueue Number>_Kernel_<Kernel Name>_Arg_<Argument Number>_Image_<Unique Memory Object Number>_<Width>x<Height>x<Depth>_<Element Size>bpp.raw\"." )
-CLI_CONTROL( bool,          DumpImagesAfterEnqueue,                 false, "If set to a nonzero value, the Intercept Layer for OpenCL Applications will dump images after calls to clEnqueueNDRangeKernel().  Only images that are kernel arguments for the kernel being enqueued are dumped.  Images are dumped as raw binary data to a \"memDumpPostEnqueue\" subdirectory of the dump directory.  The filenames will have the form \"Enqueue_<Enqueue Number>_Kernel_<Kernel Name>_Arg_<Argument Number>_Image_<Unique Memory Object Number>_<Width>x<Height>x<Depth>_<Element Size>bpp.raw\".  Note that this is the same naming convention as with DumpImagesBeforeEnqueue, so the changes resulting from an enqueue can be determined by diff'ing the preEnqueue folder with the postEnqueue folder." )
-CLI_CONTROL( std::string,   DumpImagesForKernel,                    "",    "If set, the Intercept Layer for OpenCL Applications will only dump image when the specified kernel is enqueued.  This control is ignored unless DumpImagesBeforeEnqueue or DumpImagesAfterEnqueue are enabled." )
-CLI_CONTROL( cl_uint,       DumpBuffersMinEnqueue,                  0,     "The Intercept Layer for OpenCL Applications will only dump buffers when the enqueue counter is greater than this value, inclusive." )
-CLI_CONTROL( cl_uint,       DumpBuffersMaxEnqueue,                  UINT_MAX, "The Intercept Layer for OpenCL Applications will only dump buffers when the enqueue counter is less than this value, inclusive." )
-CLI_CONTROL( cl_uint,       DumpImagesMinEnqueue,                   0,     "The Intercept Layer for OpenCL Applications will only dump images when the enqueue counter is greater than this value, inclusive." )
-CLI_CONTROL( cl_uint,       DumpImagesMaxEnqueue,                   UINT_MAX, "The Intercept Layer for OpenCL Applications will only dump images when the enqueue counter is less than this value, inclusive." )
+CLI_CONTROL( bool,          DumpBuffersBeforeEnqueue,               false, "If set to a nonzero value, the Intercept Layer for OpenCL Applications will dump buffer, SVM, and USM kernel arguments before calls to clEnqueueNDRangeKernel().  Only buffers that are kernel arguments for the kernel being enqueued are dumped.  Buffers are dumped as raw binary data to a \"memDumpPreEnqueue\" subdirectory of the dump directory.  The filenames will have the form \"Enqueue_<Enqueue Number>_Kernel_<Kernel Name>_Arg_<Argument Number>_Buffer_<Unique Memory Object Number>.bin\"." )
+CLI_CONTROL( bool,          DumpBuffersAfterEnqueue,                false, "If set to a nonzero value, the Intercept Layer for OpenCL Applications will dump buffer, SVM, and USM kernel arguments after calls to clEnqueueNDRangeKernel().  Only buffers that are kernel arguments for the kernel being enqueued are dumped.  Buffers are dumped as raw binary data to a \"memDumpPostEnqueue\" subdirectory of the dump directory.  The filenames will have the form \"Enqueue_<Enqueue Number>_Kernel_<Kernel Name>_Arg_<Argument Number>_Buffer_<Unique Memory Object Number>.bin\".  Note that this is the same naming convention as with DumpBuffersBeforeEnqueue, so the changes resulting from an enqueue can be determined by diff'ing the preEnqueue folder with the postEnqueue folder." )
+CLI_CONTROL( std::string,   DumpBuffersForKernel,                   "",    "If set, the Intercept Layer for OpenCL Applications will only dump buffer, SVM, and USM kernel arguments when the specified kernel is enqueued.  This control is ignored unless DumpBuffersBeforeEnqueue or DumpBuffersAfterEnqueue are enabled." )
+CLI_CONTROL( bool,          DumpImagesBeforeEnqueue,                false, "If set to a nonzero value, the Intercept Layer for OpenCL Applications will dump image kernel arguments before calls to clEnqueueNDRangeKernel().  Only images that are kernel arguments for the kernel being enqueued are dumped.  Images are dumped as raw binary data to a \"memDumpPreEnqueue\" subdirectory of the dump directory.  The filenames will have the form \"Enqueue_<Enqueue Number>_Kernel_<Kernel Name>_Arg_<Argument Number>_Image_<Unique Memory Object Number>_<Width>x<Height>x<Depth>_<Element Size>bpp.raw\"." )
+CLI_CONTROL( bool,          DumpImagesAfterEnqueue,                 false, "If set to a nonzero value, the Intercept Layer for OpenCL Applications will dump image kernel arguments after calls to clEnqueueNDRangeKernel().  Only images that are kernel arguments for the kernel being enqueued are dumped.  Images are dumped as raw binary data to a \"memDumpPostEnqueue\" subdirectory of the dump directory.  The filenames will have the form \"Enqueue_<Enqueue Number>_Kernel_<Kernel Name>_Arg_<Argument Number>_Image_<Unique Memory Object Number>_<Width>x<Height>x<Depth>_<Element Size>bpp.raw\".  Note that this is the same naming convention as with DumpImagesBeforeEnqueue, so the changes resulting from an enqueue can be determined by diff'ing the preEnqueue folder with the postEnqueue folder." )
+CLI_CONTROL( std::string,   DumpImagesForKernel,                    "",    "If set, the Intercept Layer for OpenCL Applications will only dump image kernel arguments when the specified kernel is enqueued.  This control is ignored unless DumpImagesBeforeEnqueue or DumpImagesAfterEnqueue are enabled." )
+CLI_CONTROL( cl_uint,       DumpBuffersMinEnqueue,                  0,     "The Intercept Layer for OpenCL Applications will only dump buffer, SVM, and USM kernel arguments when the enqueue counter is greater than this value, inclusive." )
+CLI_CONTROL( cl_uint,       DumpBuffersMaxEnqueue,                  UINT_MAX, "The Intercept Layer for OpenCL Applications will only dump buffer, SVM, and USM kernel arguments when the enqueue counter is less than this value, inclusive." )
+CLI_CONTROL( cl_uint,       DumpImagesMinEnqueue,                   0,     "The Intercept Layer for OpenCL Applications will only dump image kernel arguments when the enqueue counter is greater than this value, inclusive." )
+CLI_CONTROL( cl_uint,       DumpImagesMaxEnqueue,                   UINT_MAX, "The Intercept Layer for OpenCL Applications will only dump image kernel arguments when the enqueue counter is less than this value, inclusive." )
 
 CLI_CONTROL_SEPARATOR( Device Partitioning Controls: )
 CLI_CONTROL( bool,          AutoPartitionAllDevices,                false, "If set to a nonzero value, the Intercept Layer for OpenCL Applications will automatically partition parent devices and return all parent devices and all sub-devices." )

intercept/src/dispatch.cpp

@@ -8457,6 +8457,7 @@ CL_API_ENTRY void* CL_API_CALL clHostMemAllocINTEL(
             }
 
             CPU_PERFORMANCE_TIMING_END();
+            ADD_USM_ALLOCATION( retVal, size );
             USM_ALLOC_PROPERTIES_CLEANUP( newProperties );
             CHECK_ERROR( errcode_ret[0] );
             CALL_LOGGING_EXIT( errcode_ret[0], "returned %p", retVal );
@@ -8532,6 +8533,7 @@ CL_API_ENTRY void* CL_API_CALL clDeviceMemAllocINTEL(
             }
 
             CPU_PERFORMANCE_TIMING_END();
+            ADD_USM_ALLOCATION( retVal, size );
             USM_ALLOC_PROPERTIES_CLEANUP( newProperties );
             CHECK_ERROR( errcode_ret[0] );
             CALL_LOGGING_EXIT( errcode_ret[0], "returned %p", retVal );
@@ -8607,6 +8609,7 @@ CL_API_ENTRY void* CL_API_CALL clSharedMemAllocINTEL(
             }
 
             CPU_PERFORMANCE_TIMING_END();
+            ADD_USM_ALLOCATION( retVal, size );
             USM_ALLOC_PROPERTIES_CLEANUP( newProperties );
             CHECK_ERROR( errcode_ret[0] );
             CALL_LOGGING_EXIT( errcode_ret[0], "returned %p", retVal );
@@ -8643,6 +8646,7 @@ CL_API_ENTRY cl_int CL_API_CALL clMemFreeINTEL(
                 ptr );
 
             CPU_PERFORMANCE_TIMING_END();
+            REMOVE_USM_ALLOCATION( ptr );
             CHECK_ERROR( retVal );
             CALL_LOGGING_EXIT( retVal );
 
@@ -8679,6 +8683,7 @@ clMemBlockingFreeINTEL(
                 ptr );
 
             CPU_PERFORMANCE_TIMING_END();
+            REMOVE_USM_ALLOCATION( ptr );
             CHECK_ERROR( retVal );
             CALL_LOGGING_EXIT( retVal );
 
@@ -8757,6 +8762,7 @@ CL_API_ENTRY cl_int CL_API_CALL clSetKernelArgMemPointerINTEL(
                 arg_index,
                 arg_value );
             CHECK_KERNEL_ARG_USM_POINTER( kernel, arg_value );
+            SET_KERNEL_ARG_USM_POINTER( kernel, arg_index, arg_value );
             CPU_PERFORMANCE_TIMING_START();
 
             cl_int  retVal = dispatchX.clSetKernelArgMemPointerINTEL(