Uncaught exception: file too big for single read

[Bindong18]

I encountered the following issue when reading the content of .OVA file “test.ova” (its size is 2.1GB).
Uncaught exception: file too big for single read
e:/RailsInstaller/Ruby2.3.0/lib/ruby/gems/2.3.0/gems/rbvmomi-1.10.0/lib/rbvmomi/vim/OvfManager.rb:43:in read' e:/RailsInstaller/Ruby2.3.0/lib/ruby/gems/2.3.0/gems/rbvmomi-1.10.0/lib/rbvmomi/vim/OvfManager.rb:43:in deployOVF'

My sample code is as follows:
vim = RbVmomi::VIM.connect :host => hyper, :user => user, :password => password, :insecure => true
dc = vim.serviceInstance.find_datacenter(dcName) or abort "datacenter not found"
network = dc.network.first #Network
computeResourceArray = dc.hostFolder.children #Array for ComputeResource
computeResourceForDeploy=nil #ComputeResource
computeResourceForDeploy = computeResourceArray.find { |x| x.host.first.summary.config.name == 'crName' }
hostForDeploy=nil #HostSystem
computeResourceArray.each do |x|
host = x.host.first
if host.summary.config.name == 'hostName'
hostForDeploy = host
break;
end
end
dataStoreArray = dc.datastore #Array for DataStore
dataStoreDeploy=nil #DataStore
dataStoreDeploy = dataStoreArray.find { |x| x.name ==’dsName’ }
VIM = RbVmomi::VIM
root_vm_folder = dc.vmFolder
vm_folder = root_vm_folder
vm_name = 'test327'
ovf_url='//fileServer/test.ova'
vm = vim.serviceContent.ovfManager.deployOVF(
uri: ovf_url,
vmName: vm_name,
vmFolder: vm_folder,
host: hostForDeploy,
resourcePool: computeResourceForDeploy.resourcePool,
datastore: dataStoreDeploy,
)
Did anyone encounter the same issue ? Does anyone know how to resolve it?

[Bindong18]

Hi ,

Could you pls. tell me if there is any update?

Thank you very much

[jrgarcia]

@Bindong18 Unfortunately, I haven't been able to track this down. What version of Windows are you using? From the looks of it, you installed Ruby via the Rails Installer. Is that the case?

[Bindong18]

@jrgarcia windows: win2008r2 ,Yes Rails Installer3.3.0

[jrgarcia]

So it looks like this is an issue with reading large files with Ruby. You can see here where we call .read. Ruby can't handle reading a file of that size in that way. I'll look at changing this for reading large files, but I probably won't be able to modify that this week (I'm traveling right now).

[bergholdt]

If you are using 32bit then you have a limit of 2GB (https://en.wikipedia.org/wiki/2_GB_limit)

Could you check if your Ruby is 32 or 64 bit.

[Bindong18]

@bergholdt
C:\Users\administrator>ruby -v
ruby 2.3.3p222 (2016-11-21 revision 56859) [i386-mingw32]

C:\Users\administrator>irb
irb(main):001:0> ['foo'].pack('p').size
=> 4

[jrgarcia]

@Bindong18 Yeah, it looks like you have a 32-bit version of Ruby installed (i386-mingw32). This means there is no way to read a file that large. When I get a chance, I'll test to see if there is a way to stream this rather than just reading it that will work, but this will never work with 32-bit.

[Bindong18]

@jrgarcia
@bergholdt
I changed ruby to 64bit, but still got the same error:

Uncaught exception: file too big for single read
C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/rbvmomi-1.10.0/lib/rbvmomi/vim/OvfManager.rb:43:in read' C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/rbvmomi-1.10.0/lib/rbvmomi/vim/OvfManager.rb:43:in deployOVF'

C:\Users\administrator.CRE3>ruby -v
ruby 2.3.3p222 (2016-11-21 revision 56859) [x64-mingw32]

[jrgarcia]

@Bindong18 can you output ['foo'].pack('p').size again? If it's 8, then there isn't a problem with the OS. If it's 4, Ruby is still somehow installed as 32-bit.

Aside from that, I'm guessing the problem is just that Ruby itself can not read a file that large. I can change the implementation from the stdlib File#read, but I'll need to test it quite a bit to ensure I don't break anything that worked previously.

[Bindong18]

@jrgarcia

C:\Users\administrator.CRE3>irb
irb(main):001:0> ['foo'].pack('p').size
=> 8
irb(main):002:0>

[jrgarcia]

Ok, so the problem then is that Ruby's File#read can't read files that large. I'll look at changing to something that can handle large files.

[Bindong18]

@jrgarcia
Thank you very much