groupreduction and subgroupreduction

[brabreda]

I am unsure why my previous PR closed but here are the changes.

• I added docs
• I added tests

It was my first time writing tests, and they passed. How are these tested on GPUs, and do I need more tests?

[vchuravy]

Thank you! As you found the notion of a subgroup is not necessarily consistent across backends. So I am wondering if we need to expose that as feature or if backends could use them to implement reduction.

So as an example could we "just" add rval= @reduce(op, val) to perform a workgroup level reduction?

What is rval across all members of a workgroup. Is it the same or may it divergent and only the first workitem will receive a correct answer?

Do we need a broadcast/shuffle operation?

I don't have the answers to those questions.

I think adding @reduce is definitely something we want to do, but subgroup ops may need more thought.

[brabreda]

• rval=@reduce(op, val) is possible. In this case we only use localmem and @synchronize for the reduction.

• at the moment only the first thread of the group returns an item, but I think it would be possible to change this.

• I Think providing a neutral value will have better performance, but I am not sure. I Will check this.

I will fix this, meanwhile we can think about the wraps. If we will introduce it and in what way.

[vchuravy]

Yeah requiring a neutral value sounds good.

Could not each backend choose to use warp level ops, iff available? Instead of having the user make this decision.

[brabreda]

The problem with that approach is that warp intrinsics don’t work for every type.

We then would need something like this:

__reduce(op, val::union{type1, …},) for wraps
__reduce(op, val,) for everything else

In this case you don’t override the function. I am also not sure if this kind of type specialisation works where it takes the first function when it is one of the types. You probs have more expertise on this.

would it be prefered that every thread returns the right rval? I do think in that case I Will have to rethink the warp reduction bases groupreduce.

[vchuravy]

would it be prefered that every thread returns the right rval? I do think in that case I Will have to rethink the warp reduction bases groupreduce.

No it just needs to be documented :) and we might want to add a @broadcast.

So it looks like CUDA.jl makes this decision based on the element type?
https://github.com/JuliaGPU/CUDA.jl/blob/d57e02018ccf00ceda7672f5f4d98e2ceef9106d/src/mapreduce.jl#L177-L181

[vchuravy]

Suggested change

	"""
	@subgroupsize()
	returns the GPUs subgroupsize.
	"""
	macro subgroupsize()
	quote
	$__subgroupsize()
	end
	end

[vchuravy]

I see if we can do a subgroupreduce the memory we need here is much reduced.

[vchuravy]

I see the value of having a common implementation.

So I would define:

@reduce(op, val, neutral)
__reduce(op, val, neutral)

And then maybe:

__subgroupreduce & __subgroupsize (No @ version)
__can_subgroup_reduce(T) = false

And then we could define:

__reduce(__ctx___, op, val, neutral, ::Val{true})
__reduce(__ctx___, op, val, neutral, ::Val{false})

as you have here.

function __reduce(op, val::T, neutral::T) where T
     __reduce(op, val, neutral, Val(__can_subgroup_reduce(T)))
end

[brabreda]

Yeah so CUDA(and i think most GPU backends make the decision based on type). I was thinking if something like this would be possible:

For the implementation without warps:

@reduce(op,val,neutral)
__reduce(op,val,neutral)

And every back-end like CUDAKernels just implements something like this:

__reduce(op,val::T,neutral) where {T <: Union{Float32, Float64, etc...}}

It would allow for 1 common implementation and a few specializations that are done by the backends themselves. But,
I tried It today and I get a lot of dynamic invocation errors. But I am not sure what's at fault.

[vchuravy]

I recommed using Cthulhu, using CUDA and then using CUDA.device_code_typed interactive=true

[brabreda]

Only the group reduction without subgroups remains. It is ready to merge, once merged I will add support for Subgroups to the CUDAkernels, MetalKernels, and OneAPI.

[vchuravy]

Okay so this is API neutral? Pure addition and we don't need anything new?
Great! Otherwise we could have wrapped that in with #422

[vchuravy]

#262

[maleadt]

Workaround?