Skip to content

Conversation

@grantbruer
Copy link
Contributor

This PR updates the coupling layer in the conditional Glow implementation.

The coupling layer does the following:

  1. splits the state x in half,
  2. feeds the condition y and half of x to a smaller neural network,
  3. and uses the output of that smaller network to update the other half of x.

This PR makes the minimal changes required to allow x to be a scalar value. When the state is split in half, this PR updates the code so that the "half" that goes to the smaller network is empty while the "half" that is updated has size 1.

@grantbruer
Copy link
Contributor Author

This overlaps with #86. @rafaelorozco Are you okay with merging the changes in this PR? Will it mess up your WIP PR?

@rafaelorozco
Copy link
Collaborator

This overlaps with #86. @rafaelorozco Are you okay with merging the changes in this PR? Will it mess up your WIP PR?

Yes you can merge. We only need to worry about those GPU tests. They are tied to a SLIM computer that I believe keeps turning off. We need to move it to another one that is more resilient.

@grantbruer
Copy link
Contributor Author

Thanks for getting the gpu tests running. I'll try to figure out why the gpu test is failing before I merge.

@grantbruer
Copy link
Contributor Author

I tested this on my gpu, and it passes the tests now. I think this PR is ready to merge now, once we get the gpu CI running.

I changed the step size for a couple of the gradient tests, and in test_conditional_glow_network.jl, I reseeded the RNG more often to make the results the same whether it uses the cpu or gpu. (Every launch of a CUDA kernel generates a random number to seed the kernel's RNG, so the seeds needs to be reset to make the tests the same for cpu and gpu.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants