Dirichlet Prior - cannot bufferize concentration

[fweberling]

Hi everyone,

I would like to run a GP Model with a Dirichlet prior on a GPU. Thus, I created a Dirichlet prior based on the torch Dirichlet distribution:

# Dirichlet prior
class DirichletPrior(gpytorch.priors.Prior, torch.distributions.dirichlet.Dirichlet):
  """
  Dirichlet prior
  """

  def __init__(self, concentration, validate_args=None, transform=None):
    TModule.__init__(self)
    torch.distributions.dirichlet.Dirichlet.__init__(self, concentration, validate_args=validate_args)
    gpytorch.priors.utils._bufferize_attributes(self, ("concentration"))
    self._transform = transform 

  def expand(self, batch_shape):
    batch_shape = torch.Size(batch_shape)
    return DirichletPrior(self.concentration.expand(batch_shape))

However, it seems that the bufferize_attributes method does not work. I always get this error message:

---> 29 outputscale_prior = DirichletPrior(self.concentration)

/content/utils.py in init(self, concentration, validate_args, transform)
66 TModule.__init__(self)
67 torch.distributions.dirichlet.Dirichlet.__init__(self, concentration, validate_args=validate_args)
---> 68 gpytorch.priors.utils._bufferize_attributes(self, ("concentration"))
69 self._transform = transform
70

/usr/local/lib/python3.7/dist-packages/gpytorch/priors/utils.py in _bufferize_attributes(module, attributes)
3
4 def _bufferize_attributes(module, attributes):
----> 5 attr_clones = {attr: getattr(module, attr).clone() for attr in attributes}
6 for attr, value in attr_clones.items():
7 delattr(module, attr)

/usr/local/lib/python3.7/dist-packages/gpytorch/priors/utils.py in (.0)
3
4 def _bufferize_attributes(module, attributes):
----> 5 attr_clones = {attr: getattr(module, attr).clone() for attr in attributes}
6 for attr, value in attr_clones.items():
7 delattr(module, attr)

/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py in getattr(self, name)
1129 return modules[name]
1130 raise AttributeError("'{}' object has no attribute '{}'".format(
-> 1131 type(self).__name__, name))
1132
1133 def __setattr__(self, name: str, value: Union[Tensor, 'Module']) -> None:

AttributeError: 'DirichletPrior' object has no attribute 'c'

Does someone know what I did wrong or can point me to another way to implement the Dirichlet prior?

Thank you in advance!
Friedrich

[gpleiss]

What parameter are you placing the Dirichlet prior over? Can you include a complete code example?

[fweberling]

I was able to solve it. It seemed that "concentration" was not recognised as a string in gpytorch.priors.utils._bufferize_attributes(self, ("concentration")). Adding a comma removes this issue gpytorch.priors.utils._bufferize_attributes(self, ("concentration",))

Since there are no Dirichlet priors so far implemented in GPyTorch and this one works now, should I open a pull request?

[gpleiss]

I'd be open to that, but can you specify what the use case is? What parameters are you placing a Dirichlet prior over?

[fweberling]

I have a sum of different kernels (e.g. RBF, Matern, Linear) and each kernel is equipped with an outputscale. I place the Dirichlet prior over these outputscales to enforce sparsity during hyperparameter optimisation

[gpleiss]

Gotcha. This does seem like it might be something that is for your specific use case, and it's won't be as straight forward to apply as something like the Gaussian/gamma prior (which control parameters independently, rather than controlling interactions between a set of parameters). I'd be open to a PR, but it should carefully document a use case / how it should be applied.