Observation Likelihoods

Here we have all the observation likelihood models.

Univariate Observation Likelihoods

dkregression.likelihoods.UnivariateGaussianLikelihood(mu_correction='predict')

The univariate Gaussian observation likelihood assumes that \(y\sim\mathcal{N}(y\mid x; \mu,\sigma)\). The density of the Gaussian likelihood in the univariate case is $$ p(y\mid x) = \frac{1}{\sigma(x)\sqrt{2\pi}}\exp\left(-\frac{1}{2}\left(\frac{y-\mu(x)}{\sigma(x)}\right)^2\right). $$ The support of the univariate Gaussian is \(y\in\mathbb{R}\). Since this is the univariate case, the shape Y when calling DKR.fit(X,Y) needs to be (n,1). For the multivariate case, see the multivariate Gaussian Liklihood.

Parameters:

Name	Type	Description	Default
mu_correction	str	Whether or not a correction of the expected mean should be carried out. When fitting a Gaussian distribution to data, the mean \(\mu\) is calculated prior to the calculation of the standard deviation \(\sigma\). Both \(\mu\) and \(\sigma\) are functions of \(y\). In areas when \(\mu\) rapidly changes, the estimated standard deviation \(\sigma\) becomes inflated as the estimation for the standard deviation assume constant standard deviation across all \(x\). Using mu_prediction, uses information about the already calculated \(\mu\). Specifically, using mu_correction, \(\tilde{y}=y-(\mu(x)-\mu(x_q))\) will be used instead of \(y\). mu_prediction can be chosen from three options: always, predict, or never. When always is selected, the correction will be carried out at every optimization step which is costly. predict doesn't run the correction when DKR.fit is called, but runs it when DKR.predict is called. This option represents a suitable balance between runtime and and model fidelity. Lastly, \never, does not use the correction step during fitting or prediction.	'predict'

Attributes:

Name	Type	Description
param_names	list	A list of strings that contains the names of the model parameters. For the univariate Gaussian likelihood, this is 'mu' and 'sigma'.
mu_correction	str	Either 'always', 'predict', or 'never'. See above for a functional description of mu_correction.

Examples:

import torch
from dkregression.kernels import RBF
from dkregression.likelihoods import UnivariateGaussianLikelihood
from dkregression.cross_validation import CrossValidation
from dkregression import DKR

X = torch.rand((100,2))
Y = torch.rand((100,1))

kernel = RBF(X)
likelihood = UnivariateGaussianLikelihood()
cv = CrossValidation()

model = DKR(kernel, likelihood, cv)
model.fit(X,Y)

Source code in src/dkregression/likelihoods/univariate_gaussian.py

def __init__(self,mu_correction="predict") -> None:
    self.param_names = ["mu", "sigma"]
    self.mu_correction = mu_correction

dkregression.likelihoods.PoissonLikelihood()

The Poisson observation likelihood assumes that \(y\sim\mathrm{Pois}(y\mid x; \lambda)\). The support of the Poisson distriubtion is only the non-negative integers \(y\in\mathbb{N}_0\). Therefore, the method DKR.fit(X,Y) expects Y to be of the shape (n,1) when configured with the PoissonLikelihood and all values in Y to be non-negative integers. The probability mass is given by $$ p(y\mid x) = \frac{\lambda^y\exp (-\lambda)}{y!}. $$

Attributes:

Name	Type	Description
param_names	list	A list of strings that contains the names of the model parameters. For the Poisson likelihood, the list contains 'lambda' as entry. This list is static and corresponds to keys of the dictionary returned by the DKR.predict method.

Examples:

import torch
from dkregression.kernels import RBF
from dkregression.likelihoods import PoissonLikelihood
from dkregression.cross_validation import CrossValidation
from dkregression import DKR

X = torch.rand((100,2))
# the support of the Poisson distribution is only non-negative integers
Y = torch.randint(0,20,(100,1))

kernel = RBF(X)
likelihood = PoissonLikelihood()
cv = CrossValidation()

model = DKR(kernel, likelihood, cv)
model.fit(X,Y)

Source code in src/dkregression/likelihoods/poisson.py

def __init__(self) -> None:
    self.param_names = ["lambda"]

dkregression.likelihoods.BernoulliLikelihood()

The Poisson observation likelihood assumes that \(y\sim\mathrm{Ber}(y\mid x; p)\). The support of the Bernoulli distriubtion is only the binary set, so \(y\in\{0,1\}\). Therefore, the method DKR.fit(X,Y) expects Y to be of the shape (n,1) when configured with the BernoulliLikelihood and only contain the values \(0\) and \(1\). The probability mass is given by $$ p(y\mid x) = \begin{cases}p(x) & \mathrm{for}~y=1 \\ 1-p(x) & \mathrm{for}~y=0~. \end{cases} $$

Attributes:

Name	Type	Description
param_names	list	A list of strings that contains the names of the model parameters. For the Bernoulli likelihood, the list contains 'p' as entry. This list is static and corresponds to keys of the dictionary returned by the DKR.predict method.

Examples:

import torch
from dkregression.kernels import RBF
from dkregression.likelihoods import BernoulliLikelihood
from dkregression.cross_validation import CrossValidation
from dkregression import DKR

X = torch.rand((100,2))
# the support of the Bernoulli distribution is the set of {0,1}
Y = torch.randint(0,2,(100,1))

kernel = RBF(X)
likelihood = BernoulliLikelihood()
cv = CrossValidation()

model = DKR(kernel, likelihood, cv)
model.fit(X,Y)

Source code in src/dkregression/likelihoods/bernoulli.py

def __init__(self) -> None:
    self.param_names = ["p"]

Multivariate Observation Likelihoods

Multivariate Gaussian Likelihood

Custom Observation Likelihoods

Here we describe how to write a custom observation likelihoods:

class CustomLikelihood():
    def __init__(self) --> None:
        pass
    def hello():
        print("Hello")