We are currently solving the following issue within our repository. Here is the issue text:
--- BEGIN ISSUE ---
[Bug] Wrong formulation of the PeriodicKernel
# 🐛 Bug

The current formulation of the periodic kernel is the following:

<img src="https://render.githubusercontent.com/render/math?math=k(\mathbf{x^1, x^2}) = \exp ( \frac{-2 \sin^2(\frac{\pi}{p} \sum_i | x^{1}_{i} - x^{2}_{i} |)}{l^2} )">

However, the original formula proposed [Mackay, 1998](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.81.1927&rep=rep1&type=pdf) (Eq. 47) is the following:

<img src="https://render.githubusercontent.com/render/math?math=k(\mathbf{x^1, x^2}) = \exp ( \frac{-0.5  \sum_i \sin^2(\frac{\pi}{p} | x^{1}_{i} - x^{2}_{i} |)}{l^2} )">

The constant is not -2 or -0.5 is not really important (it will have an impact on the derivative only). However, the original formulation is taking the the sum of the sine squared of the difference. Currently, the formulation is the sine squared of the sum of the difference. Note that the confusion could come from the fact that in the literature, the kernel is used for 1D signal and thus there is no sum, thus both formulations are equivalent.

In addition, looking at the implementation, it seems that the distance computed between the samples is norm-2 distance and not the norm-1 distance.

The current formulation make it that the kernel is not PSD. In addition, there is still another bug with a missing square over the length scale as reported in https://github.com/cornellius-gp/gpytorch/issues/1020.

Note: I found this issue because I am currently working on a similar issues in scikit-learn:
https://github.com/scikit-learn/scikit-learn/issues/19343
https://github.com/scikit-learn/scikit-learn/pull/20070

## To reproduce

** Code snippet to reproduce **
```python
import numpy as np
import torch
from gpytorch.kernels import PeriodicKernel

r = np.linspace(0, 1, num=4)
train_x, train_y = np.meshgrid(r, r)
X = np.stack((train_x.flatten(), train_y.flatten()), axis=-1)

periodicity, length_scale=1.5, 0.5
kernel_pytorch = PeriodicKernel(
    period=periodicity, length_scale=length_scale
)
K_pytorch = kernel_pytorch(torch.from_numpy(X))

np.linalg.eigh(K_pytorch.numpy())[0]
```

** Stack trace/error message **
```
array([-1.27815495, -1.21435487, -1.21435487, -0.93566031, -0.75456704,
       -0.24301193, -0.24301193,  0.47435512,  0.72370254,  1.58773803,
        1.58773803,  2.42356711,  2.67356606,  2.67356606,  3.10565507,
        6.63322787])
```

There is some negative eigenvalues.

## Expected Behavior

We are investigating the issue in scikit-learn and already implemented a naive approach: https://github.com/scikit-learn/scikit-learn/pull/20070 for tracking the bug. With the code of the PR, there is no negative eigenvalues:

```python
from sklearn.gaussian_process.kernels import ExpSineSquared

kernel_sklearn = ExpSineSquared(
    periodicity=periodicity, length_scale=length_scale,
)
K_sklearn = kernel_sklearn(X)
np.linalg.eigh(K_sklearn)[0]
```

```
array([0.88422032, 0.92031869, 0.92031869, 0.95789078, 0.95961332,
       0.95961332, 0.99716727, 0.99716727, 0.99878962, 0.99878962,
       1.03787671, 1.03787671, 1.0414347 , 1.08219069, 1.08219069,
       1.12454163])
```

On the process, we also worked with the implementation of tensorflow-probability that follow the equation of the paper, and we have consistent results:

```python
from tensorflow_probability import math

kernel_tfp = math.psd_kernels.ExpSinSquared(
   period=periodicity, length_scale=length_scale
)
K_tfp = kernel_tfp.matrix(X, X)
np.linalg.eigh(K_tfp.numpy())[0]
``` 

```
array([0.8842203 , 0.92031866, 0.92031866, 0.95789075, 0.9596133 ,
       0.9596133 , 0.9971673 , 0.9971673 , 0.9987896 , 0.9987896 ,
       1.0378767 , 1.0378767 , 1.0414346 , 1.0821906 , 1.0821906 ,
       1.1245416 ], dtype=float32)
```

## System information

**Please complete the following information:**
- GPyTorch version: `1.4.1`
- PyTorch version: `1.8.1+cpu`
- Linux 5.8.0-50-generic #56-Ubuntu SMP Mon Apr 12 17:18:36 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux


--- END ISSUE ---

Below are some code segments, each from a relevant file. One or more of these files may contain bugs.
--- BEGIN FILES ---
Path: `gpytorch/kernels/periodic_kernel.py`
Content:
```
1 #!/usr/bin/env python3
2 
3 import math
4 
5 import torch
6 
7 from ..constraints import Positive
8 from .kernel import Kernel
9 
10 
11 class PeriodicKernel(Kernel):
12     r"""Computes a covariance matrix based on the periodic kernel
13     between inputs :math:`\mathbf{x_1}` and :math:`\mathbf{x_2}`:
14 
15     .. math::
16 
17        \begin{equation*}
18           k_{\text{Periodic}}(\mathbf{x_1}, \mathbf{x_2}) = \exp \left(
19             \frac{2 \sin^2 \left( \pi \Vert \mathbf{x_1} - \mathbf{x_2} \Vert_1 / p \right) }
20             { \ell^2 } \right)
21        \end{equation*}
22 
23     where
24 
25     * :math:`p` is the periord length parameter.
26     * :math:`\ell` is a lengthscale parameter.
27 
28     .. note::
29 
30         This kernel does not have an `outputscale` parameter. To add a scaling parameter,
31         decorate this kernel with a :class:`gpytorch.kernels.ScaleKernel`.
32 
33     .. note::
34 
35         This kernel does not have an ARD lengthscale option.
36 
37     Args:
38         :attr:`batch_shape` (torch.Size, optional):
39             Set this if you want a separate lengthscale for each
40              batch of input data. It should be `b` if :attr:`x1` is a `b x n x d` tensor. Default: `torch.Size([])`.
41         :attr:`active_dims` (tuple of ints, optional):
42             Set this if you want to compute the covariance of only a few input dimensions. The ints
43             corresponds to the indices of the dimensions. Default: `None`.
44         :attr:`period_length_prior` (Prior, optional):
45             Set this if you want to apply a prior to the period length parameter.  Default: `None`.
46         :attr:`lengthscale_prior` (Prior, optional):
47             Set this if you want to apply a prior to the lengthscale parameter.  Default: `None`.
48         :attr:`lengthscale_constraint` (Constraint, optional):
49             Set this if you want to apply a constraint to the value of the lengthscale. Default: `Positive`.
50         :attr:`period_length_constraint` (Constraint, optional):
51             Set this if you want to apply a constraint to the value of the period length. Default: `Positive`.
52         :attr:`eps` (float):
53             The minimum value that the lengthscale/period length can take
54             (prevents divide by zero errors). Default: `1e-6`.
55 
56     Attributes:
57         :attr:`lengthscale` (Tensor):
58             The lengthscale parameter. Size = `*batch_shape x 1 x 1`.
59         :attr:`period_length` (Tensor):
60             The period length parameter. Size = `*batch_shape x 1 x 1`.
61 
62     Example:
63         >>> x = torch.randn(10, 5)
64         >>> # Non-batch: Simple option
65         >>> covar_module = gpytorch.kernels.ScaleKernel(gpytorch.kernels.PeriodicKernel())
66         >>>
67         >>> batch_x = torch.randn(2, 10, 5)
68         >>> # Batch: Simple option
69         >>> covar_module = gpytorch.kernels.ScaleKernel(gpytorch.kernels.PeriodicKernel())
70         >>> # Batch: different lengthscale for each batch
71         >>> covar_module = gpytorch.kernels.ScaleKernel(gpytorch.kernels.PeriodicKernel(batch_size=2))
72         >>> covar = covar_module(x)  # Output: LazyVariable of size (2 x 10 x 10)
73     """
74 
75     has_lengthscale = True
76 
77     def __init__(self, period_length_prior=None, period_length_constraint=None, **kwargs):
78         super(PeriodicKernel, self).__init__(**kwargs)
79         if period_length_constraint is None:
80             period_length_constraint = Positive()
81 
82         self.register_parameter(
83             name="raw_period_length", parameter=torch.nn.Parameter(torch.zeros(*self.batch_shape, 1, 1))
84         )
85 
86         if period_length_prior is not None:
87             self.register_prior(
88                 "period_length_prior",
89                 period_length_prior,
90                 lambda m: m.period_length,
91                 lambda m, v: m._set_period_length(v),
92             )
93 
94         self.register_constraint("raw_period_length", period_length_constraint)
95 
96     @property
97     def period_length(self):
98         return self.raw_period_length_constraint.transform(self.raw_period_length)
99 
100     @period_length.setter
101     def period_length(self, value):
102         self._set_period_length(value)
103 
104     def _set_period_length(self, value):
105         if not torch.is_tensor(value):
106             value = torch.as_tensor(value).to(self.raw_period_length)
107         self.initialize(raw_period_length=self.raw_period_length_constraint.inverse_transform(value))
108 
109     def forward(self, x1, x2, diag=False, **params):
110         x1_ = x1.div(self.period_length)
111         x2_ = x2.div(self.period_length)
112         diff = self.covar_dist(x1_, x2_, diag=diag, **params)
113         res = torch.sin(diff.mul(math.pi)).pow(2).mul(-2 / self.lengthscale).exp_()
114         if diag:
115             res = res.squeeze(0)
116         return res
117
```
--- END FILES ---

Please first localize the bug based on the issue statement, and then generate a patch according to the `git diff` format fenced by three backticks.

Here is an example:

```diff
diff --git a/examples/server_async.py b/examples/server_async.py
--- a/examples/server_async.py
+++ b/examples/server_async.py
@@ -313,4 +313,4 @@
 
 
 if __name__ == "__main__":
-    asyncio.run(run_async_server("."), debug=True)
+    asyncio.run(run_async_server(), debug=True)
diff --git a/examples/server_sync.py b/examples/server_sync.py
--- a/examples/server_sync.py
+++ b/examples/server_sync.py
@@ -313,5 +313,5 @@
 
 
 if __name__ == "__main__":
-    server = run_sync_server(".")
+    server = run_sync_server()
     server.shutdown()

```

diff --git a/gpytorch/kernels/periodic_kernel.py b/gpytorch/kernels/periodic_kernel.py
--- a/gpytorch/kernels/periodic_kernel.py
+++ b/gpytorch/kernels/periodic_kernel.py
@@ -14,16 +14,22 @@
 
     .. math::
 
-       \begin{equation*}
-          k_{\text{Periodic}}(\mathbf{x_1}, \mathbf{x_2}) = \exp \left(
-            \frac{2 \sin^2 \left( \pi \Vert \mathbf{x_1} - \mathbf{x_2} \Vert_1 / p \right) }
-            { \ell^2 } \right)
-       \end{equation*}
+        \begin{equation*}
+            k_{\text{Periodic}}(\mathbf{x_1}, \mathbf{x_2}) = \exp \left(
+            -2 \sum_i
+            \frac{\sin ^2 \left( \frac{\pi}{p} (\mathbf{x_{1,i}} - \mathbf{x_{2,i}} ) \right)}{\lambda}
+            \right)
+        \end{equation*}
 
     where
 
-    * :math:`p` is the periord length parameter.
-    * :math:`\ell` is a lengthscale parameter.
+    * :math:`p` is the period length parameter.
+    * :math:`\lambda` is a lengthscale parameter.
+
+    Equation is based on [David Mackay's Introduction to Gaussian Processes equation 47]
+    (http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.81.1927&rep=rep1&type=pdf)
+    albeit without feature-specific lengthscales and period lengths. The exponential
+    coefficient was changed and lengthscale is not squared to maintain backwards compatibility
 
     .. note::
 
@@ -32,7 +38,7 @@
 
     .. note::
 
-        This kernel does not have an ARD lengthscale option.
+        This kernel does not have an ARD lengthscale or period length option.
 
     Args:
         :attr:`batch_shape` (torch.Size, optional):
@@ -107,10 +113,10 @@
         self.initialize(raw_period_length=self.raw_period_length_constraint.inverse_transform(value))
 
     def forward(self, x1, x2, diag=False, **params):
-        x1_ = x1.div(self.period_length)
-        x2_ = x2.div(self.period_length)
-        diff = self.covar_dist(x1_, x2_, diag=diag, **params)
-        res = torch.sin(diff.mul(math.pi)).pow(2).mul(-2 / self.lengthscale).exp_()
+        x1_ = x1.div(self.period_length).mul(math.pi)
+        x2_ = x2.div(self.period_length).mul(math.pi)
+        diff = x1_.unsqueeze(-2) - x2_.unsqueeze(-3)
+        res = diff.sin().pow(2).sum(dim=-1).div(self.lengthscale).mul(-2.0).exp_()
         if diag:
             res = res.squeeze(0)
         return res

{"golden_diff": "diff --git a/gpytorch/kernels/periodic_kernel.py b/gpytorch/kernels/periodic_kernel.py\n--- a/gpytorch/kernels/periodic_kernel.py\n+++ b/gpytorch/kernels/periodic_kernel.py\n@@ -14,16 +14,22 @@\n \n     .. math::\n \n-       \\begin{equation*}\n-          k_{\\text{Periodic}}(\\mathbf{x_1}, \\mathbf{x_2}) = \\exp \\left(\n-            \\frac{2 \\sin^2 \\left( \\pi \\Vert \\mathbf{x_1} - \\mathbf{x_2} \\Vert_1 / p \\right) }\n-            { \\ell^2 } \\right)\n-       \\end{equation*}\n+        \\begin{equation*}\n+            k_{\\text{Periodic}}(\\mathbf{x_1}, \\mathbf{x_2}) = \\exp \\left(\n+            -2 \\sum_i\n+            \\frac{\\sin ^2 \\left( \\frac{\\pi}{p} (\\mathbf{x_{1,i}} - \\mathbf{x_{2,i}} ) \\right)}{\\lambda}\n+            \\right)\n+        \\end{equation*}\n \n     where\n \n-    * :math:`p` is the periord length parameter.\n-    * :math:`\\ell` is a lengthscale parameter.\n+    * :math:`p` is the period length parameter.\n+    * :math:`\\lambda` is a lengthscale parameter.\n+\n+    Equation is based on [David Mackay's Introduction to Gaussian Processes equation 47]\n+    (http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.81.1927&rep=rep1&type=pdf)\n+    albeit without feature-specific lengthscales and period lengths. The exponential\n+    coefficient was changed and lengthscale is not squared to maintain backwards compatibility\n \n     .. note::\n \n@@ -32,7 +38,7 @@\n \n     .. note::\n \n-        This kernel does not have an ARD lengthscale option.\n+        This kernel does not have an ARD lengthscale or period length option.\n \n     Args:\n         :attr:`batch_shape` (torch.Size, optional):\n@@ -107,10 +113,10 @@\n         self.initialize(raw_period_length=self.raw_period_length_constraint.inverse_transform(value))\n \n     def forward(self, x1, x2, diag=False, **params):\n-        x1_ = x1.div(self.period_length)\n-        x2_ = x2.div(self.period_length)\n-        diff = self.covar_dist(x1_, x2_, diag=diag, **params)\n-        res = torch.sin(diff.mul(math.pi)).pow(2).mul(-2 / self.lengthscale).exp_()\n+        x1_ = x1.div(self.period_length).mul(math.pi)\n+        x2_ = x2.div(self.period_length).mul(math.pi)\n+        diff = x1_.unsqueeze(-2) - x2_.unsqueeze(-3)\n+        res = diff.sin().pow(2).sum(dim=-1).div(self.lengthscale).mul(-2.0).exp_()\n         if diag:\n             res = res.squeeze(0)\n         return res\n", "issue": "[Bug] Wrong formulation of the PeriodicKernel\n# \ud83d\udc1b Bug\r\n\r\nThe current formulation of the periodic kernel is the following:\r\n\r\n<img src=\"https://render.githubusercontent.com/render/math?math=k(\\mathbf{x^1, x^2}) = \\exp ( \\frac{-2 \\sin^2(\\frac{\\pi}{p} \\sum_i | x^{1}_{i} - x^{2}_{i} |)}{l^2} )\">\r\n\r\nHowever, the original formula proposed [Mackay, 1998](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.81.1927&rep=rep1&type=pdf) (Eq. 47) is the following:\r\n\r\n<img src=\"https://render.githubusercontent.com/render/math?math=k(\\mathbf{x^1, x^2}) = \\exp ( \\frac{-0.5  \\sum_i \\sin^2(\\frac{\\pi}{p} | x^{1}_{i} - x^{2}_{i} |)}{l^2} )\">\r\n\r\nThe constant is not -2 or -0.5 is not really important (it will have an impact on the derivative only). However, the original formulation is taking the the sum of the sine squared of the difference. Currently, the formulation is the sine squared of the sum of the difference. Note that the confusion could come from the fact that in the literature, the kernel is used for 1D signal and thus there is no sum, thus both formulations are equivalent.\r\n\r\nIn addition, looking at the implementation, it seems that the distance computed between the samples is norm-2 distance and not the norm-1 distance.\r\n\r\nThe current formulation make it that the kernel is not PSD. In addition, there is still another bug with a missing square over the length scale as reported in https://github.com/cornellius-gp/gpytorch/issues/1020.\r\n\r\nNote: I found this issue because I am currently working on a similar issues in scikit-learn:\r\nhttps://github.com/scikit-learn/scikit-learn/issues/19343\r\nhttps://github.com/scikit-learn/scikit-learn/pull/20070\r\n\r\n## To reproduce\r\n\r\n** Code snippet to reproduce **\r\n```python\r\nimport numpy as np\r\nimport torch\r\nfrom gpytorch.kernels import PeriodicKernel\r\n\r\nr = np.linspace(0, 1, num=4)\r\ntrain_x, train_y = np.meshgrid(r, r)\r\nX = np.stack((train_x.flatten(), train_y.flatten()), axis=-1)\r\n\r\nperiodicity, length_scale=1.5, 0.5\r\nkernel_pytorch = PeriodicKernel(\r\n    period=periodicity, length_scale=length_scale\r\n)\r\nK_pytorch = kernel_pytorch(torch.from_numpy(X))\r\n\r\nnp.linalg.eigh(K_pytorch.numpy())[0]\r\n```\r\n\r\n** Stack trace/error message **\r\n```\r\narray([-1.27815495, -1.21435487, -1.21435487, -0.93566031, -0.75456704,\r\n       -0.24301193, -0.24301193,  0.47435512,  0.72370254,  1.58773803,\r\n        1.58773803,  2.42356711,  2.67356606,  2.67356606,  3.10565507,\r\n        6.63322787])\r\n```\r\n\r\nThere is some negative eigenvalues.\r\n\r\n## Expected Behavior\r\n\r\nWe are investigating the issue in scikit-learn and already implemented a naive approach: https://github.com/scikit-learn/scikit-learn/pull/20070 for tracking the bug. With the code of the PR, there is no negative eigenvalues:\r\n\r\n```python\r\nfrom sklearn.gaussian_process.kernels import ExpSineSquared\r\n\r\nkernel_sklearn = ExpSineSquared(\r\n    periodicity=periodicity, length_scale=length_scale,\r\n)\r\nK_sklearn = kernel_sklearn(X)\r\nnp.linalg.eigh(K_sklearn)[0]\r\n```\r\n\r\n```\r\narray([0.88422032, 0.92031869, 0.92031869, 0.95789078, 0.95961332,\r\n       0.95961332, 0.99716727, 0.99716727, 0.99878962, 0.99878962,\r\n       1.03787671, 1.03787671, 1.0414347 , 1.08219069, 1.08219069,\r\n       1.12454163])\r\n```\r\n\r\nOn the process, we also worked with the implementation of tensorflow-probability that follow the equation of the paper, and we have consistent results:\r\n\r\n```python\r\nfrom tensorflow_probability import math\r\n\r\nkernel_tfp = math.psd_kernels.ExpSinSquared(\r\n   period=periodicity, length_scale=length_scale\r\n)\r\nK_tfp = kernel_tfp.matrix(X, X)\r\nnp.linalg.eigh(K_tfp.numpy())[0]\r\n``` \r\n\r\n```\r\narray([0.8842203 , 0.92031866, 0.92031866, 0.95789075, 0.9596133 ,\r\n       0.9596133 , 0.9971673 , 0.9971673 , 0.9987896 , 0.9987896 ,\r\n       1.0378767 , 1.0378767 , 1.0414346 , 1.0821906 , 1.0821906 ,\r\n       1.1245416 ], dtype=float32)\r\n```\r\n\r\n## System information\r\n\r\n**Please complete the following information:**\r\n- GPyTorch version: `1.4.1`\r\n- PyTorch version: `1.8.1+cpu`\r\n- Linux 5.8.0-50-generic #56-Ubuntu SMP Mon Apr 12 17:18:36 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux\r\n\n", "before_files": [{"content": "#!/usr/bin/env python3\n\nimport math\n\nimport torch\n\nfrom ..constraints import Positive\nfrom .kernel import Kernel\n\n\nclass PeriodicKernel(Kernel):\n    r\"\"\"Computes a covariance matrix based on the periodic kernel\n    between inputs :math:`\\mathbf{x_1}` and :math:`\\mathbf{x_2}`:\n\n    .. math::\n\n       \\begin{equation*}\n          k_{\\text{Periodic}}(\\mathbf{x_1}, \\mathbf{x_2}) = \\exp \\left(\n            \\frac{2 \\sin^2 \\left( \\pi \\Vert \\mathbf{x_1} - \\mathbf{x_2} \\Vert_1 / p \\right) }\n            { \\ell^2 } \\right)\n       \\end{equation*}\n\n    where\n\n    * :math:`p` is the periord length parameter.\n    * :math:`\\ell` is a lengthscale parameter.\n\n    .. note::\n\n        This kernel does not have an `outputscale` parameter. To add a scaling parameter,\n        decorate this kernel with a :class:`gpytorch.kernels.ScaleKernel`.\n\n    .. note::\n\n        This kernel does not have an ARD lengthscale option.\n\n    Args:\n        :attr:`batch_shape` (torch.Size, optional):\n            Set this if you want a separate lengthscale for each\n             batch of input data. It should be `b` if :attr:`x1` is a `b x n x d` tensor. Default: `torch.Size([])`.\n        :attr:`active_dims` (tuple of ints, optional):\n            Set this if you want to compute the covariance of only a few input dimensions. The ints\n            corresponds to the indices of the dimensions. Default: `None`.\n        :attr:`period_length_prior` (Prior, optional):\n            Set this if you want to apply a prior to the period length parameter.  Default: `None`.\n        :attr:`lengthscale_prior` (Prior, optional):\n            Set this if you want to apply a prior to the lengthscale parameter.  Default: `None`.\n        :attr:`lengthscale_constraint` (Constraint, optional):\n            Set this if you want to apply a constraint to the value of the lengthscale. Default: `Positive`.\n        :attr:`period_length_constraint` (Constraint, optional):\n            Set this if you want to apply a constraint to the value of the period length. Default: `Positive`.\n        :attr:`eps` (float):\n            The minimum value that the lengthscale/period length can take\n            (prevents divide by zero errors). Default: `1e-6`.\n\n    Attributes:\n        :attr:`lengthscale` (Tensor):\n            The lengthscale parameter. Size = `*batch_shape x 1 x 1`.\n        :attr:`period_length` (Tensor):\n            The period length parameter. Size = `*batch_shape x 1 x 1`.\n\n    Example:\n        >>> x = torch.randn(10, 5)\n        >>> # Non-batch: Simple option\n        >>> covar_module = gpytorch.kernels.ScaleKernel(gpytorch.kernels.PeriodicKernel())\n        >>>\n        >>> batch_x = torch.randn(2, 10, 5)\n        >>> # Batch: Simple option\n        >>> covar_module = gpytorch.kernels.ScaleKernel(gpytorch.kernels.PeriodicKernel())\n        >>> # Batch: different lengthscale for each batch\n        >>> covar_module = gpytorch.kernels.ScaleKernel(gpytorch.kernels.PeriodicKernel(batch_size=2))\n        >>> covar = covar_module(x)  # Output: LazyVariable of size (2 x 10 x 10)\n    \"\"\"\n\n    has_lengthscale = True\n\n    def __init__(self, period_length_prior=None, period_length_constraint=None, **kwargs):\n        super(PeriodicKernel, self).__init__(**kwargs)\n        if period_length_constraint is None:\n            period_length_constraint = Positive()\n\n        self.register_parameter(\n            name=\"raw_period_length\", parameter=torch.nn.Parameter(torch.zeros(*self.batch_shape, 1, 1))\n        )\n\n        if period_length_prior is not None:\n            self.register_prior(\n                \"period_length_prior\",\n                period_length_prior,\n                lambda m: m.period_length,\n                lambda m, v: m._set_period_length(v),\n            )\n\n        self.register_constraint(\"raw_period_length\", period_length_constraint)\n\n    @property\n    def period_length(self):\n        return self.raw_period_length_constraint.transform(self.raw_period_length)\n\n    @period_length.setter\n    def period_length(self, value):\n        self._set_period_length(value)\n\n    def _set_period_length(self, value):\n        if not torch.is_tensor(value):\n            value = torch.as_tensor(value).to(self.raw_period_length)\n        self.initialize(raw_period_length=self.raw_period_length_constraint.inverse_transform(value))\n\n    def forward(self, x1, x2, diag=False, **params):\n        x1_ = x1.div(self.period_length)\n        x2_ = x2.div(self.period_length)\n        diff = self.covar_dist(x1_, x2_, diag=diag, **params)\n        res = torch.sin(diff.mul(math.pi)).pow(2).mul(-2 / self.lengthscale).exp_()\n        if diag:\n            res = res.squeeze(0)\n        return res\n", "path": "gpytorch/kernels/periodic_kernel.py"}], "after_files": [{"content": "#!/usr/bin/env python3\n\nimport math\n\nimport torch\n\nfrom ..constraints import Positive\nfrom .kernel import Kernel\n\n\nclass PeriodicKernel(Kernel):\n    r\"\"\"Computes a covariance matrix based on the periodic kernel\n    between inputs :math:`\\mathbf{x_1}` and :math:`\\mathbf{x_2}`:\n\n    .. math::\n\n        \\begin{equation*}\n            k_{\\text{Periodic}}(\\mathbf{x_1}, \\mathbf{x_2}) = \\exp \\left(\n            -2 \\sum_i\n            \\frac{\\sin ^2 \\left( \\frac{\\pi}{p} (\\mathbf{x_{1,i}} - \\mathbf{x_{2,i}} ) \\right)}{\\lambda}\n            \\right)\n        \\end{equation*}\n\n    where\n\n    * :math:`p` is the period length parameter.\n    * :math:`\\lambda` is a lengthscale parameter.\n\n    Equation is based on [David Mackay's Introduction to Gaussian Processes equation 47]\n    (http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.81.1927&rep=rep1&type=pdf)\n    albeit without feature-specific lengthscales and period lengths. The exponential\n    coefficient was changed and lengthscale is not squared to maintain backwards compatibility\n\n    .. note::\n\n        This kernel does not have an `outputscale` parameter. To add a scaling parameter,\n        decorate this kernel with a :class:`gpytorch.kernels.ScaleKernel`.\n\n    .. note::\n\n        This kernel does not have an ARD lengthscale or period length option.\n\n    Args:\n        :attr:`batch_shape` (torch.Size, optional):\n            Set this if you want a separate lengthscale for each\n             batch of input data. It should be `b` if :attr:`x1` is a `b x n x d` tensor. Default: `torch.Size([])`.\n        :attr:`active_dims` (tuple of ints, optional):\n            Set this if you want to compute the covariance of only a few input dimensions. The ints\n            corresponds to the indices of the dimensions. Default: `None`.\n        :attr:`period_length_prior` (Prior, optional):\n            Set this if you want to apply a prior to the period length parameter.  Default: `None`.\n        :attr:`lengthscale_prior` (Prior, optional):\n            Set this if you want to apply a prior to the lengthscale parameter.  Default: `None`.\n        :attr:`lengthscale_constraint` (Constraint, optional):\n            Set this if you want to apply a constraint to the value of the lengthscale. Default: `Positive`.\n        :attr:`period_length_constraint` (Constraint, optional):\n            Set this if you want to apply a constraint to the value of the period length. Default: `Positive`.\n        :attr:`eps` (float):\n            The minimum value that the lengthscale/period length can take\n            (prevents divide by zero errors). Default: `1e-6`.\n\n    Attributes:\n        :attr:`lengthscale` (Tensor):\n            The lengthscale parameter. Size = `*batch_shape x 1 x 1`.\n        :attr:`period_length` (Tensor):\n            The period length parameter. Size = `*batch_shape x 1 x 1`.\n\n    Example:\n        >>> x = torch.randn(10, 5)\n        >>> # Non-batch: Simple option\n        >>> covar_module = gpytorch.kernels.ScaleKernel(gpytorch.kernels.PeriodicKernel())\n        >>>\n        >>> batch_x = torch.randn(2, 10, 5)\n        >>> # Batch: Simple option\n        >>> covar_module = gpytorch.kernels.ScaleKernel(gpytorch.kernels.PeriodicKernel())\n        >>> # Batch: different lengthscale for each batch\n        >>> covar_module = gpytorch.kernels.ScaleKernel(gpytorch.kernels.PeriodicKernel(batch_size=2))\n        >>> covar = covar_module(x)  # Output: LazyVariable of size (2 x 10 x 10)\n    \"\"\"\n\n    has_lengthscale = True\n\n    def __init__(self, period_length_prior=None, period_length_constraint=None, **kwargs):\n        super(PeriodicKernel, self).__init__(**kwargs)\n        if period_length_constraint is None:\n            period_length_constraint = Positive()\n\n        self.register_parameter(\n            name=\"raw_period_length\", parameter=torch.nn.Parameter(torch.zeros(*self.batch_shape, 1, 1))\n        )\n\n        if period_length_prior is not None:\n            self.register_prior(\n                \"period_length_prior\",\n                period_length_prior,\n                lambda m: m.period_length,\n                lambda m, v: m._set_period_length(v),\n            )\n\n        self.register_constraint(\"raw_period_length\", period_length_constraint)\n\n    @property\n    def period_length(self):\n        return self.raw_period_length_constraint.transform(self.raw_period_length)\n\n    @period_length.setter\n    def period_length(self, value):\n        self._set_period_length(value)\n\n    def _set_period_length(self, value):\n        if not torch.is_tensor(value):\n            value = torch.as_tensor(value).to(self.raw_period_length)\n        self.initialize(raw_period_length=self.raw_period_length_constraint.inverse_transform(value))\n\n    def forward(self, x1, x2, diag=False, **params):\n        x1_ = x1.div(self.period_length).mul(math.pi)\n        x2_ = x2.div(self.period_length).mul(math.pi)\n        diff = x1_.unsqueeze(-2) - x2_.unsqueeze(-3)\n        res = diff.sin().pow(2).sum(dim=-1).div(self.lengthscale).mul(-2.0).exp_()\n        if diag:\n            res = res.squeeze(0)\n        return res\n", "path": "gpytorch/kernels/periodic_kernel.py"}]}