Shallow convolutional notebook #16

awav · 2021-04-09T10:33:31Z

No description provided.

vdutor

LGTM. Some of the text could do with an extra proofread though. I already modified some of the text - feel free to adjust to your liking. Also, can you please update the text to use triple quotes """ rather than # to be consistent with the other notebooks.

# %% [markdown]
"""
# Title
Lorem ipsum
"""

vdutor · 2021-04-22T08:12:07Z

docs/notebooks/convolution.py

@@ -0,0 +1,178 @@
+# %% [markdown]
+# In this notebook we revisit the Convolutional Gaussian Processes (ConvGP), <cite data-cite="van2017convolutional"/>. Similarly to convolutional neural networks, ConvGP suits very well to model image processing tasks. As well as CNN, the ConvGP is endowed with translation invariant property. ConvGP imposes stronger and structured prior on a image response function $f(\cdot)$, using patch response function $g(\cdot) \sim GP(0, k_g(\cdot, \cdot))$. The image response function is a sum of patch responses for all (overlapping) patches in the image $f(\mathbb{x}) = \sum_{p=1}^{P}g(\mathbb{x}^{[p]})$, where $p$ is an index of a patch of image, and therefore $f(\cdot) \sim GP(0, \sum_p \sum_{p'} k_g(x^{[p]}, x^{[p']}))$. In a way, the patch response kernel can be viewed as an equivalent to a convolutional kernel of CNN.


Suggested change

# In this notebook we revisit the Convolutional Gaussian Processes (ConvGP), <cite data-cite="van2017convolutional"/>. Similarly to convolutional neural networks, ConvGP suits very well to model image processing tasks. As well as CNN, the ConvGP is endowed with translation invariant property. ConvGP imposes stronger and structured prior on a image response function $f(\cdot)$, using patch response function $g(\cdot) \sim GP(0, k_g(\cdot, \cdot))$. The image response function is a sum of patch responses for all (overlapping) patches in the image $f(\mathbb{x}) = \sum_{p=1}^{P}g(\mathbb{x}^{[p]})$, where $p$ is an index of a patch of image, and therefore $f(\cdot) \sim GP(0, \sum_p \sum_{p'} k_g(x^{[p]}, x^{[p']}))$. In a way, the patch response kernel can be viewed as an equivalent to a convolutional kernel of CNN.

# In this notebook, we revisit single-layer Convolutional Gaussian Processes (ConvGPs), <cite data-cite="van2017convolutional"/>. Similar to Convolutional Neural Networks (CNNs), ConvGPs are well suited for image processing tasks.

A ConvGP imposes a structured prior on an image response function. That is, the image response function $f(\cdot)$ is a sum of patch responses for all (overlapping) patches in the image:

$$

f(\mathbb{x}) = \sum_{p=1}^{P} g(\mathbb{x}^{[p]}),

$$

where $\mathbb{x}$ is an $H \times W$-sized image and $p$ indexes a $h \times w$-sized patch in the image. Imposing a GP on the patch response function $g(\cdot) \sim GP(0, k_g(\cdot, \cdot))$, leads to another GP for the image response function

$$

f(\cdot) \sim GP(0, \sum_p \sum_{p'} k_g(x^{[p]}, x^{[p']})),

$$

because Gaussianity is preserved under linear transformations.

vdutor · 2021-04-22T08:27:09Z

docs/notebooks/convolution.py

+# %% [markdown]
+# In this notebook we revisit the Convolutional Gaussian Processes (ConvGP), <cite data-cite="van2017convolutional"/>. Similarly to convolutional neural networks, ConvGP suits very well to model image processing tasks. As well as CNN, the ConvGP is endowed with translation invariant property. ConvGP imposes stronger and structured prior on a image response function $f(\cdot)$, using patch response function $g(\cdot) \sim GP(0, k_g(\cdot, \cdot))$. The image response function is a sum of patch responses for all (overlapping) patches in the image $f(\mathbb{x}) = \sum_{p=1}^{P}g(\mathbb{x}^{[p]})$, where $p$ is an index of a patch of image, and therefore $f(\cdot) \sim GP(0, \sum_p \sum_{p'} k_g(x^{[p]}, x^{[p']}))$. In a way, the patch response kernel can be viewed as an equivalent to a convolutional kernel of CNN.
+#
+# <img src="./convgp.png" alt="convgp" width="400px"/>


Given that this is a shallow model I think this part of the architecture is more accurate to display. However, we'll have to recompile it without the incoming arrows.

vdutor · 2021-04-22T08:31:50Z

docs/notebooks/convolution.py

+likelihood_layer = LikelihoodLayer(likelihood)
+
+# %% [markdown]
+# Below are are going to use Keras for training convolutional GP model. The details of GPflux and Keras itegration you can find [here](HOW TO ADD A CROSS REFERENCE TO A FILE?).


Suggested change

# Below are are going to use Keras for training convolutional GP model. The details of GPflux and Keras itegration you can find [here](HOW TO ADD A CROSS REFERENCE TO A FILE?).

# Below we are going to use a Keras model built using GPflux for training the convolutional GP model. More details on GPflux with Keras support can be found in [this](gpflux_features.ipynb) notebook.

vdutor · 2021-04-22T08:36:36Z

docs/notebooks/convolution.py

+
+
+#%% [markdown]
+# In this notebook we showed how to use GPflux to build and train shallow convolutional Gaussian processes using Keras. More advanced models like deep convolutional GPs require multi-output convoluitonal kernels and extensions in dispatchers of :mod:`~gpflow.conditionals` and in :mod:`~gpflow.covariances`.


vdutor · 2021-04-22T08:37:22Z

docs/refs.bib

+  author={van der Wilk, Mark and Rasmussen, Carl Edward and Hensman, James},
+  booktitle={NIPS},
+  year={2017}
+}


vdutor · 2021-04-22T08:37:57Z

docs/refs.bib

+@inproceedings{van2017convolutional,
+  title={Convolutional Gaussian Processes},
+  author={van der Wilk, Mark and Rasmussen, Carl Edward and Hensman, James},
+  booktitle={NIPS},


Suggested change

booktitle={NIPS},

booktitle={Advances in Neural Information Processing Systems},

For consistency with other references.

vdutor · 2021-04-22T08:39:52Z

docs/notebooks/convolution.py

+
+# %% [markdown]
+# In the following steps we create a GP convolutional layer and a Bernoulli likelihood layer:
+# 1. Create a convolutional kernel using :mod:`~gpflow.kernels.Convolutional`, and specify input's image shape and a patch shape.


not sure I understand this sentence?

Shallow convolutional notebook

d551a14

vdutor reviewed Apr 22, 2021

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Shallow convolutional notebook #16

Shallow convolutional notebook #16

awav commented Apr 9, 2021

vdutor left a comment

vdutor Apr 22, 2021

vdutor Apr 22, 2021

vdutor Apr 22, 2021

vdutor Apr 22, 2021

vdutor Apr 22, 2021

vdutor Apr 22, 2021

vdutor Apr 22, 2021

vdutor Apr 22, 2021

		@@ -0,0 +1,178 @@
		# %% [markdown]
		# In this notebook we revisit the Convolutional Gaussian Processes (ConvGP), <cite data-cite="van2017convolutional"/>. Similarly to convolutional neural networks, ConvGP suits very well to model image processing tasks. As well as CNN, the ConvGP is endowed with translation invariant property. ConvGP imposes stronger and structured prior on a image response function $f(\cdot)$, using patch response function $g(\cdot) \sim GP(0, k_g(\cdot, \cdot))$. The image response function is a sum of patch responses for all (overlapping) patches in the image $f(\mathbb{x}) = \sum_{p=1}^{P}g(\mathbb{x}^{[p]})$, where $p$ is an index of a patch of image, and therefore $f(\cdot) \sim GP(0, \sum_p \sum_{p'} k_g(x^{[p]}, x^{[p']}))$. In a way, the patch response kernel can be viewed as an equivalent to a convolutional kernel of CNN.

-# In this notebook we revisit the Convolutional Gaussian Processes (ConvGP), <cite data-cite="van2017convolutional"/>. Similarly to convolutional neural networks, ConvGP suits very well to model image processing tasks. As well as CNN, the ConvGP is endowed with translation invariant property. ConvGP imposes stronger and structured prior on a image response function $f(\cdot)$, using patch response function $g(\cdot) \sim GP(0, k_g(\cdot, \cdot))$. The image response function is a sum of patch responses for all (overlapping) patches in the image $f(\mathbb{x}) = \sum_{p=1}^{P}g(\mathbb{x}^{[p]})$, where $p$ is an index of a patch of image, and therefore $f(\cdot) \sim GP(0, \sum_p \sum_{p'} k_g(x^{[p]}, x^{[p']}))$. In a way, the patch response kernel can be viewed as an equivalent to a convolutional kernel of CNN.
+# In this notebook, we revisit single-layer Convolutional Gaussian Processes (ConvGPs), <cite data-cite="van2017convolutional"/>. Similar to Convolutional Neural Networks (CNNs), ConvGPs are well suited for image processing tasks.
+A ConvGP imposes a structured prior on an image response function. That is, the image response function  $f(\cdot)$ is a sum of patch responses for all (overlapping) patches in the image:
+$$
+f(\mathbb{x}) = \sum_{p=1}^{P} g(\mathbb{x}^{[p]}),
+$$
+where $\mathbb{x}$ is an $H \times W$-sized image and $p$ indexes a $h \times w$-sized patch in the image. Imposing a GP on the patch response function $g(\cdot) \sim GP(0, k_g(\cdot, \cdot))$, leads to another GP for the image response function
+$$
+f(\cdot) \sim GP(0, \sum_p \sum_{p'} k_g(x^{[p]}, x^{[p']})),
+$$
+because Gaussianity is preserved under linear transformations.

	# Below are are going to use Keras for training convolutional GP model. The details of GPflux and Keras itegration you can find [here](HOW TO ADD A CROSS REFERENCE TO A FILE?).
	# Below we are going to use a Keras model built using GPflux for training the convolutional GP model. More details on GPflux with Keras support can be found in [this](gpflux_features.ipynb) notebook.



		#%% [markdown]
		# In this notebook we showed how to use GPflux to build and train shallow convolutional Gaussian processes using Keras. More advanced models like deep convolutional GPs require multi-output convoluitonal kernels and extensions in dispatchers of :mod:`~gpflow.conditionals` and in :mod:`~gpflow.covariances`.

	booktitle={NIPS},
	booktitle={Advances in Neural Information Processing Systems},

Shallow convolutional notebook #16

Are you sure you want to change the base?

Shallow convolutional notebook #16

Conversation

awav commented Apr 9, 2021

vdutor left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment