Pooling

Computes a new matrix by selecting the maximum (max pooling) or average value in the pooling input. There is a simplified syntax for 2D pooling and more advanced for ND pooling. 2D pooling syntax is:

MaxPooling(m, windowWidth, windowHeight, stepW, stepH, imageLayout="HWC"|cudnn")
AveragePooling(m, windowWidth, windowHeight, stepW, stepH, imageLayout="HWC"|cudnn")

Where:

• m - the input matrix.
• windowWidth - width of the pooling window
• windowHeight - height of the pooling window
• stepW - step (or stride) used in the width direction
• stepH - step (or stride) used in the height direction
• imageLayout - [named optional] the storage format of each image. By default it’s HWC, which means each image is stored as [channel, width, height] in column major notation. For better performance, it is recommended to use cuDNN in which case you should set it to cudnn, which means each image is stored as [width, height, channel] in column major notation. Note that cudnn format works both on GPU and CPU.

Example (ConvReLULayer NDL macro):

    # pool2
    pool2W = 2
    pool2H = 2
    pool2hStride = 2
    pool2vStride = 2
    pool2 = MaxPooling(conv2, pool2W, pool2H, pool2hStride, pool2vStride, imageLayout=$imageLayout$)

ND Pooling

ND pooling allows to create max or average pooling of any dimensions, stride or padding. The syntax is:

Pooling(input,
        poolKind,
        {kernel dimensions}, 
        stride = {stride dimensions}, 
        autoPadding = {padding (boolean)},
        lowerPad = {lower padding (int)},
        upperPad = {upper padding (int)},
        imageLayout = "cudnn")

Where:

• input - pooling input
• {kernel dimensions} - dimensions of the pooling window
• stride - [named, optional, default is 1] stride dimensions
• autoPadding - [named, optional, default is true] automatic padding flags for each input dimension
• lowerPad - [named, optional, default is 0] precise lower padding for each input dimension
• upperPad - [named, optional, default is 0] precise upper padding for each input dimension
• imageLayout - [named optional] the storage format of each image. The only supported value is cudnn, which means each image is stored as [width, height, channel] in column major notation.

Since pooling window can have arbitrary dimensions, this allows to build various pooling configurations, for example, maxout layer (see Goodfellow et al for details):

MaxOutPool(inp, kW, kH, kC, hStride, vStride) = [
        p = Pooling(inp, "max", {kW, kH, kC}, stride={hStride, vStride, kC}, autoPadding={true, true, false}, lowerPad=0, upperPad=0, imageLayout=$imageLayout$)
]