Causal Dilated Convolution

Build the layer at the core of temporal convolutional networks (TCNs) and WaveNet: a 1D convolution that never looks at the future.

A plain nn.Conv1d with symmetric padding mixes in values from later time steps. For forecasting that is a leak. A causal convolution pads only on the left, by (kernel_size - 1) * dilation, so output t sees inputs up to t and the output keeps the input length.

y[t] = sum over j of w[j] * x[t - (kernel_size - 1 - j) * dilation]   (x is 0 before the start)

Dilation spaces out the taps. Stacking layers with dilations 1, 2, 4, 8 makes the receptive field grow exponentially with depth.

Write:

• CausalConv1d(in_channels, out_channels, kernel_size, dilation=1): an nn.Module that keeps self.conv = nn.Conv1d(in_channels, out_channels, kernel_size, dilation=dilation) with no padding of its own, and pads on the left in forward.
• receptive_field(kernel_size, dilations): how many input steps the last output of a stack of these layers can see.

Function signature

class CausalConv1d(torch.nn.Module):
    def __init__(self, in_channels, out_channels, kernel_size, dilation=1): ...
    def forward(self, x): ...


def receptive_field(kernel_size, dilations):

Example

Input

x = [1, 2, 3, 4]          # one channel, shape (1, 1, 4)
kernel_size = 2, dilation = 1, weights [1, 10], bias 0

Output

[10, 21, 32, 43]

Reasoning

The left pad is (2 - 1) * 1 = 1, so the padded input is [0, 1, 2, 3, 4]. Each output is 1 * previous + 10 * current: 0 + 10, 1 + 20, 2 + 30, 3 + 40. Output 0 uses no future value, and the length stays 4.

Constraints and notes

• x has shape (B, C, T). The output has shape (B, out_channels, T).
• Use F.pad(x, (left, 0)). The tests compare with F.conv1d on a left-padded input, using your self.conv weights.
• The tests check causality with autograd, and they measure the receptive field of a stack of your layers from the gradient.
• receptive_field(3, [1, 2, 4]) is 15.