Network Variants

Network Variants

Deepity ships seven network classes, all importable from deepity directly (from deepity import SimplePCN). Every one is declared the same way: pass Linear/Convolution and activation objects (Sigmoid, TanH, ReLU, GeLU) positionally, then call configure() to set optimization hyperparameters and build the backend.

For a one-line "which do I use" pointer, see the main README. This page goes one level deeper, with the real constructor and configure() signatures.

SequentialPCN

Wraps dy.DiscriminativePCNetwork directly. No feedback-alignment tricks, no settling variant, just the base predictive coding layer.

from deepity import SequentialPCN, Linear, TanH

net = SequentialPCN(
    Linear(2, 8), TanH(),
    Linear(8, 1),
    batch_size=4,
)

SimplePCN

The network behind the README's headline MNIST benchmark. Classic synchronous (Jacobi) settling: every layer's belief updates from the previous step's state, not from values other layers already updated this step.

from deepity import SimplePCN, Linear, TanH

net = SimplePCN(Linear(2, 8), TanH(), Linear(8, 1), batch_size=4, device="cpu")
net.configure(
    learning_rate=0.1,
    inference_rate=0.15,
    lmbda=0.0,
    optimizer="SGD",
    seed=42,
)

configure() also takes dist (weight-init distribution, e.g. "uniform(-0.3,0.3)"). train_step(X, Y, steps) settles for steps rounds then updates weights once, returning the batch's energy. predict(X, steps) runs the same settling loop without a target clamp.

GaussSeidelPCN

Same API as SimplePCN, different settling dynamics: Gauss-Seidel (sequential-sweep) instead of Jacobi. A later layer sees already-updated values from earlier layers within the same settling step, instead of waiting for the next step. Feedback runs through an independently-initialized, never-updated alignment matrix E (Lillicrap et al. feedback alignment), not W transposed.

from deepity import GaussSeidelPCN, Linear, Sigmoid

net = GaussSeidelPCN(Linear(784, 128), Sigmoid(), Linear(128, 10), batch_size=250)
net.configure(learning_rate=0.01, inference_rate=0.1, optimizer="ADAMW")

DKPPCN

Direct Kolen-Pollack feedback: a separate set of feedback weights Psi, learned alongside the forward weights W, instead of reusing W's transpose to send predictions back down. This is why it needs far fewer settling steps (the README's benchmark uses 2, versus SimplePCN's 20) at a real accuracy cost.

from deepity import DKPPCN, Linear, Sigmoid

net = DKPPCN(Linear(784, 512), Sigmoid(), Linear(512, 10), batch_size=1000, device="cpu")
net.configure(
    learning_rate=1e-6,
    inference_rate=0.1,
    feedback_rate=1e-4,
    lmbda=1e-2,
    optimizer="SGD",
    psi_optimizer="SGD",
)
net.train_step(X, Y, inference_steps=1)

feedback_rate/psi_optimizer control Psi independently from W's own learning_rate/optimizer.

FullPCN

The "everything" network. DKP direct feedback is always on, same as DKPPCN, and every other mechanism in the library is bolted on as an independently toggleable configure() keyword, off by default, so a plain FullPCN reproduces DKPPCN exactly:

Toggle	What it does	When it can change
use_mu_pc_scaling	µPC's Table 1 parameterization: per-layer forward scale a plus unit-variance weight init	configure() time only, baked in by compile()
use_residual_connections	Skip connection (mu += z) on every middle hidden layer; needs matching input/output width	configure() time only
use_ipc	Incremental PC: weights update every settling step, not once at the end	any time, via set_use_ipc()
use_epc	Error-based PC: one full forward+backward sweep per settling step (exact PC gradient via reverse-mode AD) instead of one-hop local messages	any time, via set_use_epc()
use_momentum / momentum_beta	EMA-smooths the settling update direction before applying it to z	any time
use_cross_entropy	Softmax cross-entropy energy on the terminal layer only	any time

from deepity import FullPCN, Linear, Sigmoid

net = FullPCN(Linear(784, 512), Sigmoid(), Linear(512, 10), batch_size=250)
net.configure(
    learning_rate=1e-6,
    inference_rate=0.1,
    feedback_rate=1e-4,
    optimizer="ADAMW",
    use_ipc=True,
    use_momentum=True,
    momentum_beta=0.9,
)

ConvolutionalPCN and SimpleConvolutionalPCN

Convolutional variants, used by the ImageNet research scripts (research/imagenet_pc_vgg7_ce.py, research/imagenet.py). Declared with Convolution layers instead of Linear, plus a required input_shape=(channels, height, width):

from deepity import ConvolutionalPCN, Convolution, ReLU

net = ConvolutionalPCN(
    Convolution(out_channels=32, kernel_h=3, kernel_w=3, pad_h=1, pad_w=1), ReLU(),
    input_shape=(3, 64, 64),
    batch_size=128,
)

ConvolutionalPCN wraps dy.ConvPCNetwork (standard precision-weighted PC). SimpleConvolutionalPCN wraps dy.SimpleConvPCNetwork and takes an optional batch_size (auto-detected like SimplePCN, unlike ConvolutionalPCN which requires it explicitly).