kayush1608/smai-a4-iocnn

SMAI A4: Input-Output Convex NN on CIFAR-10 (3072-256-128-10, ELU)

Input-output convex neural network (Sivaprasad et al., The Curious Case of Convex Neural Networks, arXiv:2006.05103): every class score is a convex function of the input image. Trained in PyTorch on the true labels (p = 0) with the same depth, widths and hyperparameters as the MLP baseline (Tasks 1.4/1.5).

Architecture

Widths	3072 → 256 → 128 → 10
Activations	elu, elu, identity (last layer: identity logits)
Loss	softmax cross-entropy
Parameters	820,874
Constraints	W of layers >= 2 non-negative; after every step negative w -> exp(w - 5)

Constraints: ELU hidden activations; the weights of layers 2 and 3 are non-negative (initialised U(0, 2/n_in); after every optimiser step each negative weight w is replaced by exp(w - 5)). The first layer starts from the MLP's first-layer weights and biases.

Preprocessing

flatten 32x32x3 (HWC) -> 3072, /255, standardise each column with the CIFAR-10 train-part mean/std stored in standardize_mean / standardize_std. The mean and standard deviation (3072 values each) are stored in the .npz file.

Training

CIFAR-10 train part (50,000 images), true labels (p = 0); optimiser SGD (no momentum), learning rate 0.05, batch size 64, gradient accumulation steps 1, 100 epochs, no early stopping. Seed 1049: weights from torch.manual_seed(seed + 1), batch order from torch.Generator().manual_seed(seed + 2).

Results (CIFAR-10 test part, 10,000 images, after epoch 100)

train accuracy	test accuracy	test loss	mean F1 (10 classes)
0.7954	0.4601	3.9428	0.4645

Files

• torch_iocnn_p0.npz: W0, b0, W1, b1, W2, b2 (W has shape (n_in, n_out), float64), activations, loss, standardize_mean, standardize_std
• torch_iocnn_p0.json: architecture, training and preprocessing details

Usage

import numpy as np
from huggingface_hub import hf_hub_download

d = np.load(hf_hub_download("kayush1608/smai-a4-iocnn", "torch_iocnn_p0.npz"))
act = {"sigmoid": lambda z: 0.5 * (1 + np.tanh(0.5 * z)),
       "elu": lambda z: np.where(z > 0, z, np.expm1(np.minimum(z, 0))),
       "identity": lambda z: z}

def logits(images_uint8):                      # (n, 32, 32, 3) uint8, HWC
    h = images_uint8.reshape(len(images_uint8), 3072) / 255.0
    h = (h - d["standardize_mean"]) / d["standardize_std"]
    for i, a in enumerate(d["activations"]):
        h = act[str(a)](h @ d[f"W{i}"] + d[f"b{i}"])   # W{i}: (n_in, n_out)
    return h                                   # (n, 10) class scores