Hardware-Accelerated KANs: Architecting Kolmogorov-Arnold Networks on FPGAs for Microsecond Inference

The Shift from MLPs to KANs: A Hardware Perspective

For decades, Multi-Layer Perceptrons (MLPs) have been the undisputed bedrock of deep learning. Their mathematical simplicity—relying on matrix multiplications ($Y = WX + B$) followed by static activation functions like ReLU or GELU—makes them highly compatible with the massively parallel, tensor-core-driven architecture of modern GPUs. However, a new paradigm has emerged: Kolmogorov-Arnold Networks (KANs).

Based on the Kolmogorov-Arnold representation theorem, KANs replace the static activation functions on nodes with learnable, univariate activation functions (typically parameterized as B-splines) placed directly on the edges (connections) of the network. While KANs show remarkable parameter efficiency and accuracy on complex scientific tasks, they present a massive computational bottleneck on traditional hardware. GPUs, optimized for dense matrix multiplications, struggle with the irregular memory access and computationally diverse evaluations of thousands of independent B-splines.

To unlock the true potential of KANs for real-time applications—such as autonomous driving, high-frequency trading, and robotics—we must look beyond standard silicon. Field Programmable Gate Arrays (FPGAs) offer a highly customizable fabric capable of executing non-standard mathematical operations with deterministic, microsecond-level latency. This guide explores how to architect and implement Kolmogorov-Arnold Networks directly on FPGA hardware using High-Level Synthesis (HLS).

The Math Behind KANs and the FPGA Mapping Problem

In a standard MLP layer, the computation is represented as:

$$x_{l+1} = \sigma(W_l x_l)$$

In contrast, a KAN layer with $n$ inputs and $m$ outputs computes:

$$y_j = \sum_{i=1}^{n} \phi_{i,j}(x_i)$$

Where each $\phi_{i,j}$ is a learnable, continuous univariate function. In practice, these functions are modeled as a combination of a base function (like SiLU) and a B-spline:

$$\phi(x) = w_b \cdot b(x) + w_s \cdot \text{spline}(x)$$

The B-spline itself is defined over a grid of intervals. Evaluating a B-spline requires determining which grid interval the input $x$ falls into, retrieving the corresponding local control points (weights), and computing a linear combination of basis polynomials.

On a GPU, evaluating thousands of unique B-splines involves significant branch divergence and unstructured memory lookups, which degrade throughput. On an FPGA, however, we can instantiate dedicated, parallel hardware pipelines for each spline. By leveraging localized Look-Up Tables (LUTs), dual-port Block RAMs (BRAMs), and dedicated DSP slices, we can evaluate hundreds of splines concurrently within a single clock cycle.

Architecting the FPGA-Optimized KAN Cell

To build an efficient FPGA-based KAN accelerator, we must design a modular "KAN Cell"—the hardware equivalent of a KAN edge connection. The main challenges in designing this cell are fixed-point representation, grid search localization, and spline coefficient interpolation.

1. Fixed-Point Optimization

Floating-point arithmetic is highly resource-intensive on FPGAs. To maximize throughput and conserve DSP slices, we must convert the KAN's continuous inputs, outputs, and spline coefficients into fixed-point representations (e.g., ap_fixed<16, 6> in AMD/Xilinx Vivado HLS, which provides 10 fractional bits and 6 integer bits).

2. Fast Grid Interval Search

A B-spline of degree $k$ is defined over a grid of knots. To evaluate the spline for an input $x$, we must find the knot interval $i$ such that $t_i \le x < t_{i+1}$. If we enforce a uniform grid spacing, this search simplifies from an $O(\log N)$ binary search to a simple $O(1)$ scaling and truncation operation:

$$\text{interval_index} = \lfloor (x - x_{min}) \cdot \text{grid_scale} \rfloor$$

This operation can be implemented using a single fixed-point multiplier and a bit-shift, eliminating complex branching logic in hardware.

3. Parallel Spline Evaluation Engine

Once the interval index is determined, the hardware must retrieve the $k+1$ active spline coefficients from local memory. For a cubic B-spline ($k=3$), we need 4 coefficients. By storing these coefficients in a multi-banked BRAM, we can fetch all 4 coefficients in a single clock cycle.

Let's look at how this architecture is structured inside an HLS C++ design.

#include <ap_fixed.h>
#include <hls_vector.h>

// Define fixed-point types
typedef ap_fixed<16, 6> data_t;
typedef ap_fixed<16, 2> coef_t;

#define KNOTS 8
#define DEGREE 3
#define COEFF_COUNT (KNOTS + DEGREE)

// Uniform B-spline evaluation for a single edge
data_t evaluate_spline(data_t x, coef_t weights[COEFF_COUNT]) {
    #pragma HLS INLINE
    
    // Normalize input to grid range [0, 1]
    data_t normalized_x = (x < 0) ? (data_t)0 : ((x > 1) ? (data_t)1 : x);
    
    // O(1) interval calculation for uniform grid
    data_t scaled = normalized_x * (KNOTS - 1);
    int interval = scaled.to_int();
    data_t t = scaled - interval; // Fractional part
    
    // Pre-calculate basis functions for cubic spline (Cox-de Boor recursion unrolled)
    data_t t2 = t * t;
    data_t t3 = t2 * t;
    
    data_t b0 = (-t3 + 3*t2 - 3*t + 1) / 6.0;
    data_t b1 = (3*t3 - 6*t2 + 4) / 6.0;
    data_t b2 = (-3*t3 + 3*t2 + 3*t + 1) / 6.0;
    data_t b3 = t3 / 6.0;
    
    // Read coefficients (can be mapped to registers/ROMs for single-cycle access)
    coef_t c0 = weights[interval];
    coef_t c1 = weights[interval + 1];
    coef_t c2 = weights[interval + 2];
    coef_t c3 = weights[interval + 3];
    
    // Multiply-Accumulate using DSP slices
    data_t result = (b0 * c0) + (b1 * c1) + (b2 * c2) + (b3 * c3);
    return result;
}

Scaling Up: Pipelining the KAN Layer

To build a full KAN layer, we must aggregate the outputs of multiple KAN cells. For an input vector of size $N$ and an output vector of size $M$, we require $N \times M$ spline evaluations.

By utilizing HLS pragmas, we can fully unroll the inner loops to evaluate all $N$ inputs for a single output node in parallel, forming a reduction tree.

void kan_layer(
    const data_t inputs[INPUT_SIZE],
    data_t outputs[OUTPUT_SIZE],
    const coef_t weights[OUTPUT_SIZE][INPUT_SIZE][COEFF_COUNT]
) {
    #pragma HLS INTERFACE s_axilite port=return
    #pragma HLS ARRAY_PARTITION variable=inputs complete
    #pragma HLS ARRAY_PARTITION variable=outputs complete
    #pragma HLS ARRAY_PARTITION variable=weights complete dim=1
    #pragma HLS ARRAY_PARTITION variable=weights complete dim=2

    for (int j = 0; j < OUTPUT_SIZE; j++) {
        #pragma HLS PIPELINE II=1
        data_t sum = 0;
        for (int i = 0; i < INPUT_SIZE; i++) {
            #pragma HLS UNROLL
            // Base function calculation (SiLU approximation: x * sigmoid(x))
            data_t base = inputs[i] / (1 + hls::exp(-inputs[i]));
            
            // Spline path
            data_t spline_val = evaluate_spline(inputs[i], weights[j][i]);
            
            sum += base + spline_val;
        }
        outputs[j] = sum;
    }
}

By specifying #pragma HLS PIPELINE II=1, we instruct the compiler to schedule the hardware such that a new input vector can be accepted every single clock cycle. The loop unrolling ensures that the summation occurs via a balanced binary adder tree, minimizing critical path delay and driving latency down to just a few nanoseconds per layer.

Memory Bottlenecks and Optimization Strategies

While the parallelized KAN layer achieves outstanding latency, it introduces a severe hardware constraint: parameter memory bandwidth.

Each spline requires its own set of coefficients. For a relatively small KAN layer of $64 \times 64$ with 12 coefficients per spline, we must store $64 \times 64 × 12 = 49,152$ fixed-point parameters. If we attempt to access these from external DDR memory during inference, the pipeline will stall indefinitely.

To prevent this, we must employ two strategies:

On-Chip Storage via Block RAM (BRAM) and UltraRAM (URM): Keep all spline coefficients stored locally in the FPGA's internal memory blocks. This allows parallel, multi-channel access to all parameters simultaneously.
Weight Quantization and Pruning: KANs are highly susceptible to pruning. Because many edges in a KAN can have their weights set to zero without a loss in accuracy, we can implement a sparse KAN engine. In hardware, this means we can skip evaluation for pruned edges, saving both logic gates (DSPs) and memory storage.

Comparative Benchmarks: FPGA-KAN vs. GPU-MLP

When we compare a custom FPGA KAN implementation (running on an AMD Alveo U250 card) against a traditional PyTorch-based MLP running on an NVIDIA A100 GPU, the results demonstrate the massive advantage of specialized hardware for edge deployment:

While the GPU dominates in bulk, throughput-heavy batch processing, the FPGA-driven KAN is orders of magnitude faster for single-sample, zero-batch streaming inference. This makes FPGA-accelerated KANs the ideal choice for mission-critical, ultra-low-latency control loops.

Embracing the KAN Hardware Revolution

Kolmogorov-Arnold Networks are redefining our understanding of neural network efficiency and interpretability. However, their unique mathematical formulation demands a parallel shift in how we design physical hardware. By abandoning the standard matrix-multiplication accelerators and crafting custom, spline-evaluating FPGA architectures, developers can achieve microsecond-level execution times with incredibly low power envelopes. As edge devices continue to demand smarter, faster autonomous decision-making capabilities, the marriage of KANs and FPGAs will represent a key frontier in high-performance hardware engineering.

Make sure to validate your designs using software-in-the-loop (SIL) testing, and use fixed-point analysis tools in HLS to ensure that quantization doesn't compromise the high representational accuracy that KANs naturally offer.