Multiple Statistical Fault Injections for AI systems

Memory faults affecting Neural Networks (NNs) are a critical source of vulnerability in modern AI systems. Estimating realistic failure rates is challenging because most software-side assessments do not capture how multiple, concurrent faults manifest in practice, and the space of multi-fault configurations grows combinatorially. This thesis introduces a Multiple Statistical Fault Injection (MSFI) framework that provides statistically sound failure-rate estimates for neural networks under multiple simultaneous weight faults, while keeping the number of injected faults and the memory footprint under control. The method targets application-level vulnerability: multiple bit-flip or stuck-at faults are injected into the network weights, and the resulting behavior is measured on the designated test set. A key methodological choice is to use quantized models purely for computational tractability: the discrete, bounded parameter space makes fault-site enumeration and exact, reversible bit-level edits efficient, while leaving the underlying failure mechanisms unchanged. Statistically, the framework adopts an iterative design by default: it starts with a pilot, updates the estimate and the required sample size after each block, and stops automatically once the target precision at the chosen confidence is met—avoiding unnecessary injections. A non-iterative, one-shot design is included as a baseline for comparison: it computes a conservative sample size upfront and typically requires many more injections, but confirms that the same precision can be reached. The pipeline is fully online and memory-aware: fault combinations are generated on demand; injections and restorations are performed with bit-level accuracy; clean versus faulty predictions are compared without storing large intermediate tensors. In addition to the core failure-rate estimate, the framework reports qualitative indicators useful for analysis and records the worst observed injections. The approach is validated on tabular multilayer perceptrons (Banknote, Wine, DryBean) and on a compact convolutional network (ResNet-20 on CIFAR-10), all evaluated in their quantized versions for computational tractability. Across networks and multi-fault settings, the iterative MSFI reaches the target precision while sampling only a tiny fraction of the multi-fault space compared with exhaustive baselines, and achieves a substantial reduction—often around seventy percent—in the total number of injected faults relative to the one-shot baseline. Overall, the contribution is a practical and reproducible methodology to quantify the vulnerability of neural networks to multiple simultaneous memory faults, with explicit control over statistical precision.