\chapter{Conclusion}
\label{chap:discussion_conclusion}

The research investigated the integration of approximate computing techniques into instruction-tuned \glspl{LLM}, focusing on two architectures of comparable scale: LLaMA-3.2-1B-Instruct and Gemma-3-1B-Instruct. The objective was to determine how quantization, stochastic perturbation, pruning, memory masking, and \gls{LoRA} influence model accuracy, semantic similarity, and computational efficiency across multiple instruction-following datasets. As modern transformer models continue to grow, such approximation strategies offer a viable path toward efficient deployment on constrained hardware.

A modular approximation framework was developed to allow techniques to be applied individually and in combination during fine-tuning. The experimental pipeline incorporated simulated low-precision arithmetic, controlled stochasticity, and structured sparsity, enabling a unified analysis of their effects on numerical stability, representational robustness, and decoding behaviour. Evaluation metrics included token-overlap scores, which are \gls{BLEU}, \gls{ROUGE-L} and F1, semantic similarity measured via SBERT cosine similarity, and efficiency metrics such as inference latency, output size, and model footprint.

Experiments were carried out using cross-dataset evaluation schemes rather than training and testing on the same corpus. In the primary configuration, the models were fine-tuned on the Alpaca dataset and evaluated on Dolly-15k, a setting denoted as \textit{Alpaca--Dolly}. This setup intentionally exposes the models to a distribution shift: Alpaca provides highly structured, template-like instruction--response pairs, while Dolly-15k contains more diverse, conversational, and open-ended instructions. As a result, approximation errors that remain hidden under Alpaca-style prompts become more pronounced during evaluation on Dolly, revealing how robust each technique is to linguistic variability.

The alternative configurations \textit{Alpaca--AgentInstruct} and \textit{AgentInstruct--Dolly} follow the same cross-dataset principle. AgentInstruct introduces shorter, more procedural prompts, which amplify the effect of even small approximation distortions. These cross-dataset setups consistently showed that the impact of reduced precision and stochastic computation is tightly coupled to the mismatch between training and evaluation distributions. Thus, approximation performance cannot be interpreted independently of the dataset pair, as different forms of linguistic structure expose different failure modes of each technique.


Across all experiments, LLaMA-3.2-1B-Instruct demonstrated substantially higher tolerance to quantization, pruning, and stochastic perturbation than Gemma-3-1B-Instruct. LLaMA repeatedly achieved strong improvements under INT8 quantization, often reducing latency and model size while simultaneously increasing both token-overlap and SBERT scores. In contrast, Gemma exhibited pronounced degradation under INT8, inference became an order of magnitude slower, and accuracy stagnated or decreased relative to baseline. These findings align with external observations that Gemma’s activation distribution and normalization design make it more sensitive to mid-precision quantizers.

More aggressive low-bit quantizers produced divergent outcomes between the two models. For LLaMA, INT4 and NF4 caused substantial losses in BLEU, ROUGE-L, and SBERT similarity, accompanied by large increases in latency due to simulated dequantization. Gemma, however, improved under INT4 and NF4 relative to its INT8 variant, a behaviour consistent with prior reports that small Gemma models sometimes favour lower-bit quantizers due to reduced systematic quantization bias. Although these improvements did not surpass LLaMA’s INT8 performance, they reveal architecture-dependent interactions between quantization level and representational robustness.

Pruning and stochastic perturbation produced interpretable patterns across datasets. On LLaMA, pruning preserved accuracy while reducing latency, and stochastic perturbation provided mild regularization benefits, yielding balanced improvements in both overlap-based and semantic metrics. These techniques, however, had limited or inconsistent benefits for Gemma, which frequently exhibited performance drops under structural modification or noise injection. Memory masking produced minimal changes in either model, suggesting that more adaptive masking strategies may be necessary for meaningful impact.

Hybrid techniques produced the most compelling trade-offs. For LLaMA, combinations such as Pruned + INT8 or INT8 + \gls{LoRA} consistently provided strong accuracy with moderate latency and compact outputs. On Gemma, hybrid techniques did not fully compensate for INT8 sensitivity, even with \gls{LoRA} or pruning, accuracy remained lower than baseline LLaMA performance, and inference remained significantly slower. These findings indicate that approximation gains depend on architectural resilience, and that techniques effective for one model family do not generalize universally.

Efficiency patterns further highlighted fundamental differences between the models. LLaMA’s INT8 variants achieved the lowest latencies among all tested configurations, sometimes approaching 0.6\,s per inference. In stark contrast, Gemma’s INT8 variants displayed the highest latencies of any model--often exceeding 15--20\,s per inference, suggesting limited compatibility with simulated 8-bit quantization workflows.

Overall, the results demonstrate that approximate computing techniques can be applied effectively to transformer-based \glspl{LLM}, but their success is heavily architecture-dependent. LLaMA-3.2-1B-Instruct consistently benefited from INT8 quantization, pruning, and stochastic perturbation, achieving strong accuracy and substantial latency reductions. Gemma-3-1B-Instruct, despite its similar parameter count, showed markedly lower robustness and significantly poorer performance under most approximation strategies. These differences highlight the importance of model–technique compatibility and emphasize that approximation cannot be assumed to generalize across architectures.

Several limitations remain. Experiments were conducted under simulated low-precision arithmetic, without hardware-optimized INT8 or INT4 kernels. Real quantization-aware backends may yield different latency and accuracy behaviours. The analysis was constrained to $\approx 1$B-parameter models; larger LLaMA variants may exhibit increased redundancy and therefore greater tolerance to approximation. Finally, although SBERT provided an improved measure of semantic fidelity, future work should incorporate additional embedding-based metrics, human evaluation, and energy/throughput profiling to fully characterize approximation effects.

In conclusion, the study provides a systematic analysis of how quantization, pruning, stochasticity, and low-rank adaptation interact with model architecture and task structure in small instruction-tuned \glspl{LLM}. The findings highlight INT8 and stochastic methods as the most effective techniques for LLaMA-3.2-1B-Instruct, while demonstrating that Gemma-3-1B-Instruct is significantly more fragile under approximation. These insights contribute to a deeper understanding of the trade-offs involved in deploying efficient LLMs and reinforce the potential of approximate computing as a cornerstone of sustainable and scalable \gls{AI} development.
