Architectural Backdoors

Training a deep neural network (DNN) from scratch is expensive and time consuming. It requires large datasets, powerful GPUs, and significant computational resources. To reduce cost, training is often outsourced through Machine Learning as a Service (MLaaS)[1], or developers rely on pretrained models from public repositories such as Hugging Face and fine-tune them using transfer learning.

While these approaches are efficient, they introduce a serious supply-chain security risk. When a model is outsourced or downloaded from an external source, users must trust that it is safe. But what if the model already contains a hidden vulnerability?

A backdoor is a malicious behavior embedded inside a model that remains dormant under normal inputs but activates when a specific trigger is present. Classical backdoors are usually hidden in training data or model weights and can often be mitigated through fine-tuning or pruning.

Architectural backdoors are more dangerous. Instead of modifying weights, they hard-code malicious logic directly into the model’s architecture through hidden branches or gating mechanisms. Because the vulnerability is structural, it can survive retraining and remain undetected.

Architectural Backdoors: Trigger Types

Architectural backdoors rely on a trigger detection logic that is embedded directly inside the model’s architecture. The attacker inserts a structural mechanism that checks whether a specific pattern is present in the input. If the trigger pattern is detected, the malicious branch activates and overrides the normal prediction.

According to recent studies on architectural backdoors [2, 3] trigger detectors can be categorized along several important dimensions: detection mode, signal propagation, and attack goal.

Detection Mode

Operator-Based Detectors

Operator-based detectors are built using only standard machine learning operations. Combination of average pooling, max/min pooling, subtraction, ReLU, sign functions.

These detectors behave like handcrafted pattern recognizers. For example, they can detect a checkerboard trigger by checking for alternating pixel patterns in horizontal and vertical directions. Because they use only standard operations, they are difficult to distinguish from legitimate architectural components.

Constant-Based Detectors

Constant-based detectors use fixed-value tensors embedded directly into the architecture. For example: convolution kernels with hard-coded values (e.g., ±255), and added large constant biases.

These act as template matchers for a specific trigger pattern, such as a white square or checkerboard patch. Unlike operator-based detectors, they explicitly encode a reference pattern into the network.

Both approaches allow the attacker to create precise structural trigger detection without relying on learned weights.

Signal Propagation: How the Trigger Travels

Separate Path

In a separate-path design, the malicious branch is completely isolated from the main computation. Normal path processes input as usual, and a hidden branch checks for the trigger. If detected, it directly overrides the final output.

This is conceptually simple and easier to detect through graph inspection.

Shared Path

In a shared-path design, malicious logic is embedded inside layers that are also used for normal computation. Legitimate and malicious signals share intermediate layers. Because malicious components overlap with normal operations, removal becomes more difficult.

Interleaved Path

Interleaved-path backdoors distribute the trigger logic across multiple layers. The malicious signal; partially propagates through Layer 1, continues through Layer 2, gradually accumulates influence, finally hijacks the output.

No single component seems obviously malicious. The attack activates only when all partial conditions align, making interleaved backdoors the most stealthy and hardest to detect.

Attack Goal: Targeted vs Untargeted

Finally, architectural backdoors differ in their objective.

In a targeted attack, whenever the trigger appears, the model predicts a specific attacker-chosen class. This makes the attack deterministic and precise.

In an untargeted attack:The trigger only forces the model to be wrong,The incorrect output may vary. Here, the attacker’s goal is simply to cause failure rather than redirect to a specific label.

Together, these dimensions show that architectural backdoors are not a single technique but a design space of structural manipulations. The attacker can choose how the trigger is detected, how the signal propagates, and what final behavior is enforced. This flexibility makes architectural backdoors significantly more versatile and potentially more dangerous than classical backdoor attacks.

Detection of the architectural backdoors

Fuzzing

Fuzzing is an automated testing technique where a program is given repeatedly random, unexpected, or invalid inputs to observe if it breaks. For architectural backdoors, fuzzing might be able to discover hidden triggers automatically by revealing abnormal reactions to unusual inputs.

Explainable AI (XAI)

Grad-CAM (Gradient-weighted Class Activation Mapping) visualizes which regions of an image influence a model’s decision.

The method works by; taking gradients of a target class, backpropagating them to the last convolutional layer, and producing a heatmap of important regions.

In backdoored models, Grad-CAM can sometimes reveal a suspicious focus on trigger regions. However, architectural backdoors that operate internally through gating logic may not produce obvious saliency anomalies.

Artificial Brain Stimulation

ABS detects backdoors by artificially activating internal neurons and observing output changes[6]. If stimulating certain neurons consistently forces a specific output, those neurons may belong to a hidden malicious pathway.

ABS was originally designed for classical backdoors. Its effectiveness against deeply embedded architectural subgraphs is limited, but still promising.

Activation Clustering

Activation Clustering analyzes hidden-layer activations per class and identifies anomalous clusters corresponding to triggered samples[5].

This method assumes that poisoned samples form a separable cluster in feature space. For architectural backdoors, if triggered inputs produce distinct activation patterns, clustering may reveal them.

However, if the architectural backdoor is fully dormant during clean operation, no anomaly is observable without trigger samples.

Removal of architectural backdoors

Removing architectural backdoors is significantly harder than removing weight-based ones. Fine-tuning and even re-training from the scratch are not sufficient to remove the backdoor.

Pruning

Pruning removes neurons or connections with low activation or importance.

While pruning can reduce attack success rate for simple single-branch backdoors, it often fails when the malicious logic is distributed.

Subgraph Removal

A more direct mitigation strategy is subgraph removal which works for seperated path architectural backdoors. Firsly, detect suspicious subgraphs via static analysis such as using NETRON. Than, surgically remove them and retrain the model.

Conclusion

Architectural backdoors represent a structural and persistent threat to modern deep learning systems. Unlike classical backdoors, they are embedded directly into the model’s architecture, making them resistant to retraining and many traditional defense techniques. As the ecosystem increasingly relies on pretrained and outsourced models, ensuring architectural integrity becomes critical.

In future work, we will evaluate existing detection and removal methods on more stealthy architectural backdoors, including newly designed trigger detectors. The goal is to better understand how robust current defenses are against advanced structural attacks and to identify more effective mitigation strategies.

References

[1] T. Gu, B. Dolan-Gavitt, and S. Garg, BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain, IEEE Access, vol. 7, pp. 47230–47244, 2019.

[2] H. Langford, I. Shumailov, Y. Zhao, R. Mullins, and N. Papernot, Architectural Neural Backdoors from First Principles, IEEE Symposium on Security and Privacy (S&P), 2025.

[3] M. Bober-Irizar, I. Shumailov, Y. Zhao, R. Mullins, and N. Papernot, Architectural Backdoors in Neural Networks, IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023.

[4] V. Childress, J. Collyer, and J. Knapp, Architectural Backdoors in Deep Learning: A Survey of Vulnerabilities, Detection, and Defense, 2024.

[5] B. Chen, W. Carvalho, N. Baracaldo, H. Ludwig, B. Edwards, T. Lee, I. Molloy, and B. Srivastava, Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering, IEEE Symposium on Security and Privacy (S&P), 2019.

[6] Y. Liu, W. Lee, G. Tao, S. Ma, Y. Aafer, and X. Zhang, ABS: Scanning Neural Networks for Back-doors by Artificial Brain Stimulation, ACM Conference on Computer and Communications Security (CCS), 2019.