This conclusion introduces a pioneering anomaly detection technique designed for practical application in real-world Federated Learning systems. The method employs an early cross-round check and Zero-Knowledge Proofs to efficiently detect and remove anomaly client models during attacks, preserving the integrity of benign submissions. The approach is well-suited for real-world FL, with future plans extending its capabilities to asynchronous FL and vertical FL scenarios.
People Mentioned
Company Mentioned
This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license.
We present a cutting-edge anomaly detection technique specifically designed for the real-world FL systems. Our approach utilizes an early cross-round check that activates subsequent anomaly detection exclusively in the presence of attacks. When attacks happen, our approach removes anomaly client models efficiently, ensuring that the local models submitted by benign clients remain unaffected. Further, by leveraging ZKPs, participating clients get to verify the integrity of the anomaly detection and removal performed by the server. The practical design and the inherent efficiency of our approach make it well-suited for real-world FL systems. Our plans for future works include extending our approach to asynchronous FL and vertical FL scenarios.
REFERENCES
T. Aoki. On the stability of the linear transformation in banach spaces. Journal of the Mathematical Society of Japan, 2(1-2):64–66, 1950.
E. Bagdasaryan, A. Veit, Y. Hua, D. Estrin, and V. Shmatikov. How to backdoor federated learning. In AISTATS, 2020a.
Eugene Bagdasaryan, Andreas Veit, Yiqing Hua, Deborah Estrin, and Vitaly Shmatikov. How to backdoor federated learning. In International Conference on Artificial Intelligence and Statistics, pp. 2938–2948. PMLR, 2020b.
Arjun Nitin Bhagoji, Supriyo Chakraborty, Prateek Mittal, and Seraphin Calo. Analyzing federated learning through an adversarial lens. In International Conference on Machine Learning, pp. 634– 643. PMLR, 2019.
P. Blanchard, E. M. El Mhamdi, R. Guerraoui, and J. Stainer. Machine learning with adversaries: Byzantine tolerant gradient descent. In NeurIPS, 2017.
David Byrd and Antigoni Polychroniadou. Differentially private secure multi-party computation for federated learning in financial applications. In Proceedings of the First ACM International Conference on AI in Finance, pp. 1–9, 2020.
Sebastian Caldas, Sai Meher Karthik Duddu, Peter Wu, Tian Li, Jakub Konecnˇ y, H Brendan McMa- ` han, Virginia Smith, and Ameet Talwalkar. Leaf: A benchmark for federated settings. arXiv preprint arXiv:1812.01097, 2018.
Xiaoyu Cao and Neil Zhenqiang Gong. Mpaf: Model poisoning attacks to federated learning based on fake clients. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3396–3404, 2022.
Xiaoyu Cao and Neil Zhenqiang Gong. Mpaf: Model poisoning attacks to federated learning based on fake clients. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3396–3404, 2022.
Xiaoyu Cao, Zaixi Zhang, Jinyuan Jia, and Neil Zhenqiang Gong. Flcert: Provably secure federated learning against poisoning attacks. IEEE Transactions on Information Forensics and Security, 17: 3691–3705, 2022.
Mingqing Chen, Rajiv Mathews, Tom Ouyang, and Franc¸oise Beaufays. Federated learning of outof-vocabulary words. arXiv preprint arXiv:1903.10635, 2019.
Y. Chen, L. Su, and J. Xu. Distributed statistical machine learning in adversarial settings: Byzantine gradient descent. ACM on Measurement and Analysis of Computing Systems, 1(2):1–25, 2017.
Alexander Chowdhury, Hasan Kassem, Nicolas Padoy, Renato Umeton, and Alexandros Karargyris. A review of medical federated learning: Applications in oncology and cancer research. In Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 7th International Workshop, BrainLes 2021, Held in Conjunction with MICCAI 2021, Virtual Event, September 27, 2021, Revised Selected Papers, Part I, pp. 3–24. Springer, 2022.
M. Fang, X. Cao, J. Jia, and N. Gong. Local model poisoning attacks to Byzantine-robust federated learning. In USENIX Security, 2020.
B. Feng, L. Qin, Z. Zhang, Y. Ding, and S. Chu. ZEN: An optimizing compiler for verifiable, zero-knowledge neural network inferences. 2021. Cryptology ePrint Archive.
Grant S Fletcher. Clinical epidemiology: the essentials. Lippincott Williams & Wilkins, 2019.
R. Freivalds. Probabilistic machines can use less running time. In IFIP Congress, 1977.
Shuhao Fu, Chulin Xie, Bo Li, and Qifeng Chen. Attack-resistant federated learning with residualbased reweighting. arXiv preprint arXiv:1912.11464, 2019.
Clement Fung, Chris JM Yoon, and Ivan Beschastnikh. The limitations of federated learning in sybil settings. In RAID, pp. 301–316, 2020.
S. Goldwasser, S. Micali, and C. Rackoff. The knowledge complexity of interactive proof systems. SIAM Jour. on Comp., 18(1):186–208, 1989.
J. Groth. On the size of pairing-based non-interactive arguments. In Eurocrypt, 2016.
Rachid Guerraoui, Sebastien Rouault, et al. The hidden vulnerability of distributed learning in ´ byzantium. In International Conference on Machine Learning, pp. 3521–3530. PMLR, 2018.
S. Han, H. Wang, J. Wan, and J. Li. An iterative scheme for leverage-based approximate aggregation. In IEEE ICDE, 2019.
Shanshan Han, Baturalp Buyukates, Zijian Hu, Han Jin, Weizhao Jin, Lichao Sun, Xiaoyang Wang, Chulin Xie, Kai Zhang, Qifan Zhang, et al. Fedmlsecurity: A benchmark for attacks and defenses in federated learning and federated llms. arXiv preprint arXiv:2306.04959, 2023.
Andrew Hard, Kanishka Rao, Rajiv Mathews, Swaroop Ramaswamy, Franc¸oise Beaufays, Sean Augenstein, Hubert Eichner, Chloe Kiddon, and Daniel Ramage. Federated learning for mobile ´ keyboard prediction. arXiv preprint arXiv:1811.03604, 2018.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778, 2016.
L. He, S. P. Karimireddy, and M. Jaggi. Byzantine-robust decentralized learning via self-centered clipping. 2022. Available on arXiv:2202.01545.
Xiao Jin, Pin-Yu Chen, Chia-Yi Hsu, Chia-Mu Yu, and Tianyi Chen. Cafe: Catastrophic data leakage in vertical federated learning. Advances in Neural Information Processing Systems, 34:994–1006, 2021.
Sai Praneeth Karimireddy, Lie He, and Martin Jaggi. Byzantine-robust learning on heterogeneous datasets via bucketing. arXiv preprint arXiv:2006.09365, 2020.
Sanjay Kariyappa, Chuan Guo, Kiwan Maeng, Wenjie Xiong, G. Edward Suh, Moinuddin K. Qureshi, and Hsien-Hsin S. Lee. Cocktail party attack: Breaking aggregation-based privacy in federated learning using independent component analysis. In International Conference on Machine Learning, 2022. URL //api.semanticscholar.org/CorpusID: 252211968.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Maximilian Lam, Gu-Yeon Wei, David Brooks, Vijay apa Reddi, and Michael Mitzenmacher. Gradient disaggregation: Breaking privacy in federated learning by reconstructing the user participant matrix. In International Conference on Machine Learning, pp. 5959–5968. PMLR, 2021.
S. Lee, H. Ko, J. Kim, and H. Oh. vCNN: Verifiable convolutional neural network based on zksnarks. 2020. Cryptology ePrint Archive.
David Leroy, Alice Coucke, Thibaut Lavril, Thibault Gisselbrecht, and Joseph Dureau. Federated learning for keyword spotting. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6341–6345, 2019.
Suyi Li, Yong Cheng, Wei Wang, Yang Liu, and Tianjian Chen. Learning to detect malicious clients for robust federated learning. arXiv preprint arXiv:2002.00211, 2020.
T. Liu, X. Xie, and Y. Zhang. ZkCNN: Zero knowledge proofs for convolutional neural network predictions and accuracy. In ACM CCS, 2021.
A. Lyon. Why are normal distributions normal? The British Journal for the Philosophy of Science, 65(3):621–649, 2014.
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, pp. 1273–1282. PMLR, 2017a.
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, pp. 1273–1282. PMLR, 2017b.
J. Osborne. Improving your data transformations: Applying the Box-Cox transformation. Practical Assessment, Research, and Evaluation, 15(1):12, 2010.
Mustafa Safa Ozdayi, Murat Kantarcioglu, and Yulia R Gel. Defending against backdoors in federated learning with robust learning rate. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 9268–9276, 2021.
Krishna Pillutla, Sham M Kakade, and Zaid Harchaoui. Robust aggregation for federated learning. IEEE Transactions on Signal Processing, 70:1142–1154, 2022.
Swaroop Ramaswamy, Rajiv Mathews, Kanishka Rao, and Franc¸oise Beaufays. Federated learning for emoji prediction in a mobile keyboard. arXiv preprint arXiv:1906.04329, 2019.
M. Rosenblatt. A central limit theorem and a strong mixing condition. National Academy of Sciences, 42(1):43–47, 1956.
R. M. Sakia. The Box-Cox transformation technique: A review. Journal of the Royal Statistical Society: Series D, 41(2):169–178, 1992.
E. B. Sasson, A. Chiesa, C. Garman, M. Green, I. Miers, E. Tromer, and M. Virza. Zerocash: Decentralized anonymous payments from bitcoin. In IEEE S&P, 2014.
Jingwei Sun, Ang Li, Louis DiValentin, Amin Hassanzadeh, Yiran Chen, and Hai Li. Fl-wbc: Enhancing robustness against model poisoning attacks in federated learning from a client perspective. Advances in Neural Information Processing Systems, 34:12613–12624, 2021.
Ziteng Sun, Peter Kairouz, Ananda Theertha Suresh, and H Brendan McMahan. Can you really backdoor federated learning? arXiv preprint arXiv:1911.07963, 2019.
Vale Tolpegin, Stacey Truex, Mehmet Emre Gursoy, and Ling Liu. Data poisoning attacks against federated learning systems. In European Symposium on Research in Computer Security, pp. 480– 501. Springer, 2020.
Richard Tomsett, Kevin Chan, and Supriyo Chakraborty. Model poisoning attacks against distributed machine learning systems. In Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications, volume 11006, pp. 481–489. SPIE, 2019.
H. Wang, K. Sreenivasan, S. Rajput, H. Vishwakarma, S. Agarwal, J. Sohn, K. Lee, and D. Papailiopoulos. Attack of the tails: Yes, you really can backdoor federated learning. In NeurIPS, 2020.
Jianhua Wang. Pass: Parameters audit-based secure and fair federated learning scheme against free rider. arXiv preprint arXiv:2207.07292, 2022.
S. Weisberg. Yeo-Johnson power transformations. 2001. Available at //www.stat.umn.edu/arc/yjpower.pdf.
Cong Xie, Oluwasanmi Koyejo, and Indranil Gupta. SLSGD: Secure and Efficient Distributed Ondevice Machine Learning. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 213–228. Springer, 2020.
H. Yang, X. Zhang, M. Fang, and J. Liu. Byzantine-resilient stochastic gradient descent for distributed learning: A Lipschitz-inspired coordinate-wise median approach. In IEEE CDC, 2019.
Dong Yin, Yudong Chen, Kannan Ramchandran, and Peter Bartlett. Byzantine-robust distributed learning: Towards optimal statistical rates. In International Conference on Machine Learning, pp. 5650–5659. PMLR, 2018.
Zaixi Zhang, Xiaoyu Cao, Jinyuan Jia, and Neil Zhenqiang Gong. Implementation of fldetector. //github.com/zaixizhang/FLDetector, 2022a.
Zaixi Zhang, Xiaoyu Cao, Jinyuan Jia, and Neil Zhenqiang Gong. Fldetector: Defending federated learning against model poisoning attacks via detecting malicious clients. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 2545–2555, 2022b.
Zhengming Zhang, Ashwinee Panda, Linyue Song, Yaoqing Yang, Michael W. Mahoney, Joseph Gonzalez, Kannan Ramchandran, and Prateek Mittal. Neurotoxin: Durable backdoors in federated learning. In International Conference on Machine Learning, 2022c. URL https: //api.semanticscholar.org/CorpusID:249889464.
A APPENDIX
A.1 PROOF OF THE RANGE OF PPV
B ZKP IMPLEMENTATION
B.1 CHOICE OF THE ZKP SYSTEM
In our implementation, we use the Groth16 (Groth, 2016) zkSNARK scheme implemented in the Circom library (Circom Contributors, 2022) for all the computations described above. We chose this ZKP scheme because its construction ensures constant proof size (128 bytes) and constant verification time. Because of this, Groth16 is popular for blockchain application due to small on-chain computation. There are other ZKP schemes based on different construction that can achieve faster prover time (Liu et al., 2021), but the proof size is too big and verification time is not constant, which is a problem if verifier lacks computational power. The construction of a ZKP scheme that is efficient for both prover and verifier is still an open research direction.
B.2 SUPPLEMENTARY EXPERIMENTS FOR IMPORTANCE LAYER
