TY - GEN
T1 - CryptoLLM
T2 - 29th European Symposium on Research in Computer Security, ESORICS 2024
AU - Baek, Heewon
AU - Lee, Minwook
AU - Kim, Hyoungshick
N1 - Publisher Copyright:
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.
PY - 2024
Y1 - 2024
N2 - We propose CryptoLLM, a novel static analysis tool leveraging large language models (LLMs) to detect cryptographic API misuse vulnerabilities. Integrating optimized code slicing with fine-tuned LLMs, CryptoLLM achieves superior detection capabilities. After evaluating four models, we recommend CodeT5. CryptoLLM outperforms existing rule-based tools such as CryptoGuard, CogniCrypt, and SpotBugs on the CryptoAPI-Bench dataset (F1 score: 0.935). For unseen real-world Android apps, with a 20-minute analysis limit, CryptoLLM achieved the highest F1 score of 0.898, analyzing all apps without errors, while other tools failed to analyze a significant proportion, with CryptoGuard’s highest F1 score at 0.645. Although CryptoLLM ’s performance initially dropped to 0.749 F1 score on mutated code, retraining with augmented data improved it to 0.988, demonstrating adaptability across diverse datasets.
AB - We propose CryptoLLM, a novel static analysis tool leveraging large language models (LLMs) to detect cryptographic API misuse vulnerabilities. Integrating optimized code slicing with fine-tuned LLMs, CryptoLLM achieves superior detection capabilities. After evaluating four models, we recommend CodeT5. CryptoLLM outperforms existing rule-based tools such as CryptoGuard, CogniCrypt, and SpotBugs on the CryptoAPI-Bench dataset (F1 score: 0.935). For unseen real-world Android apps, with a 20-minute analysis limit, CryptoLLM achieved the highest F1 score of 0.898, analyzing all apps without errors, while other tools failed to analyze a significant proportion, with CryptoGuard’s highest F1 score at 0.645. Although CryptoLLM ’s performance initially dropped to 0.749 F1 score on mutated code, retraining with augmented data improved it to 0.988, demonstrating adaptability across diverse datasets.
UR - https://www.scopus.com/pages/publications/85204377260
U2 - 10.1007/978-3-031-70879-4_18
DO - 10.1007/978-3-031-70879-4_18
M3 - Conference contribution
AN - SCOPUS:85204377260
SN - 9783031708787
T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
SP - 353
EP - 373
BT - Computer Security – ESORICS 2024 - 29th European Symposium on Research in Computer Security, Proceedings
A2 - Garcia-Alfaro, Joaquin
A2 - Kozik, Rafał
A2 - Choraś, Michał
A2 - Katsikas, Sokratis
PB - Springer Science and Business Media Deutschland GmbH
Y2 - 16 September 2024 through 20 September 2024
ER -