TY - GEN
T1 - Unsupervised Detection of LLM-Generated Text in Korean Using Syntactic and Semantic Cues
AU - Jeon, Heejeong
AU - Park, Minsu
AU - Choi, Yun Seok
AU - Park, Eunil
N1 - Publisher Copyright:
©2026 Association for Computational Linguistics.
PY - 2026
Y1 - 2026
N2 - As Large Language Models (LLMs) are increasingly used for content creation, detecting AI-generated text has become a critical challenge. Prior work has largely focused on English, leaving low-resource languages such as Korean underexplored. We propose an unsupervised detection framework that integrates two complementary signals: syntactic token cohesiveness (TOCSIN) and semantic regeneration similarity (SimLLM). To support evaluation, we construct a Korean pairwise dataset of 1,000 anchors with continuation- and regeneration-style generations and further assess performance across domains (news, research paper abstracts, essays) and model families (GPT-3.5 Turbo, GPT-4o, HyperCLOVA X, LLaMA-3-8B). Without any training, our ensemble achieves up to 0.963 F1 and 0.985 ROC-AUC, outperforming baselines. These results demonstrate that the combination of syntactic and semantic cues enables robust unsupervised detection in low-resource settings. Code available at https://github.com/dxlabskku/llm-detection-main.
AB - As Large Language Models (LLMs) are increasingly used for content creation, detecting AI-generated text has become a critical challenge. Prior work has largely focused on English, leaving low-resource languages such as Korean underexplored. We propose an unsupervised detection framework that integrates two complementary signals: syntactic token cohesiveness (TOCSIN) and semantic regeneration similarity (SimLLM). To support evaluation, we construct a Korean pairwise dataset of 1,000 anchors with continuation- and regeneration-style generations and further assess performance across domains (news, research paper abstracts, essays) and model families (GPT-3.5 Turbo, GPT-4o, HyperCLOVA X, LLaMA-3-8B). Without any training, our ensemble achieves up to 0.963 F1 and 0.985 ROC-AUC, outperforming baselines. These results demonstrate that the combination of syntactic and semantic cues enables robust unsupervised detection in low-resource settings. Code available at https://github.com/dxlabskku/llm-detection-main.
UR - https://www.scopus.com/pages/publications/105039156503
U2 - 10.18653/v1/2026.findings-eacl.77
DO - 10.18653/v1/2026.findings-eacl.77
M3 - Conference contribution
AN - SCOPUS:105039156503
T3 - 19th Conference of the European Chapter of the Association for Computational Linguistics, Findings of EACL 2026
SP - 1504
EP - 1518
BT - 19th Conference of the European Chapter of the Association for Computational Linguistics, Findings of EACL 2026
PB - Association for Computational Linguistics (ACL)
T2 - 19th Conference of the European Chapter of the Association for Computational Linguistics, Findings of EACL 2026
Y2 - 24 March 2026 through 29 March 2026
ER -