Skip to main navigation Skip to search Skip to main content

Unsupervised Detection of LLM-Generated Text in Korean Using Syntactic and Semantic Cues

  • Heejeong Jeon
  • , Minsu Park
  • , Yun Seok Choi
  • , Eunil Park
  • Sungkyunkwan University

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

As Large Language Models (LLMs) are increasingly used for content creation, detecting AI-generated text has become a critical challenge. Prior work has largely focused on English, leaving low-resource languages such as Korean underexplored. We propose an unsupervised detection framework that integrates two complementary signals: syntactic token cohesiveness (TOCSIN) and semantic regeneration similarity (SimLLM). To support evaluation, we construct a Korean pairwise dataset of 1,000 anchors with continuation- and regeneration-style generations and further assess performance across domains (news, research paper abstracts, essays) and model families (GPT-3.5 Turbo, GPT-4o, HyperCLOVA X, LLaMA-3-8B). Without any training, our ensemble achieves up to 0.963 F1 and 0.985 ROC-AUC, outperforming baselines. These results demonstrate that the combination of syntactic and semantic cues enables robust unsupervised detection in low-resource settings. Code available at https://github.com/dxlabskku/llm-detection-main.

Original languageEnglish
Title of host publication19th Conference of the European Chapter of the Association for Computational Linguistics, Findings of EACL 2026
PublisherAssociation for Computational Linguistics (ACL)
Pages1504-1518
Number of pages15
ISBN (Electronic)9798891763869
DOIs
StatePublished - 2026
Event19th Conference of the European Chapter of the Association for Computational Linguistics, Findings of EACL 2026 - Rabat, Morocco
Duration: 24 Mar 202629 Mar 2026

Publication series

Name19th Conference of the European Chapter of the Association for Computational Linguistics, Findings of EACL 2026

Conference

Conference19th Conference of the European Chapter of the Association for Computational Linguistics, Findings of EACL 2026
Country/TerritoryMorocco
CityRabat
Period24/03/2629/03/26

Fingerprint

Dive into the research topics of 'Unsupervised Detection of LLM-Generated Text in Korean Using Syntactic and Semantic Cues'. Together they form a unique fingerprint.

Cite this