LRTHT: An Efficient Log Clustering Framework Based on Radix Tree and Hash Table

Logs, which record detailed system operation statuses, are widely used in software optimization and system management. Log clustering is a fundamental step for various downstream tasks. However, the rapid increase in log volume severely limits the performance of the log clustering parsers. To address this challenge, we develop LRTHT, an efficient log clustering algorithm named Logram with radix tree and hash table (LRTHT). LRTHT first utilizes the existing Logram parser to convert the dynamic parts of each log entry into “<*>”. Then, these structured logs are grouped into different partitions based on their length. Finally, further fine-grained grouping is performed within each length partition. Specifically, we construct a template collection, where log templates are stored in a radix tree and a hash table. Each structured log entry matches a corresponding template label from the template collection. The radix tree addresses the inefficiency of template retrieval, while the hash table improves clustering accuracy. We evaluate LRTHT against 5 existing methods using 16 datasets. Experimental results demonstrate that, compared to existing baselines, LRTHT improves the F1 score and accuracy by at least 10.13% and 9.81%, respectively.