According to arxiv.org, researchers have developed Rule2Text, a comprehensive framework that uses large language models (LLMs) to generate natural language explanations for mined logical rules in knowledge graphs (KGs). The framework addresses the challenge that logical rules extracted from knowledge graphs are “often difficult for humans to interpret due to their inherent complexity and the idiosyncratic labeling conventions of individual KGs.”
According to the research paper, the team conducted extensive experiments using multiple datasets, including Freebase variants (FB-CVT-REV, FB+CVT-REV, and FB15k-237) and the ogbl-biokg dataset, with rules mined using AMIE 3.5.1. The researchers systematically evaluated several LLMs across various prompting strategies, including zero-shot, few-shot, variable type incorporation, and Chain-of-Thought reasoning.
The study includes both human evaluation of generated explanations for correctness and clarity, and an “LLM-as-a-judge framework that demonstrates strong agreement with human evaluators” to address evaluation scalability, according to arxiv.org. Using the best-performing model (Gemini 2.0 Flash), the team fine-tuned the open-source Zephyr model, with results showing “significant improvements in explanation quality after fine-tuning, with particularly strong gains in the domain-specific dataset.”
According to arxiv.org, all code and data are publicly available on GitHub.