Description
Often scientific papers use conceptual causal diagrams to represent complex relationships. In ecology papers, these causal graphs can be used to depict species interactions, environmental conditions, ecological processes, etc. However, determining whether each of the causal relationships in such diagrams is strongly supported by the cited literature is difficult because the evidence is distributed across many reference papers and may vary in strength, direction and certainty levels. This project develops an LLM-assisted workflow for auditing such causal claims in ecology literature using the paper “Dynamics of biotic resistance to plant invasions” as a case study. The selected paper contains an explicit conceptual causal network describing hypothetical temporal dynamics of biotic resistance, including interactions among native communities, alien invaders, abiotic conditions, invasiveness, invasibility and biotic resistance or facilitation.
The workflow extracts text from the chosen paper and its cited references, processes the reference papers into structured sections, divides the extracted content into paragraph and sentence-level chunks, applies a custom causal-ecology term mapping and uses large language models to generate summaries of the referenced papers. These summaries are then used to support an author-feedback survey, where authors of the referenced papers are asked to evaluate whether the generated summaries accurately represent their work and whether the interpreted causal evidence is appropriate. The goal is not to directly challenge the original ecology paper or the referenced literature, but rather to evaluate whether LLMs can help trace and interpret causal evidence from cited literature. This study contributes a semi-automated workflow for causal-evidence auditing and highlights the need for expert validation when using LLMs for scientific literature interpretation.
| Topic | Topic 4: Can knowledge graphs and AI-based synthesis improve ecological synthesis? |
|---|