Description
Alchemist is a test-generation tool that leverages Large Language Models to automatically create test cases for Solidity smart contracts. The approach is grounded in the principles of the scientific method: it produces test hypotheses, evaluates them using real mutation outcomes, and iteratively refines its understanding of the system under test.
Using mutants as a feedback mechanism, Alchemist systematically explores contract behaviors and incrementally improves the generated tests based on how effectively they expose injected faults. This closed-loop process allows the tool to adapt its strategies over time, guiding the LLM toward increasingly precise, meaningful, and targeted test cases.
Alchemist helps developers strengthen the reliability of decentralized applications by automating the creation of effective tests, reducing manual effort, and ensuring broader behavioral coverage through intelligent, feedback-driven test synthesis.
Resources
Alchemist Repository: link
Publications
- Barboni, M., Lampa, F., Morichetta, A., Polini, A., & Zulkoski, E. (2025, June). Alchemist: LLM-Driven Test Generation using Solidity Mutants and the Scientific Method. In 2025 IEEE International Conference on Blockchain and Cryptocurrency (ICBC) (pp. 1-5). IEEE.
- Barboni, M., Lampa, F., Morichetta, A., Polini, A., & Zulkoski, E. (2025, July). Mutant-Driven Test Generation for Ethereum Smart Contracts via LLMs. In 2025 IEEE International Conference on Artificial Intelligence Testing (AITest) (pp. 209-216). IEEE.
