Description

ASCENT is an intelligent test-prioritization tool designed to make mutation testing of Solidity smart contracts significantly more efficient. It leverages Neural Monte Carlo Tree Search (Neural-MCTS) to learn, in real time, which test methods are most effective at detecting injected faults.

Rather than relying on predefined heuristics or system-specific tuning, ASCENT models test selection as a sequential decision-making process. Each test execution provides feedback that continually refines the tool’s policy, enabling ASCENT to adapt dynamically to the characteristics of the smart contract under analysis.

By guiding the testing process toward high-impact tests earlier, ASCENT reduces unnecessary executions and focuses computational resources where they matter most. This makes mutation testing more scalable and practical for real-world decentralized application development, while remaining fully independent of the underlying testing framework.

Resources

ASCENT Repository: link

To Appear

Mutant Generation

This section provides a setup guide for configuring Sumo Mutation Testing in a project that uses Hardhat for Ethereum smart contract development. Follow these steps to configure your project to run mutation testing with Sumo, Surya, and other necessary tools. You can skip this part if you plan on trying the tool on mutants already available in this repository.

🚀 Prerequisites

Before you begin, make sure you have the following installed

• Node.js (v14.x or higher)
• Python (v3.x or higher)

🔧 Steps for Configuration

1. Add Sumo to the SUT

Add the path to the .tgz file generated by the Sumo package in the dependencies section of your SUT (System Under Test) package.json:

"dependencies": {
    "@morenabarboni/sumo": "file:../../morenabarboni-sumo-x.x.x.tgz"
}

2. Add Mochawesome, Recast, and Babel Parser to the SUT

In your package.json, include the following dependencies for reporting and parsing:

"dependencies": {
    "mochawesome": "7.1.3",
    "@babel/parser": "7.25.3",
    "recast": "0.23.9"
}

3. Update hardhat-config.js to add Mochawesome in the config

Modify your hardhat-config.js file to set mochawesome as the test reporter so you get detailed HTML/JSON test reports.

const config: HardhatUserConfig = {
  solidity: {
    version: "0.8.16",
    settings: {
      optimizer: {
        enabled: true,
        runs: 1000,
      },
    },
  },
  mocha: {
    reporter: 'mochawesome',
    reporterOptions: {
      reportFilename: 'mochawesome', // Customize the report filename
      quiet: false,
      json: true,
      html: false // Set to false if you don't want HTML output
    }
  },
  ....
}

4. Install and configure solidity-coverage in the SUT

Add solidity-coverage to your package.json to check test coverage for your smart contracts.

"dependencies": {
  "solidity-coverage": "0.7.22"
}

5. Install dependencies

From within the SUT’s folder, run the command npm install to install all required dependencies.

6. Install and run Surya

Install Surya to generate control flow graphs (CFG) for your Solidity contracts using the command npm install -g surya and run the generation with surya graph contracts/**/*.sol | dot > cfg.dot

This generates a .dot file that can be used with tools like Graphviz to visualize the flow of your contract code.

6. Compile contracts

Compile your Solidity contracts using Hardhat with the command npx hardhat compile and run mutation testing with Sumo through npx sumo test.

7. Generate Hardhat’s coverage matrix

Generate a coverage matrix for your contracts using Hardhat running npx hardhat coverage --matrix

8. Add information from CFG and coverage matrix to mutations.json

Incorporate the data from the Surya CFG and the Hardhat coverage matrix into your mutations.json file to provide additional context to the mutants:

npx sumo addMutationsContext

⚡ Running Tests Prioritization and Execution

1. Optional (suggested)

Within the ASCENT folder, create a Python virtual environment

python3 -m venv .venv

Activate the freshly created virtual environment

source .venv/bin/activate

2. Install needed Python libraries from the requirements.txt file

pip install -r requirements.txt

3. Optional (if we skipped mutant generation and want to use the mutations available in the repository)

Prerequisite: make sure you have Git LFS installed. This allows the download of large mutation files from the repository, avoiding rerunning mutant generation.

git lfs fetch --all
git lfs pull

4. Run the prioritization on the chosen project and tests

python3 prioritizer.py --mutants path_to_mutations_json_file --tests_folder path_to_test_folder --coverage path_to_test_matrix_json --sut_name project_name

Example

python3 prioritizer.py --mutants sumo_results/thorwallet/mutations.json --tests_folder case_studies/thorwallet/test/hardhat --coverage case_studies/thorwallet/testMatrix.json --sut_name thorwallet

4.1 The following parameters are available for more detailed prioritization control:

💡Note: If the plots don’t show, ensure you have set the correct Matplotlib backend for your OS. More information is available at: https://matplotlib.org/stable/users/explain/figure/backends.html