SuMo

Mutation testing is a powerful approach for assessing the fault-detection capability of a test suite. With SuMo, we propose a novel mutation testing tool for Ethereum Smart Contracts. SuMo implements a set of 44 mutation operators that were designed starting from the latest Solidity documentation, and from well-known mutation testing tools.
These allow to simulate a wide variety of faults that can be made by smart contract developers.
The set of operators was designed to limit the generation of stillborn mutants, which slow down the mutation testing process, and limit the usability of the tool.

________________________________________________________________________________________________

Usage:

Once SuMo is installed, it is possible to launch a new mutation testing campaign.
However, you must make sure that:

  1. all the contracts to be mutated are located in your contracts directory.
  2. the contracts to be mutated pass all the tests. If at least one of the tests fails, the mutation testing process cannot be run.

To exclude specific contracts from the mutation process, you must specify the contract name in the sumo\src\operator.config.json file.

Before starting the mutation testing, you can choose which mutation operators must be applied:

  • sumo list shows the currently enabled mutation operators
  • sumo enable enables all the mutation operators
  • sumo enable ID enables the mutation operator ID
  • sumo disable disables all the mutation operators
  • sumo disable ID disables the mutation operator ID

Once everything is set up, you can use:

  • sumo preflight To view all the available mutations
  • sumo test To launch the mutation testing process

________________________________________________________________________________________________

Results

SuMo automatically creates a .sumo folder in the root directory of your project.
At the end of the mutation testing process the folder will contain:

  • report.txt Test report
  • \alive Mutants that survived testing
  • \killed Mutants killed by tests

________________________________________________________________________________________________

Traditional Operators

OperatorDescription
ACMArgument Change of overloaded Method call
AORAssignment Operator Replacement
BCRDBreak and Continue Replacement and Deletion
BLRBoolean Literal Replacement
BORBinary Operator Insertion
CBDCatch Block Deletion
CSCConditional Statement Change
EREnum Replacemet
ECSExplicit Conversion to Smaller type
HLRHexadecimal Literal Replacement
ICMIncrements Mirror
ILRInteger Literal Replacement
LCSLoop Statement Change
OLFDOverloaded Function Deletion
ORFDOverridden Function Deletion
SKISuper Keyword Insertion
SKDSuper Keyword Deletion
SLRString Literal Replacement
UORDUnary Operator Replacement and Deletion

Solidity Operators

OperatorDescription
AVRAddress Value Replacement
CSCContract Constructor Deletion
DLRData Location Keyword Replacement
DODDelete Operator Deletion
ETREther Transfer function Replacement
EEDEvent Emission Deletion
EHCException Handling Change
FVRFunction Visibility Replacement
GVRGlobal Variable Replacement
MCRMathematical and Cryptographic function Replacement
MODModifier Deletion
MOIModifier Insertion
MOCModifier Order Change
MOCModifier Order Change
MORModifier Replacement
PKDPayable Keyword Deletion
RSDReturn Statement Deletion
RVSReturn Values Swap
SFDSelfdestruct Deletion
SFISelfdestruct Insertion
SFRSafeMath Function Replacement
SCECSwitch Call Expression Casting
TORTransaction Origin Replacement
VURVariable Unit Replacement
VVRVariable Visibility Replacement