Certora compared three Soroban security tools—Sunbeam (formal verifier), Komet (hybrid fuzzing/proving), and cargo-fuzz—on detecting bugs and proving correctness. Fuzzing found obvious failures quickly but missed rare edge cases. Sunbeam detected all bugs and proved correct code was mathematically sound.
Certora ran a controlled experiment comparing three security tools on Soroban smart contracts: cargo-fuzz (Rust fuzzing), Komet (Runtime Verification's hybrid tool), and Sunbeam (Certora's formal verifier). They tested each on a simple adder contract with a commutativity property across five variants—some correct, others containing bugs at specific inputs. Fuzzing tools quickly found obvious failures (returning 0 for inputs like 0 or 100) but missed a bug triggered by one input out of 4 billion. Sunbeam found all bugs and proved the correct variants were mathematically sound for all inputs. The key takeaway: fuzzing provides fast feedback but offers no correctness guarantees. Formal verification provides mathematical proof that security properties hold universally.
