Ossification (announcement and call for feedback)

Ossification

in which we attempt to quantify an ancient blockchain security primitive of permissionless, immutable smart contract code

We are working on a new L2BEAT metric: the Ossification Score.

The frontend and backend machinery is still being assembled, while the methodology and dataset already exist. So instead of waiting until the feature is shiny and emotionally harder for us to change, we are posting it here first.

Please attack it.

Dataset: https://sekuba.github.io/ossification-dataset/
Repo: https://github.com/sekuba/ossification-dataset

Old code gud?

There is some evidence from traditional software that old, unchanged code tends to be less vulnerable, while fresh and heavily modified code tends to contain more bugs (1, 2, 3, 4).

This seems especially interesting for smart contracts: they are public, attackers can stare at them early and indefinitely, and successful staring can occasionally produce several hundred million dollars. So we define code ossification as the accumulation of evidence that code has matured by successfully securing value over time (not automatically: old code = secure).

The :bone:-clock

L2BEAT already monitors project contracts and their state with our discovery system. For ossification, we define a perimeter of critical contracts and configuration. Every time something critical changes, the project’s :bone:-clock resets.

code age = time since the last critical change

We did not want to declare that 2 years = 83 score because it felt spiritually correct, so we collected exploits instead.

The score

We started, together with a robot swarm, an open dataset of EVM blockchain exploits and reviewed, among other things:

  • what was exploited,
  • how old the vulnerable code/configuration was,
  • how much money was lost.

That gives us an empirical distribution of code age at exploit time. The Ossification Score is derived from this distribution. For example:

A score of 67 means that 67% of exploits above $1k loss targeted younger code.

You can play with the current dataset and thresholds here:

:backhand_index_pointing_right: https://sekuba.github.io/ossification-dataset/

But there is $10 in this contract

Three-year-old code securing $50k has survived a somewhat different adversarial environment than three-year-old code securing $10B. So we add another metric: battle-tested exposure.

This is the integral of TVS since the last critical change, measured in dollar-years. If a contract secures $1B unchanged for one year, that is roughly $1B-year of exposure. Very advanced physics. A project like Arbitrum or Base can have relatively young code while already having accumulated enormous economic exposure.

We currently prefer showing these things next to each other rather than mathing them into One Final Security Number:

  • code age,
  • ossification score,
  • battle-tested exposure.

Ossification = security confirmed?

No, the metric does not tell you:

  • who can upgrade the contracts (check L2BEAT),
  • whether users have an exit window (check L2BEAT),
  • whether the code was audited (working on that!),
  • how large the last code change was (working on that!),
  • whether someone will lose the multisig keys in a boating accident,
  • basically anything outside the onchain code/configuration.

Why?

Very few blockchain systems are actually ossified today. That means there is something to incentivize.

Just as Stages make decentralization progress visible, we would like ossification to make another property visible: building systems that eventually become simple, robust, permissionless and boring enough that nobody needs to keep upgrading them forever.

Contribute

Before we ship, we are especially interested in attacks on:

  • the dataset: missing exploits? bad classifications?
  • the clock: what exactly should reset it?
  • the perimeter: what counts as critical code/configuration?
  • the score: is this percentile-style construction actually useful?
  • battle-tested exposure: what do you think about dollar-years?
  • obvious ways people will misinterpret this: find them before CT does.

Dataset again: https://sekuba.github.io/ossification-dataset/
Repo (open for PRs/issues): https://github.com/sekuba/ossification-dataset

5 Likes

here’s a preview of what the ossification summary page could look like

I like the idea and the thesis makes sense to me. Have you considered treating contract storage changes (config update) like code changes?

1 Like

yes, we already do! our disco update monitor can watch storage as well as public view funcs and both can be labeled as critical changes. some examples are pointers to verifier contracts, program hashes, quorum fraction/threshold changes,…