Namespaced Storage Layout

Abstract

We define the NatSpec annotation @custom:storage-location to document storage namespaces and their location in storage in Solidity or Vyper source code. Additionally, we define a formula to derive a location from an arbitrary identifier. The formula is chosen to be safe against collisions with the storage layouts used by Solidity and Vyper.

Motivation

Smart contract languages such as Solidity and Vyper rely on tree-shaped storage layout. This tree starts at slot 0 and is composed of sequential chunks for consecutive variables. Hashes are used to ensure the chunks containing values of mappings and dynamic arrays do not collide. This is sufficient for most contracts. However, it presents a challenge for various design patterns used in smart contract development. One example is a modular design where using DELEGATECALL a contract executes code from multiple contracts, all of which share the same storage space, and which have to carefully coordinate on how to use it. Another example is upgradeable contracts, where it can be difficult to add state variables in an upgrade given that they may affect the assigned storage location for the preexisting variables.

Rather than using this default storage layout, these patterns can benefit from laying out state variables across the storage space, usually at pseudorandom locations obtained by hashing. Each value may be placed in an entirely different location, but more frequently values that are used together are put in a Solidity struct and co-located in storage. These pseudorandom locations can be the root of new storage trees that follow the same rules as the default one. Provided that this pseudorandom root is constructed so that it is not part of the default tree, this should result in the definition of independent spaces that do not collide with one another or with the default one.

These storage usage patterns are invisible to the Solidity and Vyper compilers because they are not represented as Solidity state variables. Smart contract tools like static analyzers or blockchain explorers often need to know the storage location of contract data. Standardizing the location for storage layouts will allow these tools to correctly interpret contracts where these design patterns are used.

Specification

Preliminaries

A namespace consists of a set of ordered variables, some of which may be dynamic arrays or mappings, with its values laid out following the same rules as the default storage layout but rooted in some location that is not necessarily slot 0. A contract using namespaces to organize storage is said to use namespaced storage.

A namespace id is a string that identifies a namespace in a contract. It should not contain any whitespace characters.

@custom:storage-location

A namespace in a contract should be implemented as a struct type. These structs should be annotated with the NatSpec tag @custom:storage-location <FORMULA_ID>:<NAMESPACE_ID>, where <FORMULA_ID> identifies a formula used to compute the storage location where the namespace is rooted, based on the namespace id. (Note: The Solidity compiler includes this annotation in the AST since v0.8.20, so this is recommended as the minimum compiler version when using this pattern.) Structs with this annotation found outside of contracts are not considered to be namespaces for any contract in the source code.

Formula

The formula identified by erc7201 is defined as erc7201(id: string) = keccak256(keccak256(id) - 1) & ~0xff. In Solidity, this corresponds to the expression keccak256(abi.encode(uint256(keccak256(bytes(id))) - 1)) & ~bytes32(uint256(0xff)). When using this formula the annotation becomes @custom:storage-location erc7201:<NAMESPACE_ID>. For example, @custom:storage-location erc7201:foobar annotates a namespace with id "foobar" rooted at erc7201("foobar").

Future EIPs may define new formulas with unique formula identifiers. It is recommended to follow the convention set in this EIP and use an identifier of the format erc1234.

Rationale

The tree-shaped storage layout used by Solidity and Vyper follows the following grammar (with root=0):

$L_{root} := \mathit{root} \mid L_{root} + n \mid \texttt{keccak256}(L_{root}) \mid \texttt{keccak256}(H(k) \oplus L_{root}) \mid \texttt{keccak256}(L_{root} \oplus H(k))$

A requirement for the root is that it shouldn't overlap with any storage location that would be part of the standard storage tree used by Solidity and Vyper (root = 0), nor should it be part of the storage tree derived from any other namespace (another root). This is so that multiple namespaces may be used alongside each other and alongside the standard storage layout, either deliberately or accidentally, without colliding. The term keccak256(id) - 1 in the formula is chosen as a location that is unused by Solidity, but this is not used as the final location because namespaces can be larger than 1 slot and would extend into keccak256(id) + n, which is potentially used by Solidity. A second hash is added to prevent this and guarantee that namespaces are completely disjoint from standard storage, assuming keccak256 collision resistance and that arrays are not unreasonably large.

Additionally, namespace locations are aligned to 256 as a potential optimization, in anticipation of gas schedule changes after the Verkle state tree migration, which may cause groups of 256 storage slots to become warm all at once.

Naming

This pattern has sometimes been referred to as "diamond storage". This causes it to be conflated with the "diamond proxy pattern", even though they can be used independently of each other. This EIP has chosen to use a different name to clearly differentiate it from the proxy pattern.

Backwards Compatibility

No backward compatibility issues found.

Reference Implementation

pragma solidity ^0.8.20;

contract Example {
    /// @custom:storage-location erc7201:example.main
    struct MainStorage {
        uint256 x;
        uint256 y;
    }

    // keccak256(abi.encode(uint256(keccak256("example.main")) - 1)) & ~bytes32(uint256(0xff));
    bytes32 private constant MAIN_STORAGE_LOCATION =
        0x183a6125c38840424c4a85fa12bab2ab606c4b6d0e7cc73c0c06ba5300eab500;

    function _getMainStorage() private pure returns (MainStorage storage $) {
        assembly {
            $.slot := MAIN_STORAGE_LOCATION
        }
    }

    function _getXTimesY() internal view returns (uint256) {
        MainStorage storage $ = _getMainStorage();
        return $.x * $.y;
    }
}

Security Considerations

Namespaces should avoid collisions with other namespaces or with standard Solidity or Vyper storage layout. The formula defined in this ERC guarantees this property for arbitrary namespace ids under the assumption of keccak256 collision resistance, as discussed in Rationale.

@custom:storage-location is a NatSpec annotation that current compilers don't enforce any rules for or ascribe any meaning to. The contract developer is responsible for implementing the pattern and using the namespace as claimed in the annotation.

Copyright

Copyright and related rights waived via CC0.