Supporting the most users of any network in Web3 requires a scalable blockchain that is capable of having both high data availability and low-cost data posting in order for applications to scale to the level of mainstream global adoption. This post will explain NEAR’s technical approach to data availability and how NEAR DA, a solution designed for Ethereum rollups, will scale the modular ecosystem and defragment all of Web3.

The modular Ethereum ecosystem seeks to scale the network by separating the functions of a blockchain––settlement, execution, and data availability––into separate layers. The Ethereum layer-one simply could not scale to accommodate the level of demand it reached, particularly during bull markets, without prohibitively high transaction fees. This demand becomes even more problematic when users want to do swaps on Ethereum or post transaction data for layer-twos to the layer-one network. 

NEAR DA offers a solution to this scalability roadblock. With a simple architecture that is easy to implement and constantly improving efficiency, and thanks to NEAR having the cheapest transaction fees of any network in Web3, NEAR DA is an incredibly fast and cost-effective data availability option. Depending on the gas fees, NEAR DA is up to 85,000x cheaper than posting blob submissions on Ethereum and 30x cheaper than doing the same on Celestia. 

NEAR DA and the Chain Abstraction Vision

The NEAR ecosystem’s focus for 2024 is expanding chain abstraction across more chains & ecosystems, with the goal of abstracting the blockchain away from the user & developer experience. While the modular Ethereum ecosystem has made Ethereum more scalable in some ways, it’s also resulted in a lot of incompatible infrastructure and fractured liquidity that places a huge burden on app builders and end users to manage. The chain abstraction vision is that apps should just work with all chains, onboarding should be seamless, and things like gas fees and bridging should become invisible to end users. 

Data availability and NEAR DA play an important role in chain abstraction. As zero-knowledge technology enables unification of security across chains via state proofs, as described in the chain abstraction deep dive post linked above, settlement data will be easier to fetch from different networks. As dapps become truly multichain thanks to account aggregation and decentralized frontends, they can query data from multiple chains more easily when data is all posted in one place. Doing so on NEAR will be much cheaper for rollups.

How is NEAR DA Implemented?

NEAR DA leverages an important part of NEAR’s consensus mechanism, known as Nightshade, which parallelizes the network into multiple shards (essentially, multiple parallel blockchains). Each shard on NEAR produces a small portion of a block, which is called a chunk. These chunks are aggregated to produce blocks. All of this happens entirely at the protocol level and so is invisible to users and developers.

NEAR DA uses this infrastructure to an ETH rollup’s benefit. When a chunk producer processes a receipt, there is consensus around the receipt. However, once the chunk has been processed and included in the block, the receipt is no longer required for consensus and can be pruned from the blockchain’s state. The pruning time is at least 3 NEAR epochs, where each epoch is 12 hours. In practice, this is usually around 5 NEAR epochs, so data is available in the network for around 60 hours. Once the receipt is pruned, it is the responsibility of archival nodes to retain the transaction data. 

This means that NEAR doesn’t slow down its consensus with more data than it requires, yet any user of NEAR DA would have ample time to query transaction data. The advantage this architecture provides to rollups is cost-effective data availability, especially to those with high transaction volume, such as gaming chains. 

NEAR DA provides an out-of-the-box solution for developers, making it easy for rollup builders to make the most of NEAR’s infrastructure––which has 3+ years of proven reliability behind it. The Pagoda engineering team has already built three important components that are open-source and ready to be integrated into any OP Stack, Polygon CDK, or Arbitrum Nitro rollups:

• The Blob Store Contract: A contract that stores arbitrary data.
• Light Client: A trustless off-chain client that provides easy access to transaction and receipt data.
• RPC Client: A client to publish data blobs to the NEAR Protocol.

Rollup builders looking to work with a cloud-like service could also leverage NEAR DA’s rollup-as-a-service (RaaS) providers (list provided here), in order to utilize these three components all in one place in a more abstracted manner. 

What’s next? The Future of NEAR DA and Data Availability Sharding

Another advantage NEAR’s sharding approach offers to users of NEAR DA is the future scaling roadmap. The NEAR Protocol Engineering team recently announced the move towards stateless validation, the next phase of sharding. This will further decrease the hardware requirements of certain types of validators (chunk validators) and move the state into memory, which will allow for more shards and increase the decentralization in the system by lowering the requirements to become a validator in the system. 

This means that the number of shards will increase, greatly increasing the overall throughput of the NEAR Protocol. While NEAR is already fast, with 4 MB/s per shard with 4 shards, scaling towards n shards means that rollups, applications, and any other project or developer using NEAR will not have to compete for blockspace. This is a major contrast to the scaling limits of both monolithic systems as well as the modular ETH ecosystem and offers a substantial future-proof for anyone building on NEAR, including users of NEAR DA.

As the NEAR Protocol starts increasing the number of shards to n, the amount of data that has to be stored for a single shard decreases. Eventually, in theory every account on NEAR Protocol could potentially become its own shard. This allows running very light RPC nodes that would only track the shard that a given user is interested in. 

For L2s that use NEAR DA, this would mean running an in-memory RPC client that would only track the shard that your data blob contract is in for your retention period (for fraud proofs), or for a given interval of time (retention is configurable on the RPC nodes). Instead of relying on data availability sampling (DAS), a rollup can rely on its own shard by running a lightweight RPC through Data Availability with Sharding. The additional benefit here is that the RPC is also applying the chunks they are tracking, essentially acting as a validator without participating in the consensus, as opposed to statistical confidence with DAS. 

While Data Availability Sharding remains in the research phase of development, it shows the major advantages the NEAR Protocol design offers to many kinds of builders and ecosystems. As the Web3 space pursues mainstream adoption, the scalability hurdles of the first ten years of Ethereum will seem minor, and it is not clear that modularity alone will solve the problem. The NEAR ecosystem’s goal of mainstream adoption of Web3 means the network is available for all builders to use for creating great applications and delivering the next generation of the internet.

Fast, Cost-Effective DA is NEAR

As Web3 adoption grows and as the number of rollups continues to climb, the need to scale the modular scalability solutions becomes greater. NEAR offers its innovative, reliable technology as a data availability solution for rollups that can save time and money while future-proofing projects for an increasingly multichain (and cross-chain) future. The bleeding-edge research driving the future development of NEAR DA will provide rollups with even more advantages over time. Interconnecting and defragmenting Web3 via chain abstraction will be a major theme of 2024 and utilizing NEAR DA puts rollups at the forefront of this evolution.

The post Why NEAR Data Availability? appeared first on NEAR Protocol.

NEAR DA is set to launch with a number of first users, including Madara by StarkNet, Caldera, Fluent, Vistara, Dymension RollApps, and Movement Labs. 

Let’s take a brief look at NEAR DA to explore why it’s such an exciting and significant development for NEAR, Ethereum, and the larger effort to build a truly open web. 

NEAR DA — paving the way for modular blockchain development

With NEAR DA, ETH rollups and Ethereum developers get cost-effective, reliable data availability. 100kB of calldata on NEAR will cost $0.0033 as of September 2023, while the same calldata on Ethereum L1 will run users  $26.22. That’s 8,000 times cheaper! 

NEAR DA will help developers reduce costs and enhance the reliability of their rollups, while keeping the security of Ethereum. High quality projects launching an app-chain or L2 will also be able to get NEAR DA support. 

Illia Polosukhin, co-founder of NEAR Protocol, and newly announced CEO of NEAR Foundation said: “Offering a data availability layer to Ethereum rollups highlights the versatility of NEAR’s tech while also helping founders from across Web3 deliver great products that bring us closer to mainstream adoption of the Open Web.”

“NEAR’s L1 has been live with 100% uptime for more than three years, so it can offer true reliability to projects looking for secure DA while also being cost-effective,” Polosukhin added. “NEAR provides great solutions to developers no matter which stack they’re building on and now that includes the Ethereum modular blockchain landscape.”

Expanding NEAR’s Open Web Stack 

By opening up its blockchain for data availability, NEAR Protocol is expanding its offerings to deliver a modular blockchain development framework. This will let Web3 developers and founders continue building on Ethereum while leveraging NEAR’s state-of-the-art tech for one part of the stack, depending on their requirements. 

NEAR DA expands the capabilities of NEAR’s Open Web Stack, a common entry point where developers and users alike can easily build, browse, and discover Web3 products and platforms. This includes FastAuth onboarding to an account in seconds with zero crypto or seed phrases required, as well as the ability to create decentralized frontends from a library of thousands of components. 

Developers and founders building on Ethereum rollups interested in using NEAR DA can visit near.org/data-availability to get started.

The post NEAR Foundation Launches NEAR DA to Offer Secure, Cost-Effective Data Availability for ETH Rollups and Ethereum Developers  appeared first on NEAR Protocol.