The post Legends of NEAR: Expansion from 4 to 6 Shards appeared first on NEAR Protocol.
]]>Over the last six months, NEAR has experienced a significant increase in users and transactions. NEAR has 3 of the top ten dapps in Web3 by users with KAI-CHING and HOT Wallet both reaching millions of daily active users in April 2024.
Chart from Pikespeak
NEAR has 3 of the top ten dapps in Web3 by users according to dappradar with KAI-CHING and HOT Wallet both reaching millions of daily active users in April 2024.
All of this demand led to some congestion on one of the shards, so Bowen Wang, Director of Protocol, and the Protocol team worked on further optimizations that rolled out within days to address it. Over the course of 2 weeks, they added 2 more shards to NEAR – for a total of 6 live shards. This increased the network’s capacity by 50%.
At the end of Q2, stateless validation is expected to launch on Mainnet – this upgrade is projected to improve throughput for each shard by 5x and grow the total number of shards available for the network.
One of the most remarkable aspects of this upgrade is that it was implemented live—there was no need to take the blockchain offline. There was no break in user transactions.
What is sharding on a blockchain? Let’s simplify using an analogy.
Imagine you have a library that’s become too large and crowded. To manage this, you divide your books into several smaller sections and place them in different rooms. Each room now holds part of the entire collection, making finding and managing books easier. Most importantly, the separate collections will minimize congestion when the library is busy.
Now, think of the NEAR blockchain like that crowded library. Sharding is like dividing this big blockchain into smaller, manageable pieces called “shards.” Each shard is like one of those rooms, with only the contract interactions within that shard. When someone needs information, they go directly to the room (shard) where it’s stored instead of waiting for the whole chain. This way, sharding helps handle more data and serve more users efficiently without affecting users of the entire chain every time.
In NEAR’s case, the blockchain is split into multiple shards, akin to these rooms, where each shard processes transactions and maintains the state independently. This division allows for parallel processing, significantly boosting the network’s capacity and speed.
What sets NEAR apart is how these shards interact and synchronize. A developer doesn’t have to worry about which shard a specific contract might be on. This approach maintains high security and interconnectivity while allowing the network to scale effectively. As the network grows and demands increase, more shards can be added, like expanding the library with more rooms, to ensure that NEAR can adapt to increasing usage and more complex applications.
NEAR is focusing on future-proofing the protocol to handle even more growth and usage. This proactive approach aims to minimize disruptions and maintain a consistent developer and user experience, no matter how much the demand grows.
This further validates NEAR’s thesis that sharding is the best scaling approach given what is currently happening across the whole Web3 space. Many non-sharded chain/rollups are getting congested with transactions and have been forced to raise fees, while NEAR was able to expand capacity by adding more shards and keeping fees stable.
Homogeneous Sharding isn’t just a tech tweak; it’s a critical evolution in the blockchain that keeps transactions flowing. And it takes a team of superheroes for NEAR Protocol to be a top blockchain in Web3. The ultimate goal? To ensure that users never notice the complexity behind their seamless digital experience.
Claim your Proof of Legends of NEAR NFT
The post Legends of NEAR: Expansion from 4 to 6 Shards appeared first on NEAR Protocol.
]]>The post Phase 2 of NEAR Sharding Launches appeared first on NEAR Protocol.
]]>NEAR’s vision of chain abstraction to facilitate broad adoption of decentralized apps requires an extremely scalable blockchain layer. From the start, NEAR has been designed to scale with demand towards mainstream adoption. A primary benefit of the new sharding implementation will be an up-to-10x speed improvement to NEAR’s already-fast transaction throughput. Sharding is NEAR’s unique approach to scaling, which partitions the blockchain into multiple parallel “shards.” Combined with NEAR already having the lowest transaction fees in Web3, this sharding upgrade positions NEAR at the forefront of Web3 in terms of both performance and scalability. Phase 2 greatly improves the network’s capacity for end user volume and is an important landmark on the road to global-scale usage of Web3.
“NEAR continues to make progress on bringing Chain Abstraction infrastructure that can scale to a billion users and beyond,” said Illia Polosukhin, Co-Founder of NEAR Protocol and CEO of NEAR Foundation. “The scalability improvements from stateless validation can unlock even better user experiences for more end-user applications, whether multichain DeFi-style dapps or those aimed at mainstream users.”
Phase 2 introduces the most significant changes to the NEAR Protocol since its Mainnet launch in 2020. The biggest of these is the implementation of stateless validation: an innovative approach to state change, or the process of updating the status of all the data posted to the blockchain. Now, NEAR validators no longer have to maintain the state of a shard locally and can retrieve all the information they need to validate state changes, or “state witness,” from the network.
With stateless validation, NEAR can finally achieve the truest form of sharding, where shards can function mostly independently at the consensus level to improve decentralization and throughput while preserving the highest security guarantees. An added benefit is that the hardware requirements to run most validator nodes are dramatically reduced (a smaller number of “chunk proposer” validators with specialized hardware will validate blocks with state held in memory). This also paves the way for greater decentralization of the network by lowering the barrier to entry to become a validator.
“We’re very excited about the potential for future proofing the NEAR protocol design with stateless validation,” said Bowen Wang, Director of Protocol at Pagoda. “From a research perspective, we expect that as zero-knowledge tech matures, more protocols will adopt a similar approach, where a smaller set of expensive machines execute transactions and produce proofs, while a bigger validator set validates the proofs. This will enable more unified security across networks and defragment Web3, advancing a key aspect of the chain abstraction vision.”
Phase 2 is a shift in direction from the original Nightshade sharding architecture NEAR launched with in 2020, bypassing some fundamental roadblocks in that protocol design around the implementation of challenges in the initial idea of Phase 2, while also unlocking additional benefits. Other network improvements in Phase 2 include in-memory trie, wherein validator nodes can load the entire state into RAM for maximized performance. This will greatly improve transaction throughput thanks to minimizing storage access.
NEAR users should not experience any downtime with the transition to Phase 2 and no special action is required from validators, apart from adjusting hardware specs. Through dozens of protocol upgrades since Mainnet launch, including three major upgrades, the core NEAR protocol has had 100% uptime with zero disruption for developers and end users.
On Thursday, February 1, Stake Wars IV: Attack of the Transactions will launch to incentivize battle testing of the new sharding architecture. Planned to run through March 31, the latest edition of Stake Wars invites community members to test features and generate traffic. For more information about participating and deeper technical detail about Phase 2, visit the Stake Wars page on Github.
With the launch of Phase 2, NEAR becomes one of the only networks in Web3 to implement stateless validation to improve scaling and paves the way for further network performance and scalability improvements in the future roadmap. A technical paper with more detail about NEAR’s new sharding design is forthcoming later in Q1. Phase 2 is expected to launch on NEAR Mainnet in May 2024.
The post Phase 2 of NEAR Sharding Launches appeared first on NEAR Protocol.
]]>The post NEAR Q4 Protocol Roadmap Update appeared first on NEAR Protocol.
]]>TLDR: 2023 saw major experience improvements via meta transactions and zero-balance accounts, as well as node performance improvements and optimizations. Preparations are underway for a major upgrade to stateless validation, completing Phase 2 of the sharding roadmap in early 2024.
There were a total of six protocol upgrades in 2023 which introduced a number of new protocol features. Meta transactions were added as a protocol feature at the beginning of the year to support gasless transactions, which means that users can transact on NEAR and start using apps without necessarily needing to pay transaction fees in NEAR. We also added zero-balance accounts, an important feature that enables users to create an account without holding NEAR to pay for storage. This makes it easy to onboard new users to applications. These two features form a solid foundation for a seamless onboarding experience for end users — a top goal for NEAR overall.
In addition to easy user onboarding, we have also been working hard to optimize the performance of NEAR. Flat storage, which was released in Q2 this year, optimized state reads and improved the stability and performance of the network overall. Another feature that optimizes NEAR nodes is cold storage, which allows for a split between hot and cold storage for a node. This means an archival node does not need to store most of the historical data on SSD and therefore can save a lot on cost. We also optimized the network communication between validator nodes by introducing the tier1 network, which reduces the latency of network messages between two validator nodes. Furthermore, we improved state sync and reduced the time it took a node to synchronize the latest state. We have also worked on improving the stability and maintainability of contract runtime with initiatives like finite wasm and limited replayability.
There are a few major initiatives that we started this year and expect to finish next year, such as stateless validation, congestion control, and transaction priority. We will dive deeper into those in the section below.
There are a few major projects that we want to deliver in 2024 to improve the usability, scalability, and decentralization of NEAR Protocol:
The roadmap can be seen below for those interested in more timeline specifics. It is split into two parts: Experience and Core. The Experience section encompasses user and/or developer experience and the protocol features needed to enable those experiences. As an example, synchronous execution addresses the pain point that developers feel when they implement cross-contract calls.
The Core section, on the other hand, covers major efforts to improve the scalability and decentralization of the protocol. This includes stateless validation, zkWASM, improvements to data availability, and so on.
As always, the protocol team is proud to have made several important improvements to the core protocol with zero disruption for users and developers. We look forward to making some major advancements in 2024 to help the entire NEAR ecosystem deliver its vision of mainstream adoption of an Open Web, where all internet users can control their own data, assets, and power of governance.
We will share more details on Phase 2 advancements and timelines soon. If you’d like to learn more, join the NEAR Protocol X Spaces on Tuesday, January 16 at 5pm UTC to hear from Director of Protocol Bowen Wang and NEAR Foundation CEO Illia Polosukhin on upcoming protocol developments.
The post NEAR Q4 Protocol Roadmap Update appeared first on NEAR Protocol.
]]>The post NEAR Strategic Update and Outlook for 2023 appeared first on NEAR Protocol.
]]>Yet Web3 also hasn’t fully delivered on its promise to open the web. While important progress has been made, much more needs to be done for this technology to deliver fairer, freer digital systems. Today’s Web3 suffers from inaccessible user experience, siloed applications which are difficult to discover, few real-world use cases, and technology tribalism. Put simply, too much hype and not enough value for users.
Since 2018, NEAR has focused on building scalable, secure technology and infrastructure that is easy to use and enables developers to freely create usable applications. Since launching Mainnet in late 2020, the NEAR ecosystem has made incredible progress, becoming one of the top layer-one blockchains with 22 million accounts, 600K monthly active wallets (active defined as 2+ transactions), and 15x growth in users and projects just in the last year. And despite chaos in the market and a major downturn in recent months, NEAR Foundation has sufficient capital to sustain at least five years of operations in bear market conditions thanks to responsible treasury management.
While Web3 has yet to deliver on its promises, NEAR Foundation believes that NEAR is the only ecosystem where Web3 promises can be delivered and where mainstream adoption can happen. All the fundamentals have been established for some time and NEAR is charging full-speed ahead into its next growth phase in 2023. This post will cover NEAR’s convictions heading into next year, the strategic approach to drive our next phase, and a look ahead at new areas of product and development.
Enabling NEAR to make the most of these convictions in today’s climate means committing to the following goals over the next year.
Growing usage on NEAR will begin with a top-down approach focused on partnerships. The world-class business development team at the Foundation will focus on working with major applications and brands with substantial, established communities to partner with NEAR on real use cases that drive engagement, such as ticketing and earning. High-traction focus verticals include sports, entertainment, and loyalty. Major partnerships in recent months include Grupo Nutresa, Google Cloud, and Sweat Economy.
Any great Web3 ecosystem is an emergent product of a dedicated community of believers. The bottom-up, grassroots approach for the next year on NEAR will empower the community to invest in its own expansion through grants, primarily through 3 major community DAOs: Developer DAO, Marketing DAO, and Creatives DAO (visit the DAO websites to get involved). This means the NEAR Foundation will no longer directly allocate capital to projects, instead supporting the community in these decisions and further decentralizing key elements of the ecosystem. These grassroots DAOs will be supported by the NDC, which has achieved initial traction, including launching a new Governance Working Group and plans to roll out a v1 governance infrastructure for the ecosystem early next year.
In the new year, the NEAR Foundation will form an early-stage accelerator that will provide support to promising projects and founders on NEAR in areas including education, tech, hiring, legal, UX, and GTM guidance, setting them up to grow into the landmark Web3 projects of tomorrow.
With NEAR’s protocol, infrastructure, and developer tooling already well established, the next step in providing the best user experience to Web3 will be at the discovery layer: the connective tissue that connects applications, tooling, social, earning, and developer components across the open web. This stack is what we call a “blockchain operating system.”
Many of the building blocks of the blockchain operating system already exist on NEAR, either as features or apps. These include NEAR Crowd, a gig economy platform with 50% of users transacting every day, and NEAR Social, an on-chain social network with a widget framework that lets developers fork entire app frontends to build new experiences. Combined with upcoming improvements to onboarding, including from EVM wallets (which we’re calling remote accounts), native meta transactions, Keypom, and new chat/social capabilities, NEAR will deliver even more amazing value directly to developers and users.
All of these user-centric elements further up the stack, particularly at the app layer, will form the basis for a core component of the NEAR ecosystem’s product and technical focus in 2023. Pagoda, NEAR Foundation, and other key ecosystem participants will work together to deliver the best possible technology with an increasingly integrated and seamless experience for developers and end users. The goal from here is evolving NEAR to be more than just a layer-one blockchain: it will become a global blockchain operating system. It’s still early days, but building is underway and the vision is clear.
NEAR is the place where mainstream adoption of the open web will happen. Soon, anyone in the world—app users, developers, founders, creators—will be able to create without limits.
Thank you for reading our NEAR 2023 Strategy Blog Post and/or participating in the NEAR Go Forward Strategy Twitter AMA! If you have questions or comments we would love to hear them!
The post NEAR Strategic Update and Outlook for 2023 appeared first on NEAR Protocol.
]]>The post NEAR’s Road to Decentralization: A Deep Dive into Aurora appeared first on NEAR Protocol.
]]>In the web3 world, this idea can be neatly summed up by one word: decentralization. This is a process whereby control and the tools to create and reimagine everything from business to creativity are progressively handed over to the contributors.
The NEAR Foundation has been steadily progressing in this mission, from the growing number of validators to the flourishing DAO community. But NEAR isn’t alone on this journey. It relies on projects in and around the ecosystem to help accelerate this mission. Enter Aurora.
This project, started originally by a core team of NEAR developers, is now helping build bridges to other ecosystems and accelerating the adoption and decentralization in equal measure.
Built by NEAR Inc’s core team (now Pagoda), Aurora is an Ethereum Virtual Machine (EVM) smart contract platform that creates a bridge between Ethereum and NEAR. An EVM, for those who don’t know, is best thought of as a decentralized computer that allows anyone to create a smart contract on the Ethereum network.
Although an EVM, Aurora works on top of the NEAR blockchain, giving it all of the benefits of the NEAR blockchain: super fast, incredibly secure, and infinitely scalable transactions. That’s possible thanks to Nightshade, NEAR’s unique sharded protocol design that allows the network to process thousands of transactions per second without skipping a beat.
With Aurora, the idea is that anyone building Ethereum projects can make use of the NEAR’s speed and low fees. The Aurora EVM allows developers to make use of NEAR’s blockchain, while the platform uses NEAR’s Rainbow Bridge to transfer assets.
Let’s look at Rainbow Bridge, as this protocol is where things got started.
NEAR’s core developer team wanted to create a smart contract that would perform the function of allowing tokens to flow freely between Ethereum, NEAR, and other projects. The team’s vision: create a tool that allowed assets to exist on both NEAR and Ethereum, creating a multi-chain Web3 user experience.
To do this, the team would need to create a bridge that would be completely decentralized: anyone could use it, anywhere, and at any time—without permission. The developers were able to build the bridge and now, if one wants to move an Ethereum-based token—say, a stablecoin like DAI—and use it on NEAR’s network, they can do so via the Rainbow Bridge.
Some of NEAR’s core team that had helped create Rainbow Bridge split off to continue their work. The result: Aurora—a project that now offers the bridge and a whole host of other features designed to create a global network of open-source Web3 projects.
Other projects working to improve cross-chain accessibility have since taken notice of Aurora’s work and become partners. One example is Allbridge, an application that unites scattered blockchains by means of global interoperability across all networks. Since Allbridge partnered with Aurora, it has launched a bridge between Aurora and Terra, an open-source stablecoin network and one of the biggest cryptocurrencies by market cap.
Aurora is also playing a big part in decentalized finance (DeFi) growth in the NEAR ecosystem. DeFi refers to the peer-to-peer financial services built on public blockchains, and is one of the most active crypto sectors on NEAR. The NEAR community has been working tirelessly to make it easier for DeFi apps and tools to leverage the protocol’s developer-friendly advantages.
One way that Aurora has made this process much smoother and more decentralized is through its extremely low fees (so low, they’re negligible), which it achieves by using the NEAR network. A constant gripe from users about Ethereum-based DeFi products is that the fees are often too high, creating a barrier to entry for developers and end users, and creating a hurdle to Web3’s mainstream adoption.
Aurora adoption is growing so much that the team released Aurorascan, its very own version of Etherscan, the most popular Ethereum block explorer and analytics platform. Aurorascan has all of Etherscan’s features set and reliability, while giving developers the tools and data to see how Aurora’s EVM functions.
But Aurora goes beyond DeFi. The platform is also integral to the NEAR community’s NFT projects, which have exploded in popularity since 2020. When the Aurora team spoke at this year’s ETHDenver conference in February, and they were joined by members of Endemic, Chronicle, and TENKBay—all NFT platforms that have launched on the Aurora EVM and NEAR Protocol.
So what does Aurora have up its sleeve for the future? Well, Aurorascan is still in beta mode, so more features will be added to that platform. And the team is also working to build new bridges, which will be announced in the coming months.
There are also plans to allow NFTs to be able to hop between Ethereum and NEAR. You can keep up to date on all things Aurora on its Medium page.
All this and more will help those wanting to take advantage of NEAR’s super fast and extremely cost-effective network, and help the dream of a truly decentralized world become a reality.
The post NEAR’s Road to Decentralization: A Deep Dive into Aurora appeared first on NEAR Protocol.
]]>The post An Update On the NEAR Validator Upgrade appeared first on NEAR Protocol.
]]>We received an overwhelming interest of more than 160 applications for delegation.
After the latest upgrade, which now allows up to 100 validators to the Mainnet and a lowered seat price of 64,000 $NEAR, many validators (73) managed to enter the validator pool with their own funds. The Foundation only had 27 seats available for delegation, which means that the selection process was extremely competitive.
As NEAR Protocol is certified carbon neutral by South Pole, we ask validators to think through how they offset their validator operation’s carbon footprint, and communicate any CO2 offsetting that they implement on their end, such as using a renewable energy source to operate the nodes. An important criteria in the selection for delegation was ecological, and validators who took steps to offset their carbon footprint were selected.
Additionally, please note that the minimum criteria to qualify for delegation, such as keeping commission to 7% and maintaining high uptime on testnet node or mainnet node, is essential and the Foundation reserves the right to withdraw delegation and re-delegate to another validator.
For any validators that were not selected in this round, there will be many more opportunities to be involved in the NEAR Ecosystem.
NEAR Protocol will introduce a new type of validator, Chunk-Only Producer, in mid-2022 to further decentralize NEAR and increase the number of validators on Mainnet from 100 to 300.
For those interested in being considered for a Chunk-Only Producer, please fill out this typeform.
For additional information about Chunk-Only Producers, please visit the Decentralize page. Additional details can be found on the NEAR protocol roadmap.
Another route to helping secure the network is via the Open Shards Alliance. The Open Shards Alliance (OSA) and the GuildNet network have been an ecosystem guild since the Mainnet launch. It’s been a home for validators, those interested in becoming one, and new devs that enter the ecosystem. Ideas are bootstrapped in the OSA and often move on into their own projects. The OSA is comprised of technical aficionados, sysadmins, and devs.
Please join the Open Shards Alliance server at the OSA Discord.
The post An Update On the NEAR Validator Upgrade appeared first on NEAR Protocol.
]]>The post NEAR Enhances Decentralization with Validator Upgrade appeared first on NEAR Protocol.
]]>That is why, in December 2021, the NEAR Protocol team released the updated validator selection algorithm. The new selection algorithm now allows for up to 100 validator nodes on the Mainnet, and reduces the validators’ barrier to entry by bringing down the seat price from 3.6 million $NEAR to 67,000. This lowered seat price makes it easier for them to join the active validator set and to maintain their seats.
Additionally, the NEAR Foundation aims to limit delegation amounts to a maximum of 100,000 NEAR per validator, as part of its continuous commitment to building a fully decentralized network.
Historically, the NEAR Foundation has used delegation to bootstrap network decentralization by helping new validators enter the active set and build their following but it wasn’t intended to be a long term grant. Current validators on NEAR Mainnet who had received historical delegation from NEAR Foundation should expect to see current delegations from NEAR Foundation to be withdrawn in the next 48 hours as part of this change.
This will be followed by the NEAR Foundation delegating up to 100,000 NEAR to validators who had been selected to receive delegation by submitting their delegation application up to December 13, 2021. This new wave of delegation will take place during the coming weeks.
This change is only the beginning of a number of ecosystem-wide initiatives that will continue increasing the number of validators and improving network decentralization. With the upcoming launch of phase 1 of sharding in early 2022, we expect another 200 validators to join Mainnet.
NEAR has always been, and will always be committed to building a secure, scalable, and decentralized network to help onboard the world into Web3. To take advantage of the upcoming expansion, there are other opportunities for NEAR Foundation delegation, including participating in the validator governance progress.
Please consider joining the NEAR Validator governance process by participating in the NEAR Validator Advisory Board general election in early 2022. Stay tuned for more announcements.
The post NEAR Enhances Decentralization with Validator Upgrade appeared first on NEAR Protocol.
]]>The post Zero to Hero NFT Series: Happy Holidays from DevRel! appeared first on NEAR Protocol.
]]>The DevRel team is simply amazed by our dev community, and we are thankful for the support and time every developer spends building projects on NEAR. So, as a humble way to say “Thank You”, we’ve prepared an NFT tutorial series so you can go from Zero to NFT Hero before the new year knocks on your door.
If you are new to Rust and want to dive into smart contract development, this series is a great place to start, especially if you want to learn about non-fungible tokens.
In this “NFT Zero to Hero” series, you’ll find a set of tutorials that will cover every aspect of a non-fungible token (NFT) smart contract. You’ll go from minting an NFT using a pre-deployed contract to building a fully-fledged NFT smart contract that supports every extension.
As a sneak peek, here’s a list of the NFT tutorials that will bring you from Zero to Hero in no time:
If you’re ready to start, jump into our Dev Docs and begin your Hero learning journey! If you already know about non-fungible tokens and smart contracts, feel free to skip and jump directly to the tutorial of your interest.
These tutorials have been designed so you can start at any given point, so you don’t need to start from zero if you’re already a Hero. 🙂
We hope you enjoy these holidays with your friends and family, and we wish you the best for 2022! May all our Web3 projects be successful, decentralized, and environmentally friendly, just like NEAR. 🙂
Join us on Discord and let us know in the #development channels.
We also host daily Office Hours live Monday – Friday 11 AM – 12 PM PT (6 PM – 7 PM UTC), where the DevRel team will answer any questions you may have!
The post Zero to Hero NFT Series: Happy Holidays from DevRel! appeared first on NEAR Protocol.
]]>The post NEAR Sharding Rollout and Aurora IDO Postmortems appeared first on NEAR Protocol.
]]>The following postmortems on the NEAR sharding rollout and Aurora IDO summarize the incidents. They provide root cause analyses, mitigations, and the future improvements we are working on to prevent similar incidents from happening again.
During the Boca Chica Aurora token sale, community members reported an inability to access the service to buy the Aurora lottery ticket. The cause: too much load on the RPC endpoint. Due to higher than expected demand during the sale, the RPC became overloaded in the EU region.
We’ve increased our RPC capacity by rolling out new RPC nodes across all regions. In particular, we’ve bumped up the number of nodes in the EU region. In the meantime, we’ve made and are working on several other improvements to increase the RPC throughput and reliability.
All Near Inc services stopped working to some extent for a short period of time including RPC, Indexers, Explorer, and Wallet. Issues were reported from across the community and internally all alerts started to fire.
We rapidly identified the root cause: an outage affecting the load balancing stack hit our cloud provider, which impacted RPC and therefore made all other dependent services unavailable as well. Unfortunately, there was no quick mitigation we could do on our side. We had to wait for the cloud provider to resolve the issue to get our services operational again. The cloud provider was able to quickly detect and restore service in under 20 minutes.
Currently, we use one cloud provider for the services we host. We do this for simplicity of maintenance. Due to the decentralized nature of NEAR, anyone is able to run their own RPC services, providing alternative ways to access the network. Making it easy for users to switch RPC nodes is something that both wallets and developer tooling will be looking into.
To prevent issues like this from happening to Near Inc services, we are also looking into implementing a multi-cloud deployment with client-side fallbacks in case the primary service goes down.
During the sharding rollout on mainnet we encountered two issues caused by state splitting as part of the protocol upgrade: 1) high disk usage affecting the RPC service, and 2) the inability of archival nodes to split state within one epoch.
Let’s discuss them separately.
The first issue was caused by high disk usage (IOPS) during the process of splitting state. Even though the RPC traffic was very low across the network, performance of our RPC nodes drastically deteriorated and we observed that RPC latency in some regions jumped from 1s to 60s.
This was not a capacity problem: each RPC request takes longer to respond and adding more nodes wouldn’t help much. Most services were almost unavailable, but due to client side retries they could be used with high response times. Most affected users were in Europe and Asia.
The second issue was caused by archival nodes being unable to finish state splitting within one epoch as expected and, therefore, they got stuck when the new epoch arrived. This issue arose unexpectedly, as we ran simulations beforehand for regular RPC for the sharding upgrade using mainnet data. The edge case we missed was running the simulation on archival nodes, which took significantly longer. The issue had not been previously identified on testnet, as archival data is smaller there.
The failure of archival nodes affected all services depending on them: Indexer, Explorer, Wallet, Aurora, etc. The Infrastructure team was able to rapidly redirect traffic to non-archival nodes as a patch until archival nodes were restored. We waited until archival nodes finished splitting their states, and once the first node synced we generated a backup and started the rest of the archival nodes from there. This was a failure of not testing for all cases and we plan to invest more time and effort in making sure releases go as smoothly as previous ones.
It’s important to note that, despite all the aforementioned incidents, the network was always functioning as expected and only RPC services were affected, causing dependent services to have issues.
We want to be transparent about all past and future issues we face. We believe the community members will understand that “NEAR stands for iteration” and we are doing our best to prevent such incidents from happening in the future.
The post NEAR Sharding Rollout and Aurora IDO Postmortems appeared first on NEAR Protocol.
]]>The post NEAR Foundation Launches New Validator Program To Further Decentralize the Network appeared first on NEAR Protocol.
]]>NEAR Foundation is happy to announce we’re making some changes to increase the number of validators who can take part in securing the NEAR network. With the introduction of a validator selection algorithm change, set to go live in mid-December, 2021, we expect the seat price to decrease significantly, allowing more capable validators to join the NEAR validator community.
To support the growth and decentralization of the NEAR ecosystem, and to expand the validator set with the upcoming 1.23.0 Mainnet protocol upgrade, the NEAR Foundation is pleased to announce the NEAR Foundation Delegation Program.
This delegation program is looking to support validators who have contributed to the ecosystem and are ready to join the NEAR Mainnet. To ensure that we select the best and most capable validators, we’re encouraging all existing validators and potential validators to apply.
NEAR Foundation will make the decision process for the Foundation Delegation as transparent as possible.
Technical Requirements: We expect the validator to run a NEAR testnet node and a NEAR mainnet node, demonstrating that they are ready and able to maintain nodes on the NEAR network. For mainnet readiness, having a mainnet node with a staking pool ready and standing by December 13th (the nearcore 1.23.0 protocol upgrade date) is a prerequisite for an applicant to receive NEAR Foundation delegation.
We expect all nodes to diligently participate during protocol upgrades. Each node should upgrade to the latest nearcore version within 12 hours of release if the protocol upgrade is labeled “CODE_RED” or “CODE_YELLOW”, and within 24 hours if the protocol upgrade is labeled “CODE_GREEN”.
Additionally, nodes should show a high uptime on testnet and on mainnet. A monitoring period is implemented to ensure validator nodes have stable operations.
Marketing Requirements: Beyond technical prowess and uptime, validators who could demonstrate exceptional customer service to delegators and provide the best educational content to delegators will be considered as prime candidates for receiving NEAR Foundation delegation.
On the mainnet, we ask that validators present themselves to delegators and the NEAR community by providing metadata using the staking pool smart contract. Their name, description, and social media channels should be listed by providing the following metadata using the staking pool smart contract. The identifying information that we require the validators to provide are:
This information allows the NEAR community to connect with the validators for support, fostering a two-way communication between validators and delegators, as well as the wider NEAR Ecosystem.
Hosting: Validators self-declare whether they operate nodes themselves or use a third party, as well as whether they run their own hardware or use cloud services. We may ask validators to share some information regarding their setup.
As NEAR Protocol is carbon neutral, we ask validators to think through how they offset their validator operation’s CO2 footprint, and communicate any CO2 offsetting that they implement on their end, such as using a renewable energy source to operate the nodes.
Commission Limit: We ask validators who would like to apply for a NEAR Foundation delegation to charge no more than 7% commission on delegation.
Ecosystem Participation: Applicants are expected to actively participate in the NEAR Ecosystem. To further evaluate applicants, we ask the applicants to provide as much information as possible on the following questions:
If you are interested in becoming a validator and apply to the NEAR Foundation delegation, please submit your NEAR Foundation Delegation Program Application as soon as possible. The applications will be reviewed on a rolling basis.
December 13, 2021 is the deadline to submit your application.
Existing Delegation: Given the expected reduction in seat price, the NEAR Foundation delegation will be smaller, compared to historical delegation, without jeopardizing decentralization of the network. For the validators who had historically received delegation from the NEAR Foundation, we encourage you to also apply in the same application process. It’s possible that the existing delegation may be reduced to enable additional validators to receive delegations.
If you’re interested in becoming a validator and apply for delegation, we ask that you start by setting up Testnet and Mainnet nodes, then submit your application for delegation.
We have collaborated with the Open Shard Alliance to build a bootcamp to help train new validators. To get started, please check out the Validator Bootcamp and join the Open Shards Alliance server for onboarding, as well as official validators announcement channels on Discord, Telegram and Twitter.
As the NEAR Ecosystem rapidly scales to include new developers and users, we are working to quickly decentralize our organization and foster a vibrant validator community. These moves will be absolutely critical in reimagining the way NEAR’s community members can build, gather, innovate, and grow together.
The post NEAR Foundation Launches New Validator Program To Further Decentralize the Network appeared first on NEAR Protocol.
]]>