How zeroShadow Uses Virtual TestNets to Recover Stolen Assets Faster While Building Client Trust
"I can show our clients the transactions we’ll initiate on their behalf after they give us their private keys. This way, they don’t have to understand what's happening under the hood, but they can visually see the before-and-after effects of the transaction."
Harrison O.
Lead Engineer at zeroShadow
Key highlights:
- The problem: During time-sensitive incident responses, the zeroShadow team experienced friction with local tooling, with manual setups prolonging response times when every second counted. Plus, the clients had no transparent way to review recovery steps before execution.
- The solution: By switching to Virtual TestNets, the zeroShadow gained a full replica of production chains to quickly test asset recovery steps, monitoring setups, and governance proposals on complete onchain data. They are also able to share the expected results with their clients, eliminating potential trust gaps.
- Key results: Virtual TestNets saves the zeroShadow team a full day of engineering hours on manual setups every month, enabling them to offer a faster incident response service and prevent critical delays that could result in losses. It also enables the team to build trust with clients by sharing their approach in a human-readable, transparent way, ultimately minimizing friction when implementing security solutions.
As an integrator for smart contract monitoring and alerting solutions, zeroShadow offers tailored preventative security measures and incident response services in the form of a Virtual Security Operations Center (vSOC). The zeroShadow team provides 24/7 security support across different segments, including L1s, digital asset hedge funds, dapps, and lending protocols. It also assists traditional financial institutions and even law firms with security, tokenomics, and due diligence guidance.
Having froze $300M in stolen crypto, zeroShadow started as an incident response company focused on recovering stolen funds. Now, with a team spanning vSOC engineers, global investigators, compliance/cyber specialists, and a dedicated team for threat intelligence, zeroShadow provides holistic security services covering both preventative and post-exploit steps.
As part of their vSOC center, zeroShadow integrates with Tenderly for smart contract monitoring solutions, responding to high-severity alerts on behalf of their customers. Aside from using Tenderly’s monitoring solutions, the team also relies on Virtual TestNets, a full replica of production chains, throughout the entire process, from integrating and testing monitoring solutions to assisting in the recovery of stolen assets.
The overhead of using local nodes in time-sensitive situations
Whether testing monitoring integrations, simulating asset recovery, or investigating exploits, the zeroShadow team needs exact replicas of production chains to work with. While open-source tools like Anvil and Foundry can fork chains for testing, they introduce friction in moments when every second counts.
“While you can use Anvil to fork a chain, Virtual TestNets offer a more user-friendly experience and the ability to share state differences and simulations with anyone.” – Harrison O, Lead Engineer at zeroShadow
In time-critical situations, these "oops moments" can prolong incident responses, resulting in financial losses. With Virtual TestNets, the team can simply click a button to revert state, eliminating manual steps that would otherwise waste significant time.
Additionally, if the team were to interact with AWS, they would have to store the code there and configure connections to Anvil from different machines, an unnecessarily time-consuming setup when time is of the essence.
By using Tenderly Virtual TestNets, zeroShadow gains critical flexibility. When testing front-running scenarios and custom monitoring setups, they can quickly swap the Virtual TestNet RPC with Node RPC, switching between development and production environments in seconds.
“Having this setup in Anvil is more temporary, while you can easily revert state, go forward in time, and use the public explorer in Tenderly. The dev user experience is really good on Virtual TestNets.” – Harrison
Testing high-stakes asset recovery methods on Virtual TestNets
As part of their incident response and investigative services, the zeroShadow security team traces and recovers funds on behalf of their customers in the event of an exploit. Depending on the type of issue, the responses may focus on moving or unstaking funds.
During this step, the zeroShadow team not only investigates what went wrong but also simulates and presents asset recovery solutions on Virtual TestNets.
Testing front-running scenarios
For instance, if a user has their funds locked in a contract that gets unstaked in intervals, the zeroShadow team takes steps to pull the assets out of the contract. They can easily test the recovery steps on Virtual TestNets under realistic conditions, adjust and revert the state of their environments, and offer a faster incident response service to prevent financial losses.
When front-running victim transactions to recover funds before attackers can drain them, timing and accuracy are critical. The zeroShadow team developed a script that runs every single block, checking whether there are enough funds in a reward contract to successfully execute the recovery. If a client's funds unlock in three days but the reward contract might not have sufficient tokens at that moment, the team needs to test the scenario in advance.
On Virtual TestNets, they can manipulate the state and time to simulate adequate reward balances and verify their recovery script works correctly. Once confirmed, they pay significant gas fees to Flashbots and other private relayers to include the client transaction as quickly as possible. Additionally, state syncing on Virtual TestNets ensures their testing environments reflect actual onchain conditions, minimizing the risk of unexpected failures or unwanted behavior in production.
Minimizing risks by testing on a custom state
Aside from providing 24/7 incident response and asset recovery in the event of exploits, zeroShadow also implements and tests preventative security measures on behalf of its clients. Setting up monitoring solutions and ensuring DAO proposals execute as expected are essential steps during this phase.
Running monitoring on top of a custom state
To test the monitoring setup, the zeroShadow team simulates different what-if scenarios and runs the monitoring solutions on top of them. For this purpose, they configure the state on Virtual TestNets to be in a problematic state, which is particularly helpful for teams with niche use cases and testing edge case scenarios.
For instance, hedge funds and VCs are typically concerned about potential smart contract risks and may want automated responses with small execution windows. For this segment, the zeroShadow responses may involve a multisig signer with a narrow scope of actions they can perform, such as pulling the funds to a wallet.
On Virtual TestNets, the team can customize the state, attach a Safe module, and test an automated response for pausing or withdrawing funds. Testing the entire flow on a production replica enables the team to reveal potential failures and ensure the setup works as expected.
“Using Virtual TestNets, we can configure the state based on custom conditions and share it with protocols, L1s, and other teams. This way, we can show exactly what would happen if something went wrong, and the prevention or automated response mechanism.” – Harrison
Simulating time-dependent governance proposals
Beyond testing monitoring setups and incident responses, zeroShadow also uses Virtual TestNets to test DAO governance proposals for clients. Governance proposals involve complex, multi-day voting workflows and timelocks. For instance, proposals sit for three days before voting begins, then approved proposals move to a timelock for another two days before execution.
With Virtual TestNets, the zeroShadow team can manipulate block timestamps to fast-forward through the waiting periods, execute the entire governance cycle in minutes, and verify the before-and-after state changes. The ability to configure state and execute multi-step proposals on Virtual TestNets gives both the zeroShadow team and their clients confidence that governance actions will work as expected before going live.
Building client trust by bringing transparency with a built-in public explorer
Aside from facilitating the entire setup and testing process for their internal team, zeroShadow also uses Virtual TestNets to bring transparency and build trust with their users. For non-technical users, Virtual TestNets eliminate the complexity of deeply technical transactions thanks to the built-in public explorer.
By sharing a public explorer to Virtual TestNets, zeroShadow can demonstrate the actions they’ll take to recover the funds. Their clients can easily see the state and balance changes initiated with the transactions, along with their recovery addresses. This way, zeroShadow can address potential concerns their clients may have, giving them peace of mind when dealing with critical issues.
“I can show our clients the transactions we’ll initiate on their behalf after they give us their private keys. This way, they don’t have to understand what's happening under the hood, but they can visually see the before-and-after effects of the transaction.” – Harrison
Additionally, when testing the monitoring setup and implementing preventative measures, zeroShadow also demonstrates the what-if scenarios on Virtual TestNets. This approach further brings transparency to zeroShadow clients because they can look through each transaction and see what’s happening in a user-friendly, human-readable way, building confidence into their process and solutions.
“Virtual TestNets help a lot with the transparency side of things. It’s a lot easier for our clients to see exactly what we did and how monitoring is set up on Virtual TestNets instead of having to simply trust everything works.” – Harrison
Key results
By integrating Virtual TestNets into both prevention and incident response services, zeroShadow brought measurable improvements to its internal workflows while strengthening client relationships:
- A full day of engineering time saved every month on manual setups and testing. Tasks that previously required navigating local node configurations and manual chain resets, now take just a few clicks, saving critical time when responding to urgent security incidents.
- Faster incident response service to prevent financial losses. Being able to fork a chain within milliseconds and manipulate and revert the state with a button speeds up the team’s incident response service. Ultimately, this enables zeroShadow to respond quickly when every second missed can result in lost funds.
- Enhanced client trust through visual representations for transparency. Non-technical clients can review proposed asset recovery steps and monitoring configurations through the public explorer on Virtual TestNets, seeing exactly what will happen without needing to understand the underlying code. This transparency is especially valuable when clients need to provide private keys for asset recovery, giving them confidence and peace of mind.
- New testing scenarios enabled thanks to full onchain data and state manipulation. The zeroShadow team can replicate any of the 100+ supported EVM chains, configure the state of their replicas, and test edge-case scenarios on real data, catching potential before implementing solutions.
- Improved developer experience thanks to the flexibility and ease of use of Virtual TestNets. The zeroShadow engineers have a simple way to analyze, test, and present solutions to their clients all within safe, production-synced environments.
Closing the security gaps with zeroShadow and Tenderly
As a key contributor to both Web3 teams and Web2 companies, zeroShadow raises the security levels and provides critical support when it matters the most. Through their integration with Tenderly for testing incident management responses and monitoring critical events onchain, zeroShadow can provide reliable solutions and protect clients’ assets.
For protocols and institutions working with zeroShadow, the integration with Tenderly Virtual TestNets brings faster incident response, greater transparency, and higher confidence in security measures. As the team continues to protect clients' assets and respond to incidents across the ecosystem, Virtual TestNets remain central to bringing prompt security responses when it matters most.
Head over to Virtual TestNets to try running custom scenarios on real onchain data.
