Repository navigation
node, wallet: clean up derived validator keys on failed deposits (#379) - #1331
Open
Sanchay117 wants to merge 1 commit into
Open
Sanchay117 wants to merge 1 commit into
Sanchay117 wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #379
Summary
When
rocketpool node deposit(nodeDeposits) fails mid-batch or duringDepositMultisubmission, Smartnode previously left newly derived validator keystore and secret files on disk across all consensus client keystores and discarded the in-memorynext_accountincrement (becausew.Save()was only called on the happy path at the end ofnodeDeposits).Under Saturn megapool batch deposits (
count >= 1), this created two compounding hazards:#379): Everyw.CreateValidatorKey()iteration immediately writes encrypted keystore + password files for Lighthouse, Lodestar, Nimbus, Prysm (all-accounts.keystore.json), and Teku. If deposit validation, status lookup, ornode.DepositMultifailed (e.g.FeeTooLowToCompeteas in#386, RPC timeout, or contract revert), up tocountunfunded validator keys remained on disk and would be loaded on the next VC restart.#379/#427): Becauseservices.GetWalletreloadshdWalletfrom disk on each API call, omittingw.Save()on failure causednext_accountto revert to its pre-batch value. If an earlier transaction broadcast was ambiguous (e.g. mempool propagation timeout that later landed), a retry re-derived the exact same validator pubkeys.Per
@jshufro's specification in #379, failed deposits now delete all newly derived validator keys from every keystore while preserving (persisting) the incrementednext_accountcounter on live (submit == true) deposit attempts.Changes
Per-key deletion across all 5 consensus client keystores (
shared/services/wallet/keystore/...):DeleteValidatorKey(pubkey types.ValidatorPubkey) errortokeystore.Keystore.validators/0x<pubkey>directory andsecrets/0x<pubkey>file (os.ErrNotExisttreated asnil).keys/0x<pubkey>.jsonandpasswords/0x<pubkey>.txt.prysm-non-hd): Ifall-accounts.keystore.jsondoes not exist yet, returnnilwithout creating directories orsecret. Otherwise load the account store, filter out matching pubkey/privkey entries while preserving the order of all remaining keys, and re-encryptall-accounts.keystore.jsonin place.Wallet-level key deletion & partial-write rollback (
shared/services/wallet/...):DeleteValidatorKey(pubkey rptypes.ValidatorPubkey) errortowallet.Wallet,hdWallet, andmasqueradeWallet(ErrIsMasquerading).hdWallet.DeleteValidatorKeyiterates deterministically across all registered keystores and joins errors viaerrors.Joinso an error in one keystore never prevents cleanup of the remaining keystores. It intentionally leavesw.ws.NextAccountuntouched.hdWallet.CreateValidatorKeynow rolls back any partially written keystore artifacts viaDeleteValidatorKeyifStoreValidatorKeyfails mid-way across client keystores, while keepingNextAccountincremented.Transactional batch cleanup in
nodeDeposits(rocketpool/api/node/deposit.go):executeDepositBatchwith adefercleanup guard trackingcreatedPubkeysanddepositSucceeded.!depositSucceeded), allcreatedPubkeysfrom the batch are deleted across all 5 keystores, and whensubmit == true,w.Save()persists the advancednext_accountto disk.submit == false(offline/unsigned tx generation), temporary keystore files are cleaned up on failure without mutatingnext_accounton disk.depositSucceeded = trueis set immediately uponsubmitDeposits(node.DepositMulti) returningerr == nil(prior tow.Save()), guaranteeing that live validator keys for an already-broadcast deposit transaction are never deleted even if saving the wallet file subsequently fails.Testing
Added unit tests covering both layers:
shared/services/wallet/validator_cleanup_test.go:lighthouse,lodestar,nimbus,prysm,teku), verifying disk artifact removal, neighbor key preservation, Prysm reload from disk, idempotency, andnext_accountnon-decrement.all-accounts.keystore.jsondown to 0 keys and reusing it for subsequent keys.CreateValidatorKeyencounters a failing keystore.masqueradeWallet.DeleteValidatorKeyreturningErrIsMasquerading.rocketpool/api/node/deposit_cleanup_test.go:count = 2) across all 5 real keystores + persistednext_account.DepositMultisubmission failure with pre-existing active validators: verifies pre-existing keys survive, all failed batch keys are wiped across all 5 keystores, reloaded wallet preservesnext_account(5), subsequent retry derives fresh index5, andRecoverValidatorKeyrecovers index5across the burned index gap (2..4).getBondRequirement,getDepositData,getValidatorStatusRPC error,status.Existsduplicate pubkey alert,validateDepositsignature failure).submit = falsefailure leaving persisted walletnext_accountuntouched while wiping temporary keystore files.w.Save()failure retaining all live validator keys on disk.panicrecovery guard wiping derived keys and savingnext_accountbefore re-panicking.go test -v ./shared/services/wallet/... ./rocketpool/api/node/... ./shared/services ./rocketpool/api/wallet/...