Add more backing-agnostic methods to Root #17934
Conversation
|
!ci-bypass-changelog |
| } ) | ||
|
|
||
| let create_checkpoint_with_directory t ~directory_name = | ||
| let backing_type = |
There was a problem hiding this comment.
Nit: use this aux function defined in this file
let backing_of_t = function
| Stable_db _ ->
Config.Stable_db
| Converting_db _ ->
Converting_db
| [empty_hardfork_db]. The accounts are set in the target database in chunks | ||
| so the daemon is still responsive during this operation; the daemon would | ||
| otherwise stop everything as it hashed every account in the list. *) | ||
| let chunked_migration ?(chunk_size = 1 lsl 6) stable_locations_and_accounts |
There was a problem hiding this comment.
Why not use OCaml threads? Even if we don't have OCaml 5 level of thread support, this is still something?
There was a problem hiding this comment.
Yes, that's also something to consider. @georgeee and I have talked about maybe introducing a ledger variant that would work more like that. I was thinking it could maintain a queue of pending set_batch operations, that would be registered whenever we wanted to commit to a new ledger state, including the initial one. Then a separate worker thread could be spawned to actually process the queue. You'd need a way of telling the worker to stop and maybe retrieving a handle to the database. The usual thread communication stuff. That kind of structure might do a better job of hiding the details of the root backing from the rest of the code.
I think we'd need code like this even then, because the worker would still be running ocaml code in the same process; it would have to avoid blocking for long periods of time.
In any case, that kind of setup might be easier to deal with elsewhere in the code. I was thinking about that in the context of #17754.
| | Stable_db db -> | ||
| let directory_name = | ||
| Stable_db.get_directory db | ||
| |> Option.value_exn |
| ~depth:ledger_depth ~assert_synced:true () | ||
| in | ||
| L.Root.close converting_root ; | ||
| Deferred.unit ) |
There was a problem hiding this comment.
Might want a check to ensure there's no "long async job" happening? So to keep up with the guarantee "gradual"
2 participants
It will be helpful for code organization purposes when generating the hard fork config to be able to copy root ledgers (via checkpointing), convert a stable-database-backed root to a converting root, and to (unsafely) view the components of a root ledger.
I've added a test of the
make_convertingmethod in particular, which converts a stable root to a converting one by migrating the stable database in chunks.These methods were pulled out of #17816; that PR now depends on this one.
Closes: #17751