Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update sync pipelines to continue from era1 import #8403

Open
wants to merge 35 commits into
base: main
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
c777ae6
Refactor to use clock to validate expiry
Matilda-Clerke Jan 30, 2025
d27a5a6
7935: Finish unit testing E2StoreReader
Matilda-Clerke Feb 12, 2025
b7a359d
Build out E2StoreReader and associated files
Matilda-Clerke Feb 19, 2025
da718f0
Spotless
Matilda-Clerke Feb 19, 2025
c19aa23
Add javadoc to new util module classes
Matilda-Clerke Feb 20, 2025
2d99f07
Merge branch 'main' into 7935-add-e2storereader
Matilda-Clerke Feb 21, 2025
8b67d5f
Implement Era1 block import sub command
Matilda-Clerke Feb 25, 2025
bb1c713
Remove unnecessary e2 and era file stuff
Matilda-Clerke Feb 26, 2025
f2060aa
Rename files to be era1 specific
Matilda-Clerke Feb 26, 2025
079b40c
Rename e2 package to era1
Matilda-Clerke Feb 26, 2025
816e45a
Spotless
Matilda-Clerke Feb 26, 2025
dd16c26
Merge branch 'refs/heads/7935-add-e2storereader' into 7935-add-era1-f…
Matilda-Clerke Feb 26, 2025
6dea2f3
Rename Era1StoreReaderListener to Era1ReaderListener
Matilda-Clerke Feb 26, 2025
b9564d5
Merge branch 'refs/heads/7935-add-e2storereader' into 7935-add-era1-f…
Matilda-Clerke Feb 26, 2025
37c257e
Clean up after merge
Matilda-Clerke Feb 26, 2025
26fd7b5
Set up Era1BlockImporter to enable FAST sync and variations upon besu…
Matilda-Clerke Feb 27, 2025
3afd859
Use bouncycastle for little endian to long conversion
Matilda-Clerke Feb 27, 2025
abbe9af
Rename slot related variables to blockIndex
Matilda-Clerke Feb 27, 2025
f921fc0
Spotless
Matilda-Clerke Feb 27, 2025
a387b33
Merge branch 'refs/heads/7935-add-e2storereader' into 7935-add-era1-f…
Matilda-Clerke Mar 2, 2025
93afe39
Fix broken unit test
Matilda-Clerke Mar 3, 2025
11d4885
Fix AT compilation
Matilda-Clerke Mar 3, 2025
aa9a08b
Fix javadoc
Matilda-Clerke Mar 3, 2025
6e9043d
Merge branch 'main' into 7935-add-era1-format-to-blocks-importer
Matilda-Clerke Mar 4, 2025
0d2385c
Rework Era1BlockImporter to use appropriate data directory. Add unit …
Matilda-Clerke Mar 5, 2025
3730494
Fix javadoc
Matilda-Clerke Mar 5, 2025
dfd937d
changelog
Matilda-Clerke Mar 5, 2025
692ae2c
Add era1 files to .gitattributes as binary file type
Matilda-Clerke Mar 5, 2025
a3f926a
Merge branch 'main' into 7935-add-era1-format-to-blocks-importer
Matilda-Clerke Mar 6, 2025
1f3b8d7
Remove pivot block header file creation from Era1BlockImporter
Matilda-Clerke Mar 6, 2025
16f3351
Modify SnapDownloaderFactory to allow snap sync after importing era1 …
Matilda-Clerke Mar 11, 2025
db7be9e
Modify Snap, checkpoint, and fast sync downloader factories to allow …
Matilda-Clerke Mar 12, 2025
1f50a1d
Merge branch 'main' into 7935-update-sync-pipelines-to-continue-from-…
Matilda-Clerke Mar 12, 2025
b731b08
Check if world state root hash is still the genesis state root before…
Matilda-Clerke Mar 13, 2025
77da0e9
Fix broken unit tests
Matilda-Clerke Mar 13, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fix javadoc
Signed-off-by: Matilda Clerke <[email protected]>
Matilda-Clerke committed Mar 3, 2025
commit aa9a08b775f9769831e12e7f4a180d07a56f5a25
Original file line number Diff line number Diff line change
@@ -56,6 +56,9 @@
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

/**
* Tool for importing era1-encoded block data, headers, and transaction receipts from era1 files.
*/
public class Era1BlockImporter implements Closeable {
private static final Logger LOG = LoggerFactory.getLogger(Era1BlockImporter.class);

@@ -65,6 +68,19 @@ public class Era1BlockImporter implements Closeable {
private static final int ERA1_BLOCK_COUNT_MAX = 8192;
private static final int IMPORT_COUNT_FOR_LOG_UPDATE = 1000;

/** Default Constructor. */
public Era1BlockImporter() {}

/**
* Imports the blocks, headers, and transaction receipts from the file found at the supplied path
*
* @param controller The BesuController
* @param path The path
* @throws IOException IOException
* @throws ExecutionException ExecutionException
* @throws InterruptedException InterruptedException
* @throws TimeoutException TimeoutException
*/
public void importBlocks(final BesuController controller, final Path path)
throws IOException, ExecutionException, InterruptedException, TimeoutException {
File fastSyncDir = new File("./" + FASTSYNC_DIRECTORY);
2 changes: 2 additions & 0 deletions besu/src/main/java/org/hyperledger/besu/cli/BesuCommand.java
Original file line number Diff line number Diff line change
@@ -721,6 +721,7 @@ static class PrivacyOptionGroup {
*
* @param rlpBlockImporter RlpBlockImporter supplier
* @param jsonBlockImporterFactory instance of {@code Function<BesuController, JsonBlockImporter>}
* @param era1BlockImporter Era1BlockImporter supplier
* @param rlpBlockExporterFactory instance of {@code Function<Blockchain, RlpBlockExporter>}
* @param runnerBuilder instance of RunnerBuilder
* @param controllerBuilder instance of BesuController.Builder
@@ -764,6 +765,7 @@ public BesuCommand(
*
* @param rlpBlockImporter RlpBlockImporter supplier
* @param jsonBlockImporterFactory instance of {@code Function<BesuController, JsonBlockImporter>}
* @param era1BlockImporter Era1BlockImporter supplier
* @param rlpBlockExporterFactory instance of {@code Function<Blockchain, RlpBlockExporter>}
* @param runnerBuilder instance of RunnerBuilder
* @param controllerBuilder instance of BesuController.Builder
Original file line number Diff line number Diff line change
@@ -104,6 +104,7 @@ public class BlocksSubCommand implements Runnable {
*
* @param rlpBlockImporter the RLP block importer
* @param jsonBlockImporterFactory the Json block importer factory
* @param era1BlockImporter the era1 block importer supplier
* @param rlpBlockExporterFactory the RLP block exporter factory
* @param out Instance of PrintWriter where command usage will be written.
*/