-
Notifications
You must be signed in to change notification settings - Fork 2
Open
Labels
bugSomething isn't workingSomething isn't working
Description
The ingest
command ingests files into the DB.
The processing pipeline is:
- Open file in
Processor
- Parse file, send each certificate to
CertificateProcessor
- In multiple routines, create batches of certificates, in
CertificateProcessor
- For each batch, in an independent routine, update the DB.
- The
CertificateProcessor
waits until all its batches are updated. - The
Processor
waits until theCertificateProcessor
is done - The
Processor
callsOnBundleFinished
if the # of certs is a bundle, or at end
The bug appears:
- When there is no bundle size defined: the call to
OnBundleFinished
is done before theCertProcessor
is done. - When there is a defined bundle size: the
CertProcessor
is still running.
This bug makes the last batches not to be ready for the next steps, done on OnBundleFinished
,
thus creating an incomplete view of the certificate landscape represented by the files.
We need to:
- Just move the code to solve the first bug.
- On each bundle:
- Close the
parsedCertCh
and wait forCertProcessor
to finish. - Create a new
parsedCertCh
and a newCertProcessor
- Close the
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working