Skip to content

Add script to migrate a repository structure #240

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
njohner merged 74 commits into from
Jan 18, 2021
Merged

Add script to migrate a repository structure #240

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
74 commits
Select commit Hold shift + click to select a range
be8430d
Add repository_migration analyser.
phgross Dec 13, 2019
330227c
Small cleanup.
phgross Feb 21, 2020
94b4537
Add check for max repository_depth.
phgross Feb 21, 2020
a979b82
Add check for leaf_node principle violations.
phgross Feb 25, 2020
5a095ef
Switched output to real values (new_number, new_parent ...)
phgross Feb 26, 2020
e4a3c5d
Refactor Analyser so that it returns operations_list for migrator.
phgross Feb 27, 2020
bfbc6e4
Ignore empty rows during analyse.
phgross Feb 27, 2020
94f9bc4
Extend analyse with information about new repositoryfolders.
phgross Feb 27, 2020
b578023
Add first phase of the Migrator - the creation of new repositoryfolders.
phgross Feb 27, 2020
175da34
Add step "move" to the repository migrator.
phgross Mar 14, 2020
0a17f8b
Add step "referenceprefix number adjustement".
phgross Mar 14, 2020
5512a84
Also regenerate reference_number mapping during reference number adju…
phgross Mar 14, 2020
4531aaa
Add migration step "rename and id adjustment"
phgross Mar 14, 2020
767e5aa
Add reindex step.
phgross Mar 17, 2020
98d4020
Adjust excel reader to the new, simpler excel format.
phgross Nov 19, 2020
925cd07
Fixed ObjectIDUpdater: avoid id conflicts by using namechooser.
phgross Dec 11, 2020
bff1093
Add update description step to repository migration.
phgross Dec 11, 2020
5603a6d
Add validation step to the migrator.
phgross Dec 11, 2020
cb802ed
WIP: browser view to test and develop the script
phgross Dec 17, 2020
f75d658
Add init file to scripts folder.
Dec 21, 2020
4b30641
Make sure to have a string for positions.
Dec 21, 2020
251ebcf
Skip validation for now, as it is broken.
Dec 22, 2020
172f5ed
Refactoring: add OperationItem class.
Dec 22, 2020
54b35cc
Add methods for reference_number and parent_position on OperationItem.
Dec 22, 2020
214cefb
Check for deletion on new_position column.
Dec 22, 2020
9da4ecb
Add ExcelDataExtractor with validation of Excel format.
Dec 22, 2020
6e44396
Fix column mapping.
Dec 22, 2020
b475e1c
Fix indexing.
Dec 22, 2020
f1bfd55
Reindex all necessary objects and indexes.
Dec 28, 2020
6f9ab77
Add precondition checks.
Dec 28, 2020
c39ebb1
Fix creation of repository folder in repository root.
Dec 28, 2020
c415b7b
Fix creation of repository folder in created repository folder.
Dec 28, 2020
d65e187
Fix moving repository folder into the repository root.
Dec 29, 2020
f6f7ce4
Add validation step in RepositoryMigrator.
Dec 29, 2020
96407fe
Add general validation of operations in RepositoryExcelAnalyser.
Dec 29, 2020
4fb7bd7
Also check leaf node principle violation for created repofolders.
Dec 30, 2020
dd807d3
Fix repository_reference_mapping.
Dec 30, 2020
c169291
Add preconditions check for RepositoryMigrator.
Dec 30, 2020
9ff4a99
Sync task model when necessary.
Dec 30, 2020
173a101
Also check technical header in ExcelDataExtractor.
Dec 31, 2020
e03e346
Set local_roles when creating a new repository folder.
Dec 31, 2020
9928705
Skip deletion operations as these are not supported.
Dec 31, 2020
56538f5
A child of a moved object can need reference number update.
Dec 31, 2020
08695cf
Remove unused method.
Dec 31, 2020
452fbf7
Add merge operation to RepositoryExcelAnalyser.
Dec 31, 2020
9ea48f2
Add merge operation to RepositoryMigrator.
Jan 4, 2021
4566452
Fix writing of analysis excel file.
Jan 4, 2021
580efd8
Fix leaf_node_principle_violation check.
Jan 4, 2021
fe2bb14
Also execute migration when running repository_migration_analyse script.
Jan 4, 2021
de10de8
Direct logging to standard output.
Jan 4, 2021
82440e3
Improve operation validation step in analyser.
Jan 4, 2021
335546c
Add more logging.
Jan 4, 2021
80c1228
Do not fail when validation fails.
Jan 4, 2021
e88a72b
Do not modify title for repofolders that are being merged.
Jan 4, 2021
9671bdc
Make sure repofolder being merged does not get reindexed.
Jan 4, 2021
04eba51
Make sure not to fail during reindexing.
Jan 4, 2021
4aaf095
Fix migration validation for deep repofolder reference numbers.
Jan 4, 2021
a645c1f
Revert "WIP: browser view to test and develop the script"
Jan 4, 2021
61c4d82
Revert "Do not fail when validation fails."
Jan 5, 2021
165f6e5
Make sure to commit only at the end of the migration.
Jan 5, 2021
77b0dde
Save operation information on migrated objects.
Jan 5, 2021
c7e080b
Add missing import.
Jan 5, 2021
a510bce
Fix error loging for delete operations.
Jan 8, 2021
ecb366d
Fix docstrings for monkey patches.
Jan 8, 2021
80af7bf
Avoid generating bundle import reports during OS migration.
Jan 8, 2021
b9cc350
Improve naming: OperationItem to RepositoryPosition.
Jan 8, 2021
446a1de
Fix information saved on migrated objects.
Jan 8, 2021
cbd3aa7
Rename migration script.
Jan 8, 2021
33a4ace
Patch Task syncing to avoid predecessor resolution issue.
Jan 13, 2021
c44e0ac
Make sure that repofolder match data in excel before migration.
Jan 14, 2021
b6ef917
Improve progress logging.
Jan 14, 2021
0baefa4
One more patch to avoid committing during bundle import.
Jan 14, 2021
a6bcf68
Do not fail if data is inconsistent in catalog.
Jan 14, 2021
52853b5
Clean-up unnecessary print statement.
Jan 14, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Empty file added opengever/maintenance/scripts/__init__.py
View file
Open in desktop
Empty file.
Loading