feat(ci): data import via GitLab CI Job Service Container
This MR allows for using CI Jobs to update data on all instances. Therefore it introduces two new CI Stages: data_import and data_upload. The import runs in parallel based on a defined list of language aggregations (currently 5). This will allow for checking possible error messages during import and also download and check import results, as they are now stored as job artifact. the data upload stage then will send all the data to respective target instances.
Todo: It is still to define when to start a complete import and what instance(s) to target.
It leaves the app as it is with small changes in two sections sensitive to the presences of an env var CI_SERVICE.
post installation
During post-install a cleanup and non-recursive import of main and lang editions is triggered. These resources are required for the instance to provide complete data, esp. via TextAPI (not used so far, but considered).
document trigger
The trigger is only active when the env var CI_SERVICE is present to detect an ephemeral instance. This allows for running instances (targets) to receive the data without re-rendering the resources.
smaller changes
To not reach the limit of CI Job logs (4194304 bytes.) the still useful stdout output of commons:debug#0 is only returned when memory usage is more than half of available maximum.
minor typos.
Merge request reports
Activity
assigned to @mgoebel
requested review from @mrodzis
mentioned in issue #293
added 1 commit
- 8222f785 - feat(ci): allow a pipeline to be canceled by newer one on same branch.
mentioned in commit 82c00bb6
mentioned in commit f20812dc