This MR allows for using CI Jobs to update data on all instances. Therefore it introduces two new CI Stages: data_import and data_upload. The import runs in parallel based on a defined list of language aggregations (currently 5). This will allow for checking possible error messages during import and also download and check import results, as they are now stored as job artifact. the data upload stage then will send all the data to respective target instances.
Todo: It is still to define when to start a complete import and what instance(s) to target.
It leaves the app as it is with small changes in two sections sensitive to the presences of an env var CI_SERVICE.
post installation
During post-install a cleanup and non-recursive import of main and lang editions is triggered. These resources are required for the instance to provide complete data, esp. via TextAPI (not used so far, but considered).
document trigger
The trigger is only active when the env var CI_SERVICE is present to detect an ephemeral instance. This allows for running instances (targets) to receive the data without re-rendering the resources.
smaller changes
To not reach the limit of CI Job logs (4194304 bytes.) the still useful stdout output of commons:debug#0 is only returned when memory usage is more than half of available maximum.
Mathias Goebel authored