Paul Pestov (cb7df679) at 30 May 08:56
Update TIDO v3.2, update README
GitLab
GITLAB_TOKEN=""
FILE="/tmp/results.json"
for i in `seq 1 100`; do curl --silent -X GET "https://gitlab.gwdg.de/api/v4/projects/3451/repository/tree?path=TEI_doc_bearb&page=$i" --header "PRIVATE-TOKEN: $GITLAB_TOKEN" >> $FILE; done &&
jq < $FILE | grep -cs "path"
1335
also there is a hard limit to read max 100 pages
wild guess: parts of the gitlab api return are not read by the importer script.
as we can see in the latest pipeline the document is not handled by the importer. it is not in the list.
Mathias Goebel (7d33b17c) at 25 May 14:06
fix(ci): add image key
Mathias Goebel (295001c3) at 25 May 13:54
fix(ci): ssh connection to docker host
Next steps:
the gfl-app-new developed into a one-stop-shop for many tasks regarding the project:
this is bad, because a single change on the external API now triggers complete data reimport (25 minutes +), thus the whole setup has become inflexible. the main idea is to split the app for both tasks and to become more flexible. a baby step away from this monolith towards a more modular system.
Mathias Goebel (fcd4046d) at 02 May 09:19
Paul Pestov (73f0f50e) at 02 May 09:19
Merge branch '271-when-publish-to-textgrid-please-deploy-data-to-de...
... and 1 more commit
Closes #271
This MR adds jobs to the pipeline that will be triggered from the volltexte repo containing the TEI files.
When files are getting published to TextGrid, dev and live instance will reindex all the files.
Upstream pipeline is set up with https://gitlab.gwdg.de/farbenlehre/volltexte/-/commit/153099b22c2b56af0e965753bd28888f83f76f2b
now https://gitlab.gwdg.de/farbenlehre shows the active projects only. archived ones are still available at the "archived projects" tab in this view.
Closes #271
This MR adds jobs to the pipeline that will be triggered from the volltexte repo containing the TEI files.
When files are getting published to TextGrid, dev and live instance will reindex all the files.
Upstream pipeline is set up with https://gitlab.gwdg.de/farbenlehre/volltexte/-/commit/153099b22c2b56af0e965753bd28888f83f76f2b
tidying up Archivierung folgender Repos in der "Farbenlehre" Gruppe:
Zu prüfen wäre noch:
Thanks Mathias! So we just wait for @paul.pestov s review and then it can be deployed and tested live?