kopal Library of Retrieval and Ingest issueshttps://gitlab.gwdg.de/dariah-de/kopal-library-of-retrieval-and-ingest/-/issues2022-10-11T09:43:33Zhttps://gitlab.gwdg.de/dariah-de/kopal-library-of-retrieval-and-ingest/-/issues/1Unmarshalling Error: java.lang.IllegalArgumentException: Illegal character in...2022-10-11T09:43:33ZStefan E. Funkfunk@sub.uni-goettingen.deUnmarshalling Error: java.lang.IllegalArgumentException: Illegal character in opaque partSee https://gitlab.gwdg.de/jose.calvotello/eltec_import/-/issues/19:
Error at de.langzeitarchivierung.kolibri.actionmodule.textgrid.SubmitFiles:
`Exception in thread "pool-1-thread-2" javax.xml.ws.soap.SOAPFaultException: Unmarshalling...See https://gitlab.gwdg.de/jose.calvotello/eltec_import/-/issues/19:
Error at de.langzeitarchivierung.kolibri.actionmodule.textgrid.SubmitFiles:
`Exception in thread "pool-1-thread-2" javax.xml.ws.soap.SOAPFaultException: Unmarshalling Error: java.lang.IllegalArgumentException: Illegal character in opaque part at index 12: viaf:1657716`https://gitlab.gwdg.de/dariah-de/kopal-library-of-retrieval-and-ingest/-/issues/2Workflow ends before all queues are finished2021-11-19T15:05:42ZStefan E. Funkfunk@sub.uni-goettingen.deWorkflow ends before all queues are finishedWorkflow says:
`[#1] de.langzeitarchivierung.kolibri.WorkflowTool.process() [2021-11-19T15:02:30.129+01:00]
[INFO] The process queue has been processed. Total time elapsed: 33 minutes 532 milliseconds
[INFO] Everything has bee...Workflow says:
`[#1] de.langzeitarchivierung.kolibri.WorkflowTool.process() [2021-11-19T15:02:30.129+01:00]
[INFO] The process queue has been processed. Total time elapsed: 33 minutes 532 milliseconds
[INFO] Everything has been done! This process was logged to file: /Users/fugu/work/kolibri-distant-reading_fu/folders/log/kolibri.log
[INFO] *removing log handlers*
[INFO] *stop*`
...and processes are still running.https://gitlab.gwdg.de/dariah-de/kopal-library-of-retrieval-and-ingest/-/issues/3Add import.zip as gitlab flow artifact?2021-11-24T13:43:32ZStefan E. Funkfunk@sub.uni-goettingen.deAdd import.zip as gitlab flow artifact?Normally we commit every time the version changes a new import.zip that is build via mvn. Could we do this using Gitlab artifacts?Normally we commit every time the version changes a new import.zip that is build via mvn. Could we do this using Gitlab artifacts?https://gitlab.gwdg.de/dariah-de/kopal-library-of-retrieval-and-ingest/-/issues/5Reduce logs while building project2021-11-24T16:41:53ZStefan E. Funkfunk@sub.uni-goettingen.deReduce logs while building projectTo avoid:
`Job's log exceeded limit of 4194304 bytes.
Job execution will continue but no more output will be collected.`
...maybe we do not need all the test logs during Gitlab builds?To avoid:
`Job's log exceeded limit of 4194304 bytes.
Job execution will continue but no more output will be collected.`
...maybe we do not need all the test logs during Gitlab builds?https://gitlab.gwdg.de/dariah-de/kopal-library-of-retrieval-and-ingest/-/issues/7Reduce JUnit tests to less objects for TG-import2021-11-24T16:48:28ZStefan E. Funkfunk@sub.uni-goettingen.deReduce JUnit tests to less objects for TG-importTo gain a bit of build duration time :-) We do not need to test the BIIIIG projects every time, I guess!To gain a bit of build duration time :-) We do not need to test the BIIIIG projects every time, I guess!Stefan E. Funkfunk@sub.uni-goettingen.deStefan E. Funkfunk@sub.uni-goettingen.dehttps://gitlab.gwdg.de/dariah-de/kopal-library-of-retrieval-and-ingest/-/issues/11Check pathes to work files in editon metadata files [was #33057]2021-11-25T15:13:36ZGhost UserCheck pathes to work files in editon metadata files [was #33057]<isEditionOf>BLUBB/ENG18400.work</isEditionOf> must be tested for existance, there are errors: java.io.IOException: info.textgrid.namespaces.middleware.tgcrud.services.tgcrudservice.IoFault: info.textgrid.namespaces.middleware.tgcrud.ser...<isEditionOf>BLUBB/ENG18400.work</isEditionOf> must be tested for existance, there are errors: java.io.IOException: info.textgrid.namespaces.middleware.tgcrud.services.tgcrudservice.IoFault: info.textgrid.namespaces.middleware.tgcrud.services.tgcrudservice.IoFault: Failure storing to the Sesame databasehttps://gitlab.gwdg.de/dariah-de/kopal-library-of-retrieval-and-ingest/-/issues/15koLibRI CLI: DeleteFiles configuration needs an existing hotfolder and it nee...2021-11-25T15:15:02ZGhost UserkoLibRI CLI: DeleteFiles configuration needs an existing hotfolder and it needs to be filled with some file(s) [was #30147]You shuld not be needing a (filled) hotfolder for deleting files! Hotfolder must only be existing if ID mapping files are used for deletion. If configuration is set to "uri:" od "tgpr:" hotfolder should not be needed/checked at all.You shuld not be needing a (filled) hotfolder for deleting files! Hotfolder must only be existing if ID mapping files are used for deletion. If configuration is set to "uri:" od "tgpr:" hotfolder should not be needed/checked at all.https://gitlab.gwdg.de/dariah-de/kopal-library-of-retrieval-and-ingest/-/issues/17kolibri-addon-textgrid-import mit Policy "publish_import" (sicher auch "delet...2021-11-25T15:14:49ZGhost Userkolibri-addon-textgrid-import mit Policy "publish_import" (sicher auch "delete_import) fängt nur an, wenn der konfigurierte Hotfolder nicht leer ist. [was #23988]Ist der Hotfolder (z. B. ./folders/hotfolder) leer, dann laufen die Policies "publish_import" und "delete_import" Workflow gar nicht erst an! Es muss nur eine einzige Datei im Hotfolder vorhanden sein, dann rennt er los (diese darf auch ...Ist der Hotfolder (z. B. ./folders/hotfolder) leer, dann laufen die Policies "publish_import" und "delete_import" Workflow gar nicht erst an! Es muss nur eine einzige Datei im Hotfolder vorhanden sein, dann rennt er los (diese darf auch leer sein!). Das ist nicht Sinn der Sache und soll gefixed werden. Als Workaround bitte einfach eine Datei ins Hotfolder kopieren oder dort eine anlegen/speichern.https://gitlab.gwdg.de/dariah-de/kopal-library-of-retrieval-and-ingest/-/issues/19unable to upload large size file (koLibRI) [was #23537]2021-11-25T15:13:18ZGhost Userunable to upload large size file (koLibRI) [was #23537]…nor the kolibir import is successful for a 60GB binary file. org.apache.cxf.interceptor.Fault: Could not write attachments.…nor the kolibir import is successful for a 60GB binary file. org.apache.cxf.interceptor.Fault: Could not write attachments.https://gitlab.gwdg.de/dariah-de/kopal-library-of-retrieval-and-ingest/-/issues/20No aggregation if only one fileGrp is ingested. [was #20599]2021-11-25T15:13:18ZGhost UserNo aggregation if only one fileGrp is ingested. [was #20599]If only one METS File Group is ingested, it is not necessary to create a special aggregation during koLibRI Ingest.If only one METS File Group is ingested, it is not necessary to create a special aggregation during koLibRI Ingest.https://gitlab.gwdg.de/dariah-de/kopal-library-of-retrieval-and-ingest/-/issues/24Enhance the (automatic) mimetype extraction of the DfgViewerMetadataProcessor...2021-11-25T15:13:18ZGhost UserEnhance the (automatic) mimetype extraction of the DfgViewerMetadataProcessor, and others [was #9462]Enhance the (automatic) mimetype extraction of the DfgViewerMetadataProcessor, and othersEnhance the (automatic) mimetype extraction of the DfgViewerMetadataProcessor, and othershttps://gitlab.gwdg.de/dariah-de/kopal-library-of-retrieval-and-ingest/-/issues/25PIDs should be re-used if objects were deleted from the sandbox [was #9331]2021-11-25T15:13:18ZGhost UserPIDs should be re-used if objects were deleted from the sandbox [was #9331]PIDs should be re-used if objects were deleted from the sandboxPIDs should be re-used if objects were deleted from the sandboxhttps://gitlab.gwdg.de/dariah-de/kopal-library-of-retrieval-and-ingest/-/issues/26Aggregation-Import must ensure creation of a root-edition or collection [was ...2022-09-23T12:45:06ZGhost UserAggregation-Import must ensure creation of a root-edition or collection [was #12181]If importing using the aggregation_import policy, it must be ensured that a root edition or colection ia (a) existing or (b) must be forced by the import module!If importing using the aggregation_import policy, it must be ensured that a root edition or colection ia (a) existing or (b) must be forced by the import module!https://gitlab.gwdg.de/dariah-de/kopal-library-of-retrieval-and-ingest/-/issues/27TG-import external should provide a minimal METS import profile [was #9459]2021-11-25T15:13:18ZGhost UserTG-import external should provide a minimal METS import profile [was #9459]The METS import right now creates lots of aggreagations, reassembling the internal METS Structure. It would be nice to have the possbility to chosse a minimized import, which just creates an edititon and only one aggregation the selected...The METS import right now creates lots of aggreagations, reassembling the internal METS Structure. It would be nice to have the possbility to chosse a minimized import, which just creates an edititon and only one aggregation the selected images (e.g. MAX) in an image Folder, e.g.: /TheMetsEdidition/ +/theMetsFile.xml +/images/ ++/Vorderdeckel-MAX.jpg ++/ ... ++/Vita-Alexandri-Magni-Blatt21r.jpg ++/ ... ++/Rückdeckel-MAX.jpghttps://gitlab.gwdg.de/dariah-de/kopal-library-of-retrieval-and-ingest/-/issues/29Check RDF metadata at re-import using TG-import. Have we to delete e.g. the i...2021-11-25T15:13:18ZGhost UserCheck RDF metadata at re-import using TG-import. Have we to delete e.g. the image metadata to not have two of them? [was #14150]https://gitlab.gwdg.de/dariah-de/kopal-library-of-retrieval-and-ingest/-/issues/30Make it possible to use URLs for DFG-Viewer METS import [was #9188]2021-11-25T15:13:19ZGhost UserMake it possible to use URLs for DFG-Viewer METS import [was #9188]Make it possible to use URLs for DFG-Viewer METS importMake it possible to use URLs for DFG-Viewer METS importhttps://gitlab.gwdg.de/dariah-de/kopal-library-of-retrieval-and-ingest/-/issues/56Attributes sorted alphabetically after import [was #9288]2021-11-25T15:12:58ZGhost UserAttributes sorted alphabetically after import [was #9288]Attributes sorted alphabetically after importAttributes sorted alphabetically after importhttps://gitlab.gwdg.de/dariah-de/kopal-library-of-retrieval-and-ingest/-/issues/57Link Rewriter should rewrite urls in <?xsl-stylesheet?> processing instructio...2021-11-25T15:12:58ZGhost UserLink Rewriter should rewrite urls in <?xsl-stylesheet?> processing instruction [was #9438]Link Rewriter should rewrite urls in <?xsl-stylesheet?> processing instructionLink Rewriter should rewrite urls in <?xsl-stylesheet?> processing instructionhttps://gitlab.gwdg.de/dariah-de/kopal-library-of-retrieval-and-ingest/-/issues/58Ingest von Metadaten aus Datenbanken [was #9130]2021-11-25T15:12:58ZGhost UserIngest von Metadaten aus Datenbanken [was #9130]Ingest von Metadaten aus DatenbankenIngest von Metadaten aus Datenbankenhttps://gitlab.gwdg.de/dariah-de/kopal-library-of-retrieval-and-ingest/-/issues/99Editions inside collections should be metadata checked [was #8919]2021-11-25T15:34:03ZGhost UserEditions inside collections should be metadata checked [was #8919]Editions inside collections should be metadata checked (I think)Editions inside collections should be metadata checked (I think)