Skip to content
Snippets Groups Projects

Compare revisions

Changes are shown as if the source revision was being merged into the target revision. Learn more about comparing revisions.

Source

Select target project
No results found

Target

Select target project
  • dariah-de/dariah-de-oai-pmh-services
1 result
Show changes
Commits on Source (23)
Hide whitespace changes
Inline Side-by-side
Showing
with 155 additions and 163 deletions
CHANGELOG.md
View file @ 3009dc27
## [6.2.3](https://gitlab.gwdg.de/dariah-de/dariah-de-oai-pmh-services/compare/v6.2.2...v6.2.3) (2024-05-23)
### Bug Fixes
* add new common version ([3bc5dba](https://gitlab.gwdg.de/dariah-de/dariah-de-oai-pmh-services/commit/3bc5dbab5c5e1e45f56005a4a3bcc448d741f1d3))
* fix some source issues with totalHits ([e423a07](https://gitlab.gwdg.de/dariah-de/dariah-de-oai-pmh-services/commit/e423a0798c2416947d3edb00972180244bbf20de))
* fix test ([94ec286](https://gitlab.gwdg.de/dariah-de/dariah-de-oai-pmh-services/commit/94ec2867833560559eb5e6a4e588d5ff2b5a38ee))
* increase common to 5.0.0 release (es7) ([a643f5d](https://gitlab.gwdg.de/dariah-de/dariah-de-oai-pmh-services/commit/a643f5dd62362d75b9d8b899f9bfe42773bd239c))
* increase mets mods mapping version ([c3c15ca](https://gitlab.gwdg.de/dariah-de/dariah-de-oai-pmh-services/commit/c3c15ca7e835ca8fc7532cd42aa1d0620d5d35aa))
* remove dep to ltputils ([bcc455c](https://gitlab.gwdg.de/dariah-de/dariah-de-oai-pmh-services/commit/bcc455ce84926ad008b9d158e63ea50795b4bac5))
* remove type from es client and configuration ([24a8ee0](https://gitlab.gwdg.de/dariah-de/dariah-de-oai-pmh-services/commit/24a8ee01811d1017648eb112c057e31a1d26b6a0))
* remove unused type getters and setters ([bc3704a](https://gitlab.gwdg.de/dariah-de/dariah-de-oai-pmh-services/commit/bc3704a4ef6d65904f1595fd3da1337e925e1d12))
* upgrade to increased es version ([77ecf94](https://gitlab.gwdg.de/dariah-de/dariah-de-oai-pmh-services/commit/77ecf94001faa78d57d046100d7593fb9c1324aa))
## [6.2.2](https://gitlab.gwdg.de/dariah-de/dariah-de-oai-pmh-services/compare/v6.2.1...v6.2.2) (2023-03-14)
......
docs_dhrep/index.rst
View file @ 3009dc27
......@@ -15,6 +15,8 @@ All objects stored in the DARIAH-DE Repository are indexed within and are delive
API Documentation
-----------------
DARIAH-DE Repository OAI-PMH base URL: https://repository.de.dariah.eu/1.0/oaipmh/oai
All requests are implemented to be consistent with the OAI-PMH `The Open Archives Initiative Protocol for Metadata Harvesting <https://www.openarchives.org/OAI/openarchivesprotocol.html>`_
Two metadata formats are provided for the DARIAH-DE Repository: The mandatory DC metadata format (*oai_dc*) and the enhanced DataCite format according to `OpenAIRE Guidelines for Data Archives <https://guidelines.openaire.eu/en/latest/data/index.html>`_ (*oai_datacite*).
......@@ -23,53 +25,42 @@ Two metadata formats are provided for the DARIAH-DE Repository: The mandatory DC
Query Examples
--------------
Get the OAI-PMH service's version:
<https://repository.de.dariah.eu/1.0/oaipmh/oai/version>
[Identify] Get basic information:
<https://repository.de.dariah.eu/1.0/oaipmh/oai?verb=Identify>
[ListMetadataFormats] Get provided metadata formats:
<https://repository.de.dariah.eu/1.0/oaipmh/oai?verb=ListMetadataFormats>
[ListSets] List all public TextGrid projects as OAI-POMH sets:
<https://repository.de.dariah.eu/1.0/oaipmh/oai?verb=ListSets>
[ListRecords] Get all Edition records (*oai_dc*):
<https://repository.de.dariah.eu/1.0/oaipmh/oai?verb=ListRecords&metadataPrefix=oai_dc>
[ListRecords] Get all Edition records (*oai_datacite*):
<https://repository.de.dariah.eu/1.0/oaipmh/oai?verb=ListRecords&metadataPrefix=oai_datacite>
Get the OAI-PMH service's version: https://repository.de.dariah.eu/1.0/oaipmh/oai/version
[ListRecords] Get all Edition records of a set (*oai_dc*):
Identify
""""""""
<https://repository.de.dariah.eu/1.0/oaipmh/oai?verb=ListRecords&set=hdl:21.11113/0000-000B-C8F2-2&metadataPrefix=oai_dc>
* Get basic information: https://repository.de.dariah.eu/1.0/oaipmh/oai?verb=Identify
[ListIdentifiers] Get all Edition records (*oai_dc*):
ListMetadataFormats
"""""""""""""""""""
<https://repository.de.dariah.eu/1.0/oaipmh/oai?verb=ListIdentifiers&metadataPrefix=oai_dc>
* Get provided metadata formats: https://repository.de.dariah.eu/1.0/oaipmh/oai?verb=ListMetadataFormats
[ListIdentifiers] Get all Edition record identifiers (*oai_datacite*):
ListSets
""""""""
<https://repository.de.dariah.eu/1.0/oaipmh/oai?verb=ListIdentifiers&metadataPrefix=oai_datacite>
* List all DARIAH-DE Repository collections as OAI-POMH sets: https://repository.de.dariah.eu/1.0/oaipmh/oai?verb=ListSets
[ListIdentifiers] Get all Edition record identifiers of a set (*oai_datacite*):
ListRecords
"""""""""""
<https://repository.de.dariah.eu/1.0/oaipmh/oai?verb=ListIdentifiers&set=hdl:21.11113/0000-000B-C8F2-2&metadataPrefix=oai_dc>
* Get all records (*oai_dc*): https://repository.de.dariah.eu/1.0/oaipmh/oai?verb=ListRecords&metadataPrefix=oai_dc
* Get all records (*oai_datacite*): https://repository.de.dariah.eu/1.0/oaipmh/oai?verb=ListRecords&metadataPrefix=oai_datacite
* Get all records of a set (*oai_dc*): https://repository.de.dariah.eu/1.0/oaipmh/oai?verb=ListRecords&set=hdl:21.11113/0000-000B-C8F2-2&metadataPrefix=oai_dc
[GetRecord] Get a single Edition record (*oai_dc):
ListIdentifiers
"""""""""""""""
<https://repository.de.dariah.eu/1.0/oaipmh/oai?verb=GetRecord&identifier=hdl:21.11113/0000-000B-C9ED-8&metadataPrefix=oai_dc>
* Get all record identifiers (*oai_dc*): https://repository.de.dariah.eu/1.0/oaipmh/oai?verb=ListIdentifiers&metadataPrefix=oai_dc
* Get all record identifiers (*oai_datacite*): https://repository.de.dariah.eu/1.0/oaipmh/oai?verb=ListIdentifiers&metadataPrefix=oai_datacite
* Get all record identifiers of a set (*oai_datacite*): https://repository.de.dariah.eu/1.0/oaipmh/oai?verb=ListIdentifiers&set=hdl:21.11113/0000-000B-C8F2-2&metadataPrefix=oai_dc
[GetRecord] Get a single Edition record (*oai_datacite):
GetRecord
"""""""""
<https://repository.de.dariah.eu/1.0/oaipmh/oai?verb=GetRecord&identifier=hdl:21.11113/0000-000B-C9ED-8&metadataPrefix=oai_datacite>
* Get a single record (*oai_dc): https://repository.de.dariah.eu/1.0/oaipmh/oai?verb=GetRecord&identifier=hdl:21.11113/0000-000B-C9ED-8&metadataPrefix=oai_dc
* Get a single Edition record (*oai_datacite): https://repository.de.dariah.eu/1.0/oaipmh/oai?verb=GetRecord&identifier=hdl:21.11113/0000-000B-C9ED-8&metadataPrefix=oai_datacite
Sources
......
docs_tgrep/index.rst
View file @ 3009dc27
......@@ -6,14 +6,16 @@
TextGrid OAI-PMH
================
The TextGrid OAI-PMH Service is the service to harvest all metadata from the collections stored in the TextGrid Repository (public objects only). For example the `DARIAH-DE Generic Search <https://search.de.dariah.eu>`_ can index all the data that is entered into the `DARIAH-DE Collection Registry <https://colreg.de.dariah.eu>`_.
The TextGrid OAI-PMH Service is the service to harvest all metadata from the editions stored in the TextGrid Repository (public editions only). For example the `DARIAH-DE Generic Search <https://search.de.dariah.eu>`_ can index all the data that is entered into the `DARIAH-DE Collection Registry <https://colreg.de.dariah.eu>`_.
The TextGrid OAI-PMH Service is delivering all public projects as OAI-PMH sets (no objects from the sandbox at the moment), and all Edition objects within the projects. You can get to the project's data by using the TextGrid Search service, please see `TG-search <https://textgridlab.org/doc/services/submodules/tg-search/docs/index.html> `_, or the `TextGrid Aggregator <https://textgridlab.org/doc/services/submodules/aggregator/docs/index.html>`_.
The TextGrid OAI-PMH Service is delivering all public projects as OAI-PMH sets (no objects from the sandbox at the moment), and all Edition objects within the projects. You can get to the project's data by using the TextGrid Search service, please see `TG-search <https://textgridlab.org/doc/services/submodules/tg-search/docs/index.html>`_, or the `TextGrid Aggregator <https://textgridlab.org/doc/services/submodules/aggregator/docs/index.html>`_.
API Documentation
-----------------
TextGrid Repository OAI-PMH base URL: https://textgridlab.org/1.0/tgoaipmh/oai
All requests are implemented to be consistent with the OAI-PMH `The Open Archives Initiative Protocol for Metadata Harvesting <https://www.openarchives.org/OAI/openarchivesprotocol.html>`_
Two metadata formats are provided for the TextGrid Repository: The mandatory DC metadata format (*oai_dc*) and the enhanced DataCite format according to `OpenAIRE Guidelines for Data Archives <https://guidelines.openaire.eu/en/latest/data/index.html>`_ (*oai_datacite*).
......@@ -22,53 +24,40 @@ Two metadata formats are provided for the TextGrid Repository: The mandatory DC
Query Examples
--------------
Get the OAI-PMH service's version:
<https://textgridlab.org/1.0/tgoaipmh/oai/version>
[Identify] Get basic information:
<https://textgridlab.org/1.0/tgoaipmh/oai?verb=Identify>
[ListMetadataFormats] Get provided metadata formats:
<https://textgridlab.org/1.0/tgoaipmh/oai?verb=ListMetadataFormats>
[ListSets] List all public TextGrid projects as OAI-POMH sets:
<https://textgridlab.org/1.0/tgoaipmh/oai?verb=ListSets>
[ListRecords] Get all Edition records (*oai_dc*):
<https://textgridlab.org/1.0/tgoaipmh/oai?verb=ListRecords&metadataPrefix=oai_dc>
[ListRecords] Get all Edition records (*oai_datacite*):
<https://textgridlab.org/1.0/tgoaipmh/oai?verb=ListRecords&metadataPrefix=oai_datacite>
[ListRecords] Get all Edition records of a set (*oai_dc*):
<https://textgridlab.org/1.0/tgoaipmh/oai?verb=ListRecords&set=project:TGPR-8b44ca41-6fa1-9b49-67b7-6374d97e29eb&metadataPrefix=oai_dc>
Get the OAI-PMH service's version: https://textgridlab.org/1.0/tgoaipmh/oai/version
[ListIdentifiers] Get all Edition records (*oai_dc*):
Identify
""""""""
<https://textgridlab.org/1.0/tgoaipmh/oai?verb=ListIdentifiers&metadataPrefix=oai_dc>
* Get basic information: https://textgridlab.org/1.0/tgoaipmh/oai?verb=Identify
[ListIdentifiers] Get all Edition record identifiers (*oai_datacite*):
ListMetadataFormats
"""""""""""""""""""
* Get provided metadata formats: https://textgridlab.org/1.0/tgoaipmh/oai?verb=ListMetadataFormats
<https://textgridlab.org/1.0/tgoaipmh/oai?verb=ListIdentifiers&metadataPrefix=oai_datacite>
ListSets
""""""""
* List all public TextGrid projects as OAI-POMH sets: https://textgridlab.org/1.0/tgoaipmh/oai?verb=ListSets
[ListIdentifiers] Get all Edition record identifiers of a set (*oai_datacite*):
ListRecords
"""""""""""
<https://textgridlab.org/1.0/tgoaipmh/oai?verb=ListIdentifiers&set=project:TGPR-8b44ca41-6fa1-9b49-67b7-6374d97e29eb&metadataPrefix=oai_dc>
* Get all Edition records (*oai_dc*): https://textgridlab.org/1.0/tgoaipmh/oai?verb=ListRecords&metadataPrefix=oai_dc
* Get all Edition records (*oai_datacite*): https://textgridlab.org/1.0/tgoaipmh/oai?verb=ListRecords&metadataPrefix=oai_datacite
* Get all Edition records of a set (*oai_dc*): https://textgridlab.org/1.0/tgoaipmh/oai?verb=ListRecords&set=project:TGPR-8b44ca41-6fa1-9b49-67b7-6374d97e29eb&metadataPrefix=oai_dc
[GetRecord] Get a single Edition record (*oai_dc):
ListIdentifiers
"""""""""""""""
<https://textgridlab.org/1.0/tgoaipmh/oai?verb=GetRecord&identifier=textgrid:40wr9.0&metadataPrefix=oai_dc>
* Get all Edition records (*oai_dc*): https://textgridlab.org/1.0/tgoaipmh/oai?verb=ListIdentifiers&metadataPrefix=oai_dc
* Get all Edition record identifiers (*oai_datacite*):https://textgridlab.org/1.0/tgoaipmh/oai?verb=ListIdentifiers&metadataPrefix=oai_datacite
* Get all Edition record identifiers of a set (*oai_datacite*): https://textgridlab.org/1.0/tgoaipmh/oai?verb=ListIdentifiers&set=project:TGPR-8b44ca41-6fa1-9b49-67b7-6374d97e29eb&metadataPrefix=oai_dc
[GetRecord] Get a single Edition record (*oai_datacite):
GetRecord
"""""""""
<https://textgridlab.org/1.0/tgoaipmh/oai?verb=GetRecord&identifier=textgrid:40wr9.0&metadataPrefix=oai_datacite>
* Get a single Edition record (*oai_dc*): https://textgridlab.org/1.0/tgoaipmh/oai?verb=GetRecord&identifier=textgrid:40wr9.0&metadataPrefix=oai_dc
* Get a single Edition record (*oai_datacite*): https://textgridlab.org/1.0/tgoaipmh/oai?verb=GetRecord&identifier=textgrid:40wr9.0&metadataPrefix=oai_datacite
Sources
......
oaipmh-core/pom.xml
View file @ 3009dc27
......@@ -5,7 +5,7 @@
<parent>
<artifactId>oaipmh</artifactId>
<groupId>info.textgrid.middleware</groupId>
<version>6.2.2</version>
<version>6.2.3</version>
</parent>
<artifactId>oaipmh-core</artifactId>
<packaging>jar</packaging>
......@@ -27,11 +27,6 @@
<artifactId>httpclients</artifactId>
<version>${common.version}</version>
</dependency>
<dependency>
<groupId>info.textgrid.middleware</groupId>
<artifactId>ltputils</artifactId>
<version>${common.version}</version>
</dependency>
<dependency>
<groupId>info.textgrid.middleware.clients</groupId>
<artifactId>textgrid-clients</artifactId>
......
oaipmh-core/src/main/java/info/textgrid/middleware/DublinCoreFieldLoader.java
View file @ 3009dc27
package info.textgrid.middleware;
import java.text.ParseException;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
......
oaipmh-core/src/main/java/info/textgrid/middleware/IdentifierListDelivererAbstract.java
View file @ 3009dc27
......@@ -123,7 +123,7 @@ public abstract class IdentifierListDelivererAbstract implements IdentifierListD
public SearchResponse hitHandling(SearchResponse listFurtherValues, ListIdentifiersType lit,
String set, String resumptionToken, Map<String, Integer> cursorCollector) {
long size = listFurtherValues.getHits().totalHits;
long size = listFurtherValues.getHits().getTotalHits().value;
setResultSize(size);
for (SearchHit hit : listFurtherValues.getHits().getHits()) {
......@@ -147,7 +147,7 @@ public abstract class IdentifierListDelivererAbstract implements IdentifierListD
// Check the need for a resumption token!
ResumptionTokenType responseToken = OaipmhUtilities.getResumptionToken(
listFurtherValues.getHits().getTotalHits(), resumptionToken, cursorCollector,
listFurtherValues.getHits().getTotalHits().value, resumptionToken, cursorCollector,
listFurtherValues.getScrollId(), this.searchResponseSize);
if (responseToken != null) {
lit.setResumptionToken(responseToken);
......
oaipmh-core/src/main/java/info/textgrid/middleware/IdentifierListDelivererIdiom.java
View file @ 3009dc27
......@@ -135,7 +135,7 @@ public class IdentifierListDelivererIdiom extends IdentifierListDelivererAbstrac
String scrollID = scrollResp.getScrollId();
long completeListSize = scrollResp.getHits().totalHits;
long completeListSize = scrollResp.getHits().getTotalHits().value;
log.fine("complete list size: " + completeListSize);
......
oaipmh-core/src/main/java/info/textgrid/middleware/MetadataFormatListDelivererTG.java
View file @ 3009dc27
......@@ -35,9 +35,8 @@ public class MetadataFormatListDelivererTG extends MetadataFormatListDelivererAb
FetchSourceContext fetchSourceContext = new FetchSourceContext(true, includes, excludes);
GetRequest getRequest =
new GetRequest(this.oaiEsClient.getEsIndex(), this.oaiEsClient.getEsType(),
id.replace("textgrid:", "")).fetchSourceContext(fetchSourceContext);
new GetRequest(this.oaiEsClient.getEsIndex(), id.replace("textgrid:", ""))
.fetchSourceContext(fetchSourceContext);
GetResponse tgObject = null;
try {
tgObject = this.oaiEsClient.getEsClient().get(getRequest, RequestOptions.DEFAULT);
......
oaipmh-core/src/main/java/info/textgrid/middleware/OaipmhElasticSearchClient.java
View file @ 3009dc27
......@@ -13,16 +13,14 @@ import org.elasticsearch.client.RestHighLevelClient;
*
* @author Maximilian Brodhun, SUB Göttingen
* @author Stefan E. Funk, SUB Göttingen
* @version 2022-09-08
* @version 2023-09-06
* @since 2014-01-09
*/
public class OaipmhElasticSearchClient {
private String url;
private int[] ports;
private String esCluster;
private String esIndex;
private String esType;
private RestHighLevelClient esClient;
/**
......@@ -62,34 +60,6 @@ public class OaipmhElasticSearchClient {
this.esIndex = esIndex;
}
/**
* @return esType
*/
public String getEsType() {
return this.esType;
}
/**
* @param esType
*/
public void setEsType(String esType) {
this.esType = esType;
}
/**
* @return esCluster
*/
public String getClusterName() {
return this.esCluster;
}
/**
* @param esCluster
*/
public void setClusterName(String esCluster) {
this.esCluster = esCluster;
}
/**
* @return url
*/
......@@ -107,7 +77,7 @@ public class OaipmhElasticSearchClient {
/**
* @return port
*/
public int[] getPort() {
public int[] getPorts() {
return this.ports;
}
......
oaipmh-core/src/main/java/info/textgrid/middleware/OaipmhUtilities.java
View file @ 3009dc27
......@@ -445,9 +445,8 @@ public class OaipmhUtilities {
}
// Building the getRequest against the elastic search index.
GetRequest getRequest =
new GetRequest(theESClient.getEsIndex(), theESClient.getEsType(), idInElasticSearchIndex)
.fetchSourceContext(fetchSourceContext);
GetRequest getRequest = new GetRequest(theESClient.getEsIndex(), idInElasticSearchIndex)
.fetchSourceContext(fetchSourceContext);
log.fine("get request id: " + getRequest.id());
......
oaipmh-core/src/main/java/info/textgrid/middleware/RecordDelivererDatacite.java
View file @ 3009dc27
......@@ -9,7 +9,6 @@ import org.elasticsearch.action.get.GetResponse;
import org.elasticsearch.common.Strings;
import org.json.JSONObject;
import org.springframework.stereotype.Component;
import info.textgrid.middleware.common.LTPUtils;
import info.textgrid.middleware.common.RDFConstants;
import info.textgrid.middleware.common.TextGridMimetypes;
import info.textgrid.middleware.oaipmh.ContributorType;
......@@ -492,10 +491,10 @@ public class RecordDelivererDatacite extends RecordDelivererAbstract {
RelatedIdentifier relatedID = new RelatedIdentifier();
if (i.startsWith(RDFConstants.HDL_PREFIX)) {
relatedID.setRelatedIdentifierType(RelatedIdentifierType.HANDLE);
relatedID.setValue(LTPUtils.omitHdlPrefix(i));
relatedID.setValue(omitHdlPrefix(i));
} else if (i.startsWith(RDFConstants.DOI_PREFIX)) {
relatedID.setRelatedIdentifierType(RelatedIdentifierType.DOI);
relatedID.setValue(LTPUtils.omitDoiPrefix(i));
relatedID.setValue(omitDoiPrefix(i));
} else if (i.startsWith("http")) {
relatedID.setRelatedIdentifierType(RelatedIdentifierType.URL);
}
......@@ -685,10 +684,10 @@ public class RecordDelivererDatacite extends RecordDelivererAbstract {
AlternateIdentifier alternateID = new AlternateIdentifier();
if (i.startsWith(RDFConstants.HDL_PREFIX)) {
alternateID.setAlternateIdentifierType(ID_TYPE_HANDLE);
alternateID.setValue(LTPUtils.omitHdlPrefix(i));
alternateID.setValue(omitHdlPrefix(i));
} else if (i.startsWith(RDFConstants.DOI_PREFIX)) {
alternateID.setAlternateIdentifierType(ID_TYPE_DOI);
alternateID.setValue(LTPUtils.omitDoiPrefix(i));
alternateID.setValue(omitDoiPrefix(i));
} else {
alternateID.setAlternateIdentifierType(ID_TYPE_OTHER);
alternateID.setValue(i);
......@@ -859,7 +858,7 @@ public class RecordDelivererDatacite extends RecordDelivererAbstract {
// Datacite schema validation!
if (idValue != null && !idValue.isEmpty()) {
identifier.setIdentifierType(ID_TYPE_HANDLE);
identifier.setValue(LTPUtils.omitHdlPrefix(idValue));
identifier.setValue(omitHdlPrefix(idValue));
}
}
......@@ -870,7 +869,7 @@ public class RecordDelivererDatacite extends RecordDelivererAbstract {
else if (this.dariah) {
// We set DOI for DH, 'cause we HAVE it! :-D
identifier.setIdentifierType(ID_TYPE_DOI);
identifier.setValue(LTPUtils.omitDoiPrefix(idValue));
identifier.setValue(omitDoiPrefix(idValue));
}
return identifier;
......@@ -1009,6 +1008,51 @@ public class RecordDelivererDatacite extends RecordDelivererAbstract {
.getYear());
}
/**
* <p>
* Removes the "doi:" from the given DOI, if existing.
* </p>
*
* @param theDOI
* @return
*/
private static String omitDoiPrefix(final String theDOI) {
return omitPrefix(theDOI, RDFConstants.DOI_PREFIX);
}
/**
* <p>
* Removes the "hdl:" from the given PID, if existing.
* </p>
*
* @param theHDL
* @return
*/
private static String omitHdlPrefix(final String theHDL) {
return omitPrefix(theHDL, RDFConstants.HDL_PREFIX);
}
/**
* <p>
* Removes a prefix, including the ":" from the beginning of a string.
* </p>
*
* @param theURI
* @param thePrefix
* @return
*/
private static String omitPrefix(final String theURI, final String thePrefix) {
String result = theURI;
if (theURI != null && theURI.startsWith(thePrefix + ":")) {
result = theURI.replaceFirst(thePrefix + ":", "");
}
return result;
}
// **
// GETTERS & SETTERS
// **
......
oaipmh-core/src/main/java/info/textgrid/middleware/RecordListDelivererDC.java
View file @ 3009dc27
......@@ -101,9 +101,8 @@ public class RecordListDelivererDC extends RecordListDelivererAbstract {
String[] includes = this.workFields;
String[] excludes = Strings.EMPTY_ARRAY;
FetchSourceContext fetchSourceContext = new FetchSourceContext(true, includes, excludes);
GetRequest getRequest =
new GetRequest(this.oaiEsClient.getEsIndex(), this.oaiEsClient.getEsType(), identifier)
.fetchSourceContext(fetchSourceContext);
GetRequest getRequest = new GetRequest(this.oaiEsClient.getEsIndex(), identifier)
.fetchSourceContext(fetchSourceContext);
GetResponse responseWorkValues = null;
try {
......@@ -195,7 +194,7 @@ public class RecordListDelivererDC extends RecordListDelivererAbstract {
scrollID = scrollResp.getScrollId();
long completeListSize = scrollResp.getHits().totalHits;
long completeListSize = scrollResp.getHits().getTotalHits().value;
setResultSize(completeListSize);
if (completeListSize > 0) {
......
oaipmh-core/src/main/java/info/textgrid/middleware/RecordListDelivererDatacite.java
View file @ 3009dc27
......@@ -175,31 +175,27 @@ public class RecordListDelivererDatacite extends RecordListDelivererAbstract {
searchSourceBuilder.size(this.searchResponseSize);
searchRequest.source(searchSourceBuilder);
SearchResponse scrollResp = new SearchResponse();
if (resumptionToken != null) {
SearchScrollRequest scrollRequest = new SearchScrollRequest(resumptionToken);
scrollRequest.scroll(TimeValue.timeValueHours(24L));
try {
SearchResponse scrollResp = null;
// SearchResponse scrollResp = new SearchResponse(null);
try {
if (resumptionToken != null) {
SearchScrollRequest scrollRequest = new SearchScrollRequest(resumptionToken);
scrollRequest.scroll(TimeValue.timeValueHours(24L));
scrollResp = this.oaiEsClient.getEsClient().scroll(scrollRequest, RequestOptions.DEFAULT);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
} else {
searchRequest.source(searchSourceBuilder);
searchRequest.scroll(TimeValue.timeValueHours(24L));
try {
} else {
searchRequest.source(searchSourceBuilder);
searchRequest.scroll(TimeValue.timeValueHours(24L));
scrollResp = this.oaiEsClient.getEsClient().search(searchRequest, RequestOptions.DEFAULT);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
// TODO Check if scrollResp == null and what to do then!
String scrollID = scrollResp.getScrollId();
long completeListSize = scrollResp.getHits().totalHits;
long completeListSize = scrollResp.getHits().getTotalHits().value;
setResultSize(completeListSize);
if (completeListSize > 0) {
......@@ -223,6 +219,7 @@ public class RecordListDelivererDatacite extends RecordListDelivererAbstract {
} else {
id2add = hit.getId();
}
uriList.add(id2add);
}
}
......
oaipmh-core/src/main/java/info/textgrid/middleware/RecordListDelivererIdiom.java
View file @ 3009dc27
......@@ -135,7 +135,7 @@ public class RecordListDelivererIdiom extends RecordListDelivererAbstract {
String scrollID = scrollResp.getScrollId();
long completeListSize = scrollResp.getHits().totalHits;
long completeListSize = scrollResp.getHits().getTotalHits().value;
log.fine("total hits: " + completeListSize);
......
oaipmh-core/src/test/java/info/textgrid/middleware/test/TestDHOaipmhLocally.java
View file @ 3009dc27
......@@ -133,7 +133,6 @@ public class TestDHOaipmhLocally {
int[] ports = new int[] {9202};
oaiEsClient = new OaipmhElasticSearchClient("localhost", ports, 100);
oaiEsClient.setEsIndex("dariah-public");
oaiEsClient.setEsType("metadata");
record = new RecordDelivererDC(false, true);
record.setOaiEsClient(oaiEsClient);
......
oaipmh-core/src/test/java/info/textgrid/middleware/test/TestTGOaipmhLocally.java
View file @ 3009dc27
......@@ -161,7 +161,6 @@ public class TestTGOaipmhLocally {
int[] ports = new int[] {9202};
oaiEsClient = new OaipmhElasticSearchClient("localhost", ports, 100);
oaiEsClient.setEsIndex("textgrid-public");
oaiEsClient.setEsType("metadata");
record = new RecordDelivererDC(true, false);
record.setOaiEsClient(oaiEsClient);
......
oaipmh-webapp/pom.xml
View file @ 3009dc27
......@@ -5,7 +5,7 @@
<parent>
<artifactId>oaipmh</artifactId>
<groupId>info.textgrid.middleware</groupId>
<version>6.2.2</version>
<version>6.2.3</version>
</parent>
<artifactId>oaipmh-webapp</artifactId>
<packaging>war</packaging>
......
oaipmh-webapp/src/main/webapp/WEB-INF/beans.xml
View file @ 3009dc27
......@@ -41,7 +41,6 @@
<constructor-arg index="2"
value="${elasticSearch.itemLimit}" />
<property name="esIndex" value="${elasticSearch.index}" />
<property name="esType" value="${elasticSearch.type}" />
</bean>
<bean id="NonpublicElasticSearchClient"
......@@ -54,7 +53,6 @@
value="${elasticSearch.itemLimit}" />
<property name="esIndex"
value="${elasticSearch.nonpublicIndex}" />
<property name="esType" value="${elasticSearch.type}" />
</bean>
<!-- Main class bean -->
......
oaipmh-webapp/src/main/webapp/WEB-INF/oaipmh.textgrid.properties
View file @ 3009dc27
......@@ -11,7 +11,6 @@ RS_ENDPOINT = https://dev.textgridlab.org/1.0/tgoaipmh
elasticSearch.url = searchindex-main
elasticSearch.ports = 9200
elasticSearch.index = ***
elasticSearch.type = metadata
elasticSearch.itemLimit = 100
# NONPUBLIC index only TextGrid IDIOM queries!
elasticSearch.nonpublicIndex = ***
......
pom.xml
View file @ 3009dc27
......@@ -5,7 +5,7 @@
<modelVersion>4.0.0</modelVersion>
<groupId>info.textgrid.middleware</groupId>
<artifactId>oaipmh</artifactId>
<version>6.2.2</version>
<version>6.2.3</version>
<packaging>pom</packaging>
<name>DARIAHDE :: OAI-PMH DataProvider</name>
<properties>
......@@ -13,10 +13,10 @@
<jdk.version>1.8</jdk.version>
<antlr-maven-plugin.version>3.2</antlr-maven-plugin.version>
<antlr-runtime.version>3.2</antlr-runtime.version>
<common.version>4.4.1</common.version>
<common.version>5.0.0</common.version>
<cxf.version>3.3.13</cxf.version>
<cyclonedx-maven-plugin.version>2.7.0</cyclonedx-maven-plugin.version>
<elasticsearch.version>6.5.4</elasticsearch.version>
<elasticsearch.version>7.9.3</elasticsearch.version>
<jaxb2-namespace-prefix.version>1.3</jaxb2-namespace-prefix.version>
<jetty-maven-plugin.version>9.4.12.v20180830</jetty-maven-plugin.version>
<junit.version>4.13.1</junit.version>
......@@ -26,7 +26,7 @@
<maven-deploy-plugin.version>2.8.2</maven-deploy-plugin.version>
<maven-eclipse-plugin.version>2.10</maven-eclipse-plugin.version>
<maven-jaxb2-plugin.version>0.15.2</maven-jaxb2-plugin.version>
<mets-mods-mapping.version>2.4.1</mets-mods-mapping.version>
<mets-mods-mapping.version>2.4.2</mets-mods-mapping.version>
<package-info-maven-plugin.version>1.4.5</package-info-maven-plugin.version>
<properties-maven-plugin.version>1.0-alpha-2</properties-maven-plugin.version>
<rdf4j-repository-api.version>3.0.2</rdf4j-repository-api.version>
......