Commit 5f9448c8 authored by j.hoerdt's avatar j.hoerdt
Browse files

Merge branch...

Merge branch '3-improve-graphdb-model-by-splitting-off-measurementlocation-from-sensor-nodes' into 'master'

Resolve "Improve GraphDB model by splitting off MeasurementLocation from Sensor nodes"

Closes #9, #10, and #3

See merge request !7
parents d48783af d90dbbab
# sensor2graph
[![pipeline status](https://gitlab.gwdg.de/j.hoerdt/sensor2graph/badges/master/pipeline.svg)](https://gitlab.gwdg.de/j.hoerdt/sensor2graph/-/commits/master)
This project aims to deliver metadata about the sensors involved in the open [sensor.community](https://sensor.community) project in the form of the graph database [neo4j](https://neo4j.com).
The latest artifacts can be browsed [here](https://gitlab.gwdg.de/j.hoerdt/sensor2graph/-/jobs/artifacts/master/browse?job=buildjarfile).
## usage instructions:
### executing the latest binaries:
#### via docker image:
1. Make sure to have [docker installed](https://docs.docker.com/engine/install/) on your system.
2. Run the program by executing the following command in the command line:
```
docker run --rm -d -v sensor2graph_data:/data docker.gitlab.gwdg.de/j.hoerdt/sensor2graph:master <username> <password>
```
In the command, replace `<username>` and `<password>` by your username and password for the neo4j database respectively.
#### via jar file:
1. Download the latest artifacts [here](https://gitlab.gwdg.de/j.hoerdt/sensor2graph/-/jobs/artifacts/master/download?job=buildjarfile).
1. Execute
**using docker:**
1. Make sure to have [docker installed](https://docs.docker.com/engine/install/) on your system.
1. Run this command in the root of the artifacts archive:
```
docker run --rm -u gradle -v "$PWD":/home/gradle/project -w /home/gradle/project gradle:jre14 java -Djava.util.logging.config.file=logging.properties -jar build/libs/sensor2graph.jar <username> <password>
```
**using the system's jre:**
1. <a name="installjdk"></a>Install a java jre on your system if you do not already have one. Java versions 11 and 13 are known to work. Installation instructions for OpenJDK are [here](https://openjdk.java.net/install/).
1. <a name="runjar"></a>Run the program using the following command from the root of the directory:
```
java -Djava.util.logging.config.file=logging.properties -jar build/libs/sensor2graph.jar <username> <password>
```
In the command, replace `<username>` and `<password>` by your username and password for the neo4j database respectively.
### building the docker image from source
1. Make sure to have [docker installed](https://docs.docker.com/engine/install/) on your system.
2. In the repository, run the following command:
```
docker build -t sensor2graph .
```
3. The container can be executed with the following command:
```
docker run --rm -d -v sensor2graph_data:/data sensor2graph <username> <password>
```
In the command, replace `<username>` and `<password>` by your username and password for the neo4j database respectively.
### building the jarfile from source :
#### with docker:
1. If docker is installed on the system you can run
```
docker run --rm -u gradle -v "$PWD":/home/gradle/project -w /home/gradle/project gradle:jdk14 gradle jar
```
#### without docker:
1. To build the project a jdk is needed. Refer to [this point 2.](#installjdk) in "via jar file:" for more.
1. Then, after you have cloned this repository, create a fat jar file by running the following command in the repository's root:
```
./gradlew jar
```
This is the [gradle wrapper](https://docs.gradle.org/current/userguide/gradle_wrapper.html) which automatically installs the correct gradle version into your home directory and creates the jar file.
The internal api docs can be built with `./gradlew javadoc`.
1. Proceed with [point 2 of via jar file](#runjar).
## Developers:
### Updating dependencies:
Look for new versions here:
- https://search.maven.org/search?q=g:com.google.code.gson%20AND%20a:gson
- https://search.maven.org/search?q=g:org.neo4j.driver%20AND%20a:neo4j-java-driver
and update the entries in the `build.gradle` file.
# sensor2graph
[![pipeline status](https://gitlab.gwdg.de/j.hoerdt/sensor2graph/badges/master/pipeline.svg)](https://gitlab.gwdg.de/j.hoerdt/sensor2graph/-/commits/master)
This project aims to deliver metadata about the sensors involved in the open [sensor.community](https://sensor.community) project in the form of the graph database [neo4j](https://neo4j.com).
The latest artifacts can be browsed [here](https://gitlab.gwdg.de/j.hoerdt/sensor2graph/-/jobs/artifacts/master/browse?job=buildjarfile).
## usage instructions:
### executing the latest binaries:
#### via docker image:
1. Make sure to have [docker installed](https://docs.docker.com/engine/install/) on your system.
2. Run the program by executing the following command in the command line:
```
docker run --rm -d -v sensor2graph_data:/data docker.gitlab.gwdg.de/j.hoerdt/sensor2graph:master <username> <password>
```
In the command, replace `<username>` and `<password>` by your username and password for the neo4j database respectively.
#### via jar file:
1. Download the latest artifacts [here](https://gitlab.gwdg.de/j.hoerdt/sensor2graph/-/jobs/artifacts/master/download?job=buildjarfile).
1. Execute
**using docker:**
1. Make sure to have [docker installed](https://docs.docker.com/engine/install/) on your system.
1. Run this command in the root of the artifacts archive:
```
docker run --rm -u gradle -v "$PWD":/home/gradle/project -w /home/gradle/project gradle:jre14 java -Djava.util.logging.config.file=logging.properties -jar build/libs/sensor2graph.jar <username> <password>
```
**using the system's jre:**
1. <a name="installjdk"></a>Install a java jre on your system if you do not already have one. Java versions 11 and 13 are known to work. Installation instructions for OpenJDK are [here](https://openjdk.java.net/install/).
1. <a name="runjar"></a>Run the program using the following command from the root of the directory:
```
java -Djava.util.logging.config.file=logging.properties -jar build/libs/sensor2graph.jar <username> <password>
```
In the command, replace `<username>` and `<password>` by your username and password for the neo4j database respectively.
### building the docker image from source
1. Make sure to have [docker installed](https://docs.docker.com/engine/install/) on your system.
2. In the repository, run the following command:
```
docker build -t sensor2graph .
```
3. The container can be executed with the following command:
```
docker run --rm -d -v sensor2graph_data:/data sensor2graph <username> <password>
```
In the command, replace `<username>` and `<password>` by your username and password for the neo4j database respectively.
### building the jarfile from source :
#### with docker:
1. If docker is installed on the system you can run
```
docker run --rm -u gradle -v "$PWD":/home/gradle/project -w /home/gradle/project gradle:jdk14 gradle jar
```
#### without docker:
1. To build the project a jdk is needed. Refer to [this point 2.](#installjdk) in "via jar file:" for more.
1. Then, after you have cloned this repository, create a fat jar file by running the following command in the repository's root:
```
./gradlew jar
```
This is the [gradle wrapper](https://docs.gradle.org/current/userguide/gradle_wrapper.html) which automatically installs the correct gradle version into your home directory and creates the jar file.
The internal api docs can be built with `./gradlew javadoc`.
1. Proceed with [point 2 of via jar file](#runjar).
## Developers:
### Updating dependencies:
Look for new versions here:
- https://search.maven.org/search?q=g:com.google.code.gson%20AND%20a:gson
- https://search.maven.org/search?q=g:org.neo4j.driver%20AND%20a:neo4j-java-driver
and update the entries in the `build.gradle` file.
### Updating gradle and gradle wrapper
1. Check https://gradle.org/releases/.
2. `gradle wrapper --gradle-version <version>`
3. commit
\ No newline at end of file
plugins {
id 'java'
id 'application'
}
repositories {
jcenter()
}
dependencies {
testImplementation 'junit:junit:4.12'
implementation 'org.neo4j.driver:neo4j-java-driver:4.1.1'
implementation 'com.google.code.gson:gson:2.8.6'
}
mainClassName = 'sensor2graph.Main'
jar {
manifest {
attributes 'Main-Class': 'sensor2graph.Main'
}
from {
configurations.runtimeClasspath.collect { it.isDirectory() ? it : zipTree(it) }
}
plugins {
id 'java'
id 'application'
}
repositories {
jcenter()
}
dependencies {
testImplementation 'junit:junit:4.12'
implementation 'org.neo4j.driver:neo4j-java-driver:4.2.1'
implementation 'com.google.code.gson:gson:2.8.6'
}
mainClassName = 'sensor2graph.Main'
jar {
manifest {
attributes 'Main-Class': 'sensor2graph.Main'
}
from {
configurations.runtimeClasspath.collect { it.isDirectory() ? it : zipTree(it) }
}
}
\ No newline at end of file
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-6.8.1-bin.zip
distributionUrl=https\://services.gradle.org/distributions/gradle-6.8.3-bin.zip
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists
......@@ -68,11 +68,11 @@ class Sensor {
);
add_attribute(body,
"21.T11148/22c62082a4d2d9ae2602",
"[{\"Date\":{\"date\":\"" + rec.get("s.first_msg").asLocalDate().toString() + "\",\"dateType\":\"first measurement\"}},{\"Date\":{\"date\":\"" + rec.get("s.last_msg").asLocalDate().toString() + "\",\"dateType\":\"last measurement\"}}]"
"[{\"Date\":{\"date\":\"" + rec.get("min(all_measurements.first_msg)").asLocalDate().toString() + "\",\"dateType\":\"first measurement\"}},{\"Date\":{\"date\":\"" + rec.get("max(all_measurements.last_msg)").asLocalDate().toString() + "\",\"dateType\":\"last measurement\"}}]"
);
add_attribute(body,
"21.T11148/709a23220f2c3d64d1e1",
rec.get("type.sensor_type").asString()
rec.get("sensor_type").asString()
);
Main.glogger.fine("sent body: " + body);
......@@ -123,19 +123,23 @@ class Sensor {
"merge (station:Station {location: $location })" +
//sensor merge here:
"merge (type)<-[:HAS_TYPE]-(s:Sensor {sensor_id: $sensor_id, coordinates: $coordinates})-[:BELONGS_TO]->(station)" +
"merge (type)<-[:HAS_TYPE]-(s:Sensor {sensor_id: $sensor_id})-[:BELONGS_TO]->(station)" +
"merge (s)-[:MEASURED_AT]->(measurement:MeasurementLocation {coordinates: $coordinates})" +
"merge (s)-[:IS_LOCATED]->(environment)" +
"merge (s)-[:HAS_MOBILITY]->(mobility)" +
string_if_exists.apply("country", "merge (s)-[:IN]->(country)") +
string_if_exists.apply("state", "merge (s)-[:IN]->(state)") +
string_if_exists.apply("city", "merge (s)-[:IN]->(city)") +
string_if_exists.apply("country", "merge (measurement)-[:IN]->(country)") +
string_if_exists.apply("state", "merge (measurement)-[:IN]->(state)") +
string_if_exists.apply("city", "merge (measurement)-[:IN]->(city)") +
"set" +
" s.first_msg = case when date_ > s.first_msg then s.first_msg else date_ end," +
" s.last_msg = case when date_ < s.last_msg then s.last_msg else date_ end " +
"return distinct s.pid, id(s), s.first_msg, s.last_msg, type.sensor_type"
" measurement.first_msg = case when date_ > measurement.first_msg then measurement.first_msg else date_ end," +
" measurement.last_msg = case when date_ < measurement.last_msg then measurement.last_msg else date_ end " +
"with s, type.sensor_type as sensor_type\n" +
"match (s)--(all_measurements:MeasurementLocation)" +
"return s.pid, id(s), sensor_type, min(all_measurements.first_msg), max(all_measurements.last_msg)"
).withParameters(properties);
}
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment