Skip to content
Snippets Groups Projects
Commit 9ef1a0f6 authored by Michelle Weidling's avatar Michelle Weidling :blowfish:
Browse files

Language and minor (closes #17)

parent 7f502b64
No related branches found
No related tags found
No related merge requests found
# SUB RDD Technical Reference ---
title: 'SUB RDD Technical Reference'
...
**Author**: Software Quality Working Group
# About this Document
**Author**: Software Quality Working Group\
\
**Purpose**: **Purpose**:
This guideline should help you getting started a new software development project (or improving an existing one!) in the Research and Development Department of the Göttingen State and University Library. This guideline should help you getting started a new software development project (or improving an existing one!) in the Research and Development Department of the Göttingen State and University Library.
Our goal is to establish better software quality by following standards the developer team has mutually agreed upon. Roughly basing on the [DARIAH Technical Reference](https://dariah-eric.github.io/technical-reference/), these standards are discussed, worked out, and decided in the Software Quality Working Group, which meets biweekly on Tuesdays at 12:30-13:30. However, they aren't cast in stone, so in case you have a good idea for a better standard, feel free to contribute! Our goal is to establish better software quality by following standards the developer team has mutually agreed upon. Roughly basing on the [EURISE Network Technical Reference](https://eurise-network.github.io/), these standards are discussed, worked out, and decided in the Software Quality Working Group, which meets biweekly on Tuesdays at 13:00-14:00. However, they aren't cast in stone, so in case you have a good idea for a better standard, feel free to contribute!\
\
**Status**: **Status**:
This document is a living document and will be extended as soon as the Software Quality Working Group has agreed upon a new standard for software projects in RDD. This document is a living document and will be extended as soon as the Software Quality Working Group has agreed upon a new standard for software projects in RDD.
TODOs and addenda of this document are maintained [here](https://github.com/subugoe/rdd-technical-reference/issues/).
## Guidelines # Style Guides
**Do you stick to our code style guides?**
### General ## General
The basic definitions are given by our [EditorConfig](http://editorconfig.org/), The basic definitions are given by our [EditorConfig](http://editorconfig.org/) file, `.editorconfig`,
i.e. unix line breaks and 2 space indentation. i.e. Unix line breaks and 2 space indentation.
Unfortunately, not all editors support [EditorConfig](http://editorconfig.org/). ## Specific for Programming Languages
In case you use **eXide**, the IDE that comes with [exist-db](http://exist-db.org/),
you can set 2 space indentation as default by editing `/db/apps/eXide/src/preferences.js`.
### Specific for programming languagues
For the more prominent programming languages we have formatting and general style guides we ask you to follow: For the more prominent programming languages we have formatting and general style guides we ask you to follow:
- **Java**: The Java style guide can be found [here](./styles/rdd-eclipse-java-google-style.xml). It's based on the [Google style guide for Java](https://github.com/google/styleguide) with some minor RDD specific setting. You can configure Eclipse to use it automatically at *Eclipse > Preferences > Java > Code Style > Formatter*. Just load the [RDD Eclipse Java Google Style](https://raw.githubusercontent.com/subugoe/rdd-technical-reference/master/styles/rdd-eclipse-java-google-style.xml) in the formatter preferences and use it in your RDD projects. - **Java**: The Java style guide can be found [here](./styles/rdd-eclipse-java-google-style.xml). It's based on the [Google style guide for Java](https://github.com/google/styleguide) with some minor RDD specific settings. You can configure Eclipse to use it automatically at *Eclipse > Preferences > Java > Code Style > Formatter*. Just load the [RDD Eclipse Java Google Style](https://raw.githubusercontent.com/subugoe/rdd-technical-reference/master/styles/rdd-eclipse-java-google-style.xml) in the formatter preferences and use it in your RDD projects.
- **JavaScript**: For JS we use the [Airbnb JavaScript Style Guide](https://github.com/airbnb/javascript). - **JavaScript**: For JS we use the [Airbnb JavaScript Style Guide](https://github.com/airbnb/javascript).
...@@ -35,18 +36,18 @@ For the more prominent programming languages we have formatting and general styl ...@@ -35,18 +36,18 @@ For the more prominent programming languages we have formatting and general styl
- **XQuery**: We use the [xqdoc style guide](http://xqdoc.org/xquery-style.pdf) with the following addenda: - **XQuery**: We use the [xqdoc style guide](http://xqdoc.org/xquery-style.pdf) with the following addenda:
- use double quotes instead of single quotes (for easy escaping) - use double quotes instead of single quotes (for easy escaping)
- use four spaces for a TAB (because eXide makes it so) - use four spaces for a TAB (because eXide switching the preferences in eXide's setting isn't permanent)
- **XSLT**: Since there is no official style guide for XSLT, we decided to write - **XSLT**: Since there is no official style guide for XSLT, we decided to write
[our own](https://github.com/subugoe/rdd-technical-reference/tree/master/style-guides/FE-XSLT.pdf), resulting from common best practices and own experiences within [our own](https://github.com/subugoe/rdd-technical-reference/tree/master/style-guides/FE-XSLT.pdf), resulting from common best practices and own experiences within
the department. the department.
- **Python**: For Python [PEP 8](https://www.python.org/dev/peps/pep-0008/) should be used, Django has a styleguide based on PEP-8 with some exceptions: [Django-Styleguide](https://docs.djangoproject.com/en/dev/internals/contributing/writing-code/coding-style/). There are linters and tools like [flake-8](https://pypi.org/project/flake8/) and [pep-8](https://pypi.org/project/pep8/) availabe to support. - **Python**: For Python [PEP 8](https://www.python.org/dev/peps/pep-0008/) should be used, Django has a style guide based on PEP-8 with some exceptions: [Django-Styleguide](https://docs.djangoproject.com/en/dev/internals/contributing/writing-code/coding-style/). There are linters and tools like [flake-8](https://pypi.org/project/flake8/) and [pep-8](https://pypi.org/project/pep8/) available as support.
- **SPARQL**: For SPARQL there is not really any official style guide and there is no possibility to simply include any code style automatically using a code style file. We just collect some advices how to format and use SPARQL code. - **SPARQL**: For SPARQL there is not really any official style guide and there is no possibility to simply include any code style automatically using a code style file. We just collect some advice how to format and use SPARQL code.
- declaration of variables should start with a **?** (and not with a **$**). - declaration of variables should start with a **?** (and not with a **$**).
- opening parenthesis **{** should be at the end of the line. Closing parenthesis in a separate line. - opening parenthesis **{** should be at the end of the line. Closing parenthesis in a separate line. Example:
``` ```
SELECT * WHERE { SELECT * WHERE {
...@@ -79,11 +80,15 @@ SELECT DISTINCT ...@@ -79,11 +80,15 @@ SELECT DISTINCT
``` ```
## Is your software fully documented? # Is Your Software Fully Documented?
## Docs Sprints
### General issues To ensure the best documentation of our code we can we meet on a weekly base for a code sprint to document everything we have coded throughout the week and haven't been able to document properly yet. The meeting takes place in the meeting room at 1pm. Cookies may be provided.
- don't document computer language's interna ## General
- don't document a language's specifics, e.g. operators
- best use language structure to document - best use language structure to document
...@@ -91,30 +96,31 @@ SELECT DISTINCT ...@@ -91,30 +96,31 @@ SELECT DISTINCT
- documentation and variable language is American English - documentation and variable language is American English
- should be as code-near as possible - docs should be as close to the code as possible
- every code repo must have - every code repository must have
- a README.md file that contains - a README.md file that contains
- link to original repository - link to original repository (if the software is forked or otherwise based on preexisting software)
- short introduction - short introduction on what the repository is about
- a guide how to get the software running (if makes sense)
- link to demo instance - link to demo instance
- example or demo installation - example or demo installation
- link to licence file - link to license file
- contribution guide - contribution guide
- link to style guide - link to style guide
- link to bugtracker/project managemenmt system - link to bug tracker/project management system
- known issues - known issues
- badges to ci status - badges to CI status
A good example can be found [here](https://gist.github.com/PurpleBooth/109311bb0361f32d87a2). A good example can be found [here](https://gist.github.com/PurpleBooth/109311bb0361f32d87a2).
- a LICENCE file - a LICENCE file
### Developer documentation ## Developer Documentation
#### Architecture of the software ### Architecture of the Software
Each software project should be documented using an architecture diagram that helps understanding its basic functionality (using tools to generate diagrams such as UML class diagrams seems not to be possible in every case). Each software project should be documented using an architecture diagram that helps understanding its basic functionality (even though using tools to generate diagrams such as UML class diagrams doesn't seem to be possible in every case).
Examples: Examples:
...@@ -122,25 +128,24 @@ Examples: ...@@ -122,25 +128,24 @@ Examples:
Call diagrams can be useful to follow code and service calls and should be existing for every API method. Call diagrams can be useful to follow code and service calls and should be existing for every API method.
#### API documentation ### API Documentation
- Used parameters, author and since annotations
- Example for Java: <https://lab.sub.uni-goettingen.de/self-updating-docs.html> - The docs should comprise used parameters, author and @since annotations
- Links to callers must not be listed in the documentation, because this info will be deprecated soon. Strongly recommended is using call stacks of tools like Eclipse (Java) and/or Call Graph Module (SADE). - [Example for Java](https://lab.sub.uni-goettingen.de/self-updating-docs.html)
- We do meet and write documentation together regularly (documentation sprint) every friday from 1 PM in the RDD meeting room. WE NEED COOKIES! - Links to callers must not be listed in the documentation, because this info will be deprecated soon. It is strongly recommended to use call stacks of tools like Eclipse (Java) and/or Call Graph Module (SADE).
- Document REST-APIs using [openAPI](https://github.com/OAI/OpenAPI-Specification) if possible. OpenAPI docs should be located at /doc/api on servers. - Document REST-APIs using [openAPI](https://github.com/OAI/OpenAPI-Specification) if possible. OpenAPI docs should be located at `/doc/api` on servers.
### Admin Documentation ## Admin Documentation
- how to install the software, how to run and/or restart it, how to test the installation, ... - The docs should comprise how to install the software, how to run and/or restart it, how to test the installation, ...
- server documentation - Server documentation: This type of documentation is provided and maintained in our wiki space.
We have a template encompassing all information necessary: To create a wiki page for a new server navigate to the FE Server list, select "..." right beside the "Create" button and search for "FE-Server".
### User Documentation ## User Documentation
- how to use the software and APIs, FAQs, walkthroughs, ... - how to use the software and APIs, FAQs, walkthroughs, ...
...@@ -152,83 +157,88 @@ Call diagrams can be useful to follow code and service calls and should be exist ...@@ -152,83 +157,88 @@ Call diagrams can be useful to follow code and service calls and should be exist
- screencasts - screencasts
## Which version control do you use? You do use version control, do you? # Which version control do you use? You do use version control, do you?
We are using GIT in RDD! Nothing else! How it works, please see <https://git-scm.com/doc>. We exclusively use git in RDD. Please see <https://git-scm.com/doc> for information on how it works.
We recommend to use Gitflow Workflow: <https://www.atlassian.com/git/tutorials/comparing-workflows/gitflow-workflow>, Cheat Sheet: <https://danielkummer.github.io/git-flow-cheatsheet>), if possible on server side: use protection of the develop and master branches. All specific branches working on an issue descibed in a bug tracker may utilize the following naming scheme: `[track]/#[ISSUENUMBER]-[KEYWORD]`, e.g. `bugfix/#12-flux-capacitor`. We recommend to use the [Git flow Workflow](https://www.atlassian.com/git/tutorials/comparing-workflows/gitflow-workflow) (also consult the [Cheat Sheet](https://danielkummer.github.io/git-flow-cheatsheet)).
For git flow it is safest to protect your master and develop branch on server side to avoid accidental pushes into these branches. All specific branches working on an issue described in a bug tracker may utilize the following naming scheme: `[track]/#[ISSUENUMBER]-[KEYWORD]`, e.g. `bugfix/#12-flux-capacitor`.
A github workflow used in DARIAH-DE and related services is described in the DARIAH-DE Wiki: <https://wiki.de.dariah.eu/display/DARIAH3/DARIAH-DE+Release+Management#DARIAH-DEReleaseManagement-Beispielmitdevelop-undmaster-Branch(Gitflow)> A GitHub workflow used in DARIAH-DE and related services is described in the [DARIAH-DE Wiki](https://wiki.de.dariah.eu/display/DARIAH3/DARIAH-DE+Release+Management#DARIAH-DEReleaseManagement-Beispielmitdevelop-undmaster-Branch(Gitflow)).
Automatically closing issues via commit message depends on the Git repository server. Issues can also be referenced across repositories (cf. [link](https://help.github.com/articles/autolinked-references-and-urls/#commit-shas)). It is also recommended to automatically close issues via commit message; How this works exactly depends on the Git repository server. Issues can also be referenced across repositories (cf. [link](https://help.github.com/articles/autolinked-references-and-urls/#commit-shas)).
Which repo you are using depends on: We use the following Git servers at the moment in RDD:
- the project - Projects (GWDG) --> <https://projects.gwdg.de>
- GitLab (GWDG) --> <https://gitlab.gwdg.de>
- github.org --> <https://github.com/subugoe>
Which one is suitable for you depends on:
- the project you are working on
- existing code - existing code
- using Gitlab Runners - whether or not you want to use CI/CD or GitLab Runners
- ... - ...
We use the following at the moment in RDD:
- Projects (GWDG) --> <https://projects.gwdg.de>
- Gitlab (GWDG) --> <https://gitlab.gwdg.de>
- github.org --> <https://github.com/subugoe>
We have got an RDD team on Github: <https://github.com/orgs/subugoe/teams/fe> We have got an [RDD team on GitHub](https://github.com/orgs/subugoe/teams/fe). Feel free to join us!
Consider mirroring of repos for project visibility (e.g. mirror Gitlab/Projects code to Github?) Consider mirroring of repos for project visibility (e.g. mirror GitLab/Projects code to Github?)
## Are you tracking your bugs properly? # Do you track your bugs properly?
A bug tracking system is obligatory! Please use the respective bug tracking system of your repo and/or project management solution (please see chapter version control)! A bug tracking system is obligatory! Please use the respective bug tracking system of your repo and/or project management solution (please see chapter version control)!
## What is your test coverage? # Do you test your software?
We aim to have a test coverage of **100%** (except for getter and setter methods). This is understood on a component level, which means that every method should have at least one test. Whether you achive this by Test Driven Development (TDD) or not is specific to your preferred way to work. We aim to have a test coverage of **100%** (except for getter and setter methods). This is understood on a component level, which means that every method should have at least one test. Whether you achieve this by Test Driven Development (TDD) or not is specific to your preferred way to work.
Please keep in mind not only to write a test for each of your functions but also Please keep in mind not only to write a test for each of your functions but also
to consider all possible outcomes. It is e.g. not sufficient to test if a function to consider all possible outcomes. It is e.g. not sufficient to test if a function
creates a file if the written content depends on variables etc. creates a file if the written content depends on variables etc.
Examples for different programming languages are: Examples for writing tests in different programming languages are:
- **XQuery**: <https://gist.github.com/joewiz/fa32be80749a69fcb8da> - [**XQuery**](https://gist.github.com/joewiz/fa32be80749a69fcb8da)
## Building code and continuous integration # Building Code and Continuous Integration
### Building code ## Building Code
Ideally we use build tools to conveniently get a software running.
The reason for using a build tool is to be able to build and/or test a code project with one command (after checking out). Another reason is to include dependency management. The reason for using a build tool is to be able to build and/or test a code project with one command (after checking out). Another reason is to include dependency management.
##### Build tools we are using at the moment #### Build tools we are using at the moment
* **bash scripting**: (BdN Print, Fontane Print) * **bash scripting**: (BdNPrint, FontanePrint)
* **eXist**: Ant (SADE) * **eXist**: Ant (SADE)
* **Java**: Maven (TextGrid) * **Java**: Maven (TextGrid)
* **JavaScript**: * **JavaScript**:
* bower (DARIAH-DE GeoBrowser, tgForms) + bower (DARIAH-DE GeoBrowser, tgForms)
* cake (tgForms) + cake (tgForms)
* NPM (DARIAH-DE Publikator, tgForms) + NPM (DARIAH-DE Publikator, tgForms)
* rake (DARIAH-DE GeoBrowser) + rake (DARIAH-DE GeoBrowser)
* **Phython**: * **Phython**:
* make (Sphinx documentation) + make (Sphinx documentation)
* PIP (DiscussData) + PIP (DiscussData)
* **Ruby**: bundler (DARIAH status page) * **Ruby**: bundler (DARIAH status page)
##### Build tools we want to evaluate #### Build tools we want to evaluate
* gradle * gradle
### Continuous integration ## Continuous Integration
We want to use CI as soon as possible in new projects. We want to use CI as soon as possible in new projects.
The workflows we are using currently in Jenkins and Gitlab Runner are: The workflows we are using currently in Jenkins and GitLab Runner are:
* Code building * Code building
* Testing * Testing
...@@ -237,10 +247,10 @@ The workflows we are using currently in Jenkins and Gitlab Runner are: ...@@ -237,10 +247,10 @@ The workflows we are using currently in Jenkins and Gitlab Runner are:
* Distribution (Nexus, APTLY repo, eXist repo) * Distribution (Nexus, APTLY repo, eXist repo)
* Release Management (via GitLab Environments and gitflow) * Release Management (via GitLab Environments and gitflow)
#### Sample configuration of the GitLab Runner ### Sample configuration of the GitLab Runner
The following example illustrates how the GitLab Runner is used in SADE. The following example illustrates how the GitLab Runner is used in SADE.
The fully documented version of this file can be viewed [here](https://gitlab.gwdg.de/SADE/SADE/blob/develop/.gitlab-ci.yml). The full and documented version of this file can be viewed [here](https://gitlab.gwdg.de/SADE/SADE/blob/develop/.gitlab-ci.yml).
``` ```
...@@ -248,8 +258,6 @@ image: docker.gitlab.gwdg.de/fontane-notizbuecher/build:latest ...@@ -248,8 +258,6 @@ image: docker.gitlab.gwdg.de/fontane-notizbuecher/build:latest
stages: stages:
- build - build
- test
- deploy
build-develop: build-develop:
except: except:
...@@ -274,46 +282,14 @@ build-master: ...@@ -274,46 +282,14 @@ build-master:
paths: paths:
- build/*.xar - build/*.xar
- test/ - test/
installation:
except:
- tags
stage: test
script:
- bash test/eXist-db-*/bin/startup.sh | tee output.log &
# wait for eXist to have started
- while [ $(curl --head --silent http://localhost:8080 | grep -c "200 OK") == 0 ]; do sleep 2s; done
# shutdown eXist
- bash test/eXist-db-*/bin/shutdown.sh
- ls -al /tmp; mv /tmp/tests-* . || true
artifacts:
paths:
- output.log
- test/tests-*.xml
- test/eXist-db-*/webapp/WEB-INF/logs/expath-repo.log
# this enables us to get information like test coverage.
reports:
junit: test/tests-*.xml
upload:
only:
- master
- develop
except:
- tags
stage: deploy
script:
- FILENAME=$(ls build/*.xar)
- curl -u ci:${EXIST_UPLOAD_PW} -X POST -F file=@${FILENAME} https://ci.de.dariah.eu/exist-upload
``` ```
#### Sample configuration of the Jenkins CI (Multibranch Pipelines) ### Sample configuration of the Jenkins CI (Multibranch Pipelines)
- On commit and push to the <https://projects.gwdg.de> gitolite repo (such as <https://projects.gwdg.de/projects/tg-crud/repository>) Jenkins on ci.de.dariah.eu <https://ci.de.dariah.eu/jenkins> is notified (see projects' gitolite configuration) - On commit and push to the <https://projects.gwdg.de> gitolite repo (such as <https://projects.gwdg.de/projects/tg-crud/repository>) Jenkins on [ci.de.dariah.eu](https://ci.de.dariah.eu/jenkins) is notified (see projects' gitolite configuration)
- Jenkins then is doing a checkout and build of configured branches (see Jenkins' project's multibranch pipeline configuration such as <https://ci.de.dariah.eu/jenkins/job/DARIAH-DE-CRUD-Services>. Here comes THE CRUD Jenkinsfile, as it is used for other projects in a very similar way: - Jenkins then does a checkout and build of configured branches (see Jenkins' project's multibranch pipeline configuration such as <https://ci.de.dariah.eu/jenkins/job/DARIAH-DE-CRUD-Services>. Here comes THE CRUD Jenkinsfile, as it is used for other projects in a very similar way:
``` ```
...@@ -365,29 +341,27 @@ node { ...@@ -365,29 +341,27 @@ node {
- Stage *Preparation*: Prepare things - Stage *Preparation*: Prepare things
- Stage *Build*: Build, JAR, WAR, and DEB packages from source code, deploy JAR and WAR packages to the Nexus repo (at the moment <https://ci.de.dariah.eu/nexus> --> in the near future <https://nexus.gwdg.de>). Jenkins is configured to deploy JARs and WARs via Maven and a Nexus deploy-account. - Stage *Build*: Build, JAR, WAR, and DEB packages from source code, deploy JAR and WAR packages to the [Nexus repo](https://nexus.gwdg.de). For further information on Nexus cf. [ci's server documentation](https://wiki.de.dariah.eu/display/FEAD/ci.de.dariah.eu). Jenkins is configured to deploy JARs and WARs via Maven and a Nexus deploy-account.
- Stage *Publish*: Publish DEB packages to the DARIAH-DE Aptly Repo <https://ci.de.dariah.eu/aptly>. Jenkins is using a shared library of scripts and publishing is devided into four conditionals: TG version, DH version, SNAPSHOT version, or RELEASE version due to given version suffixes! - Stage *Publish*: Publish DEB packages to the [DARIAH-DE Aptly Repo](https://ci.de.dariah.eu/aptly). Jenkins is using a shared library of scripts and publishing is devided into four conditionals: TG version, DH version, SNAPSHOT version, or RELEASE version due to given version suffixes!
## Deployment and maintenance # Deployment and maintenance
### Puppet ## Puppet
### Monitoring ## Monitoring
### Release Management ## Release Management
# Code quality level for RDD # Code quality level for RDD
- Evaluate Software maturity levels from CESSDA: @TODO @mw
## Code review ## Code review
We want to ensure code review for all major commits, in gitflow for everything We want to ensure code review for all major commits, in gitflow for everything
that is subject to be merged into `develop`. that is subject to be merged into `develop`.
Projects with more than one developer in the team, it is preferred to have code For projects with more than one developer in the team it is preferred to have code
reviewer within the team, in other cases your friendly rdd developer team is reviews within the team, in other cases your friendly RDD developer team is
on your side. on your side.
### Proof of concept ### Proof of concept
...@@ -397,19 +371,21 @@ not necessary. ...@@ -397,19 +371,21 @@ not necessary.
# Licensing # Licensing
- clarify software licence before programming - clarify software license before programming
- add licence to code header - add license to code header
Best practice is to maintain a file listing all third-party packages that are Best practice is to maintain a file listing all third-party packages that are
part of the software. This list should hold the following metadata and SHOULD be part of the software. This list should hold the following metadata and SHOULD be
prepared like the table below, always in alphanumeric order. prepared like the table below, always in alphanumeric order.
```
| name | license | origin | | name | license | origin |
|---|---|---|---| |---|---|---|---|
| foo | barware | github.com/foo/bar | | foo | barware | github.com/foo/bar |
```
May be the `license-maven` plugin will help you. Maybe the `license-maven` plugin will help you.
# Retirement of software # Retirement of software
......
No preview for this file type
...@@ -9,7 +9,7 @@ ...@@ -9,7 +9,7 @@
\title{$title$} \title{$title$}
\date{\today \vspace{5cm} \small{\texttt{<https://github.com/subugoe/rdd-technical-reference>}}} \date{\today \\ \vspace{1cm} \small{\texttt{<https://github.com/subugoe/rdd-technical-reference>}}}
\begin{document} \begin{document}
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment