Dear Gitlab users, due to maintenance reasons, Gitlab will not be available on Thursday 30.09.2021 from 5:00 pm to approximately 5:30 pm.

Commit a7985e5a authored by hynek's avatar hynek 🤤
Browse files

Merge branch '127-switch-build-release-to-single-branch-main' into 'main'

Resolve "switch build/release to single branch "main""

Closes #127

See merge request !94
parents d3aaadc0 a9752c1c
......@@ -2,3 +2,4 @@ build/
test/
local.build.properties
expath-pkg.xml
node_modules
# This GitLab runner configuration ensures every version of the SADE app is
# working properly before making it public.
#
# Final versions of SADE (i.e. develop or master branches) that have been built
# Release versions of SADE that have been built
# successfully are uploaded to DARIAH-DE's public eXist repository at
# https://ci.de.dariah.eu/exist-repo/index.html where other projects can reuse it.
#
# Please make sure you have all necessary passwords set as secret variables.
#
# In case you're using SADE for an non-DARIAH-DE project, 'upload' should be
# adapted or completely omitted.
#
# Environment variables used:
# * ${EXIST_UPLOAD_PW} - the password needed to push to DARIAH-DE's eXist repo
# * ${FILENAME} - the name if the XAR package created, e.g. SADE-develop-1-1-1.xar
# a general image that contains everything eXist-db needs to run properly
image: docker.gitlab.gwdg.de/fontane-notizbuecher/build:latest
---
# pipelines are only created on commits to open merge request branches and
# merges to the default branch
workflow:
rules:
- if: $CI_MERGE_REQUEST_IID
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
.npm-setup: &npm-setup
- curl -sL https://deb.nodesource.com/setup_14.x | bash -
- apt-get install -y nodejs
- npm ci --unsafe-perm --cache .npm --prefer-offline
stages:
- build
- test
- deploy
- release
# in the build stage eXist-db is (as the name implies) built considering all
# dependencies that are stated in build.properties and/or generic.xml.
# build-develop is triggered after pushing/merging into any branch except master
build-develop:
except:
- master
- tags
stage: build
script:
- ant test
artifacts:
paths:
- build/*.xar
- test/
default:
# a general image that contains everything eXist-db needs to run properly
image: docker.gitlab.gwdg.de/fontane-notizbuecher/build:latest
# build-master is only triggered after pushing/merging into master
build-master:
only:
- master
- tags
stage: build
script:
- cp master.build.properties local.build.properties
- ant test
artifacts:
paths:
- build/*.xar
- test/
# eXist-db is installed and all tests stated in post-install.xq are executed.
# ATTENTION: tests failing in post-install.xq will NOT cause the stage to fail!
installation:
except:
- tags
stage: test
script:
- bash test/eXist-db-*/bin/startup.sh | tee output.log &
# build the xar-file and prepare a test environment
- ./build_test
- bash test/exist-distribution-*/bin/startup.sh | tee output.log &
# wait for eXist to have started
- while [ $(curl --head --silent http://localhost:8080 | grep -c "200 OK") == 0 ]; do sleep 1s; done
# shutdown eXist
- bash test/eXist-db-*/bin/shutdown.sh
- bash test/exist-distribution-*/bin/shutdown.sh
- ls -al /tmp; mv /tmp/tests-* . || true
artifacts:
paths:
- output.log
- test/tests-*.xml
- test/eXist-db-*/logs/expath-repo.log
- test/exist-distribution-*/logs/expath-repo.log
# this enables us to get information like test coverage.
reports:
junit: test/tests-*.xml
# upload the EXPath package to the repo
upload:
only:
- master
- develop
except:
- tags
stage: deploy
semantic-release:
stage: release
before_script:
- *npm-setup
script:
- FILENAME=$(ls build/*.xar)
- curl -u ci:${EXIST_UPLOAD_PW} -X POST -F file=@${FILENAME} https://ci.de.dariah.eu/exist-upload
- npx semantic-release
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
# when a git flow release is made, a tag will be pushed starting this job. it
# will keep the resulting artifact from job number one and it will set up a
# Gitlab release at the repo. therefore the merge message starting at the release
# branch should be written in markdown.
release:
stage: deploy
only:
- tags
script:
# get number of first job in pipeline (assuming that this will create the needed artifact)
# it is also possible to query for a job name by altering the jq filter
- 'curl --output jobs.json --header "PRIVATE-TOKEN: $GITLAB_TOKEN" "$CI_API_V4_URL/projects/$CI_PROJECT_ID/pipelines/$CI_PIPELINE_ID/jobs"'
- CI_JOB_TARGET=$(jq ".[0].id" < jobs.json)
- echo $CI_JOB_TARGET
# keep artifact (release will link there)
- 'curl --request POST --header "PRIVATE-TOKEN: $GITLAB_TOKEN" "$CI_API_V4_URL/projects/$CI_PROJECT_ID/jobs/$CI_JOB_TARGET/artifacts/keep"'
# create release data
# parse commit message (markdown, lines starting with “\”)
- MARKDOWN=$(echo "$CI_COMMIT_MESSAGE" | sed 's=^\\==g')
# prepare the json file
- 'jq
".name = \"$CI_PROJECT_PATH $CI_COMMIT_TAG\" |
.tag_name = \"$CI_COMMIT_TAG\" |
.description = \"$MARKDOWN\" |
.assets.links[0].name = \"package\" |
.assets.links[0].url = \"https://gitlab.gwdg.de/$CI_PROJECT_PATH/-/jobs/$CI_JOB_TARGET/artifacts/download\"
" < gitlab-release.json.tmpl > gitlab-release.json'
- 'curl --header "Content-Type: application/json" --header "PRIVATE-TOKEN: $GITLAB_TOKEN" --data @gitlab-release.json --request POST $CI_API_V4_URL/projects/$CI_PROJECT_ID/releases'
artifacts:
paths:
- gitlab-release.json
......@@ -7,43 +7,41 @@ This is the main application for SADE. It serves as XAR package according to the
[EXPath Packaging System](http://expath.org/spec/pkg) and is created as an application for
[eXist-db](https://exist-db.org).
## Getting Started
These instructions will get you a copy of the project up and running on your local machine for development and testing purposes. See deployment for notes on how to deploy the project on a live system.
### Prerequisites
* ant >= 1.10.0
* ant >= 1.10.0
### Local installation
To get a version of SADE running locally on your computer, clone this repository in your preferred git directory:
```
```sh
mkdir SADE
cd SADE
git clone git@gitlab.gwdg.de:SADE/SADE.git
```
Once the repository is cloned, run
```
ant test
```sh
./build_test
```
which installs all necessary dependencies and creates a local version of SADE in `/test`.
To start the database, run
```
bash test/eXist-db-*/bin/startup.sh
```sh
bash test/exist-distribution-*/bin/startup.sh
```
SADE's backend is now available at `localhost:8080`. To have a look at the website, view `http://localhost:8080/exist/apps/sade/index.html` in your preferred browser.
## The Git repo
This repo contains the sources of the application which contain a brief
documentation as long as you are not dealing with a fork. Please find the
documentation in the folder `docs` where they are stored in Markdown. Any
......@@ -51,50 +49,57 @@ default installation of this package serves the documentation as website
integrated in your template and in your instance.
### Git hooks
The folder `.hooks` contains git hook scripts to be executed on certain git
events. The usage of these scripts is recommended, as they ensure a smooth
workflow.
#### pre-commit
We test for `DONOTCOMMIT` comments in the code. As long as a comment like this
is in files, no commit is possible.
#### post-commit
On any commit a new artifact is being created, so you can proceed and test it
immediately.
## Build
`ant`
```sh
./build_xar
```
Artifacts will be stored in `build/`.
The default artifact is the xar package.
## Test
`ant test` will download the specified version of eXist and all dependencies to
`./build_test` will download the specified version of eXist and all dependencies to
a ready-to-go instance in the `test/` directory. at the first start all tests
will be executed by the `post-install.xq` script. To start the database use
```bash
test/eXist-db-*/bin/startup.sh
```sh
test/exist-distribution-*/bin/startup.sh
```
Guide your favorite browser to the [dashboard](http://localhost:8080/exist/).
### Port settings
When the port 8080 is blocked by a different service, simply set different ones
with the following command and restart the database.
```bash
```sh
sed -i 's/"8080"/"8090"/g; s/"8443"/"8444"/g' \
test/eXist-db-4.5.0/tools/jetty/etc/jetty-http.xml \
test/eXist-db-4.5.0/tools/jetty/etc/jetty.xml \
test/eXist-db-4.5.0/tools/jetty/etc/jetty-ssl.xml
```
## Deploy
We recommend using Debian packages for deployment. There built-in routines
in the ant script. The [CI configuration](.gitlab-ci.yml) has a part to
publish the package at the [DARIAH-DE Aptly Repository](https://ci.de.dariah.eu/packages/pool/snapshots/s/).
## Installation
## Known Issues
In the unlikely event of cycling to the next digit in TextGrid URIs you may
encounter issues when you going to use an older URI together with a newer that
starts with the same characters. Example: One will publish the text
......
project.name=http://sade.textgrid.de/ns/SADE-develop
project.version=4.2.1
project.title=Scalable Architecture for Digital Editions powered by TextGrid
project.abbrev=SADE-develop
project.processorversion=5.2.0
destfile=${build.dir}/${project.abbrev}-${project.version}.xar
test.dir=test
dependency.track=-develop
assets.abbrev=sade_assets${dependency.track}
code-viewer.abbrev=cv
<?xml version="1.0" encoding="UTF-8"?>
<project default="xar" name="SADE" xmlns:xdb="http://exist-db.org/ant">
<property environment="env"/>
<!-- bind environment variables to 'env'//-->
<property environment="env" />
<!-- default values for building outside CI //-->
<property name="project.abbrev" value="sade" />
<property name="project.name" value="http://sade.textgrid.de/ns/${project.abbrev}" />
<condition property="project.version" value="${env.RELEASE_VERSION}" else="${env.PREVIOUS_VERSION}+${env.LAST_COMMIT_SHORT_SHA}">
<isset property="env.RELEASE_VERSION" />
</condition>
<property name="project.title" value="Scalable Architecture for Digital Editions powered by TextGrid" />
<property name="project.processorversion" value="5.2.0" />
<property name="build.dir" value="build"/>
<property file="local.build.properties"/>
<property file="build.properties"/>
<property name="test.dir" value="test"/>
<property name="exist.root" value="${test.dir}/exist-distribution-${project.processorversion}"/>
<property name="destfile" value="${build.dir}/${project.abbrev}-${project.version}.xar" />
<xmlproperty file="expath-pkg.xml.tmpl"/>
<antversion property="antversion"/>
......@@ -23,12 +36,11 @@
<filter token="project.abbrev" value="${project.abbrev}"/>
<filter token="project.name" value="${project.name}"/>
<filter token="project.processorversion" value="${project.processorversion}"/>
<filter token="dependency.track" value="${dependency.track}"/>
</filterset>
</copy>
<mkdir dir="${build.dir}"/>
<zip basedir="." destfile="${destfile}" defaultexcludes="no"
excludes=".git/,${build.dir}/,${test.dir}/"/>
includes="docs/**, modules/**, templates/**, textgrid/**, CHANGELOG.md, collection.xconf, config.xml, controller.xql, expath-pkg.xml, icon.png, lang.xml, LICENSE, navigation.xml, post-install.xq, pre-install.xq, README.md, repo.xml, search-hits.xslt, test.xq, test.xqm" />
</target>
<target name="cleanup">
......@@ -37,26 +49,23 @@
<target name="test" depends="antversion-test, cleanup, xar">
<!-- this path may be and is subject to change! -->
<get src="https://bintray.com/existdb/releases/download_file?file_path=exist-distribution-${project.processorversion}-unix.tar.bz2" dest="${build.dir}/eXist-db-${project.processorversion}.tar.bz2" skipexisting="true" />
<get src="https://github.com/eXist-db/exist/releases/download/eXist-${project.processorversion}/exist-distribution-${project.processorversion}-unix.tar.bz2"
dest="${build.dir}/eXist-db-${project.processorversion}.tar.bz2"
skipexisting="true" />
<untar src="${build.dir}/eXist-db-${project.processorversion}.tar.bz2" dest="${test.dir}" compression="bzip2" />
<!-- directory name changed: exist-distribution-5.0.0-->
<move todir="${test.dir}/eXist-db-${project.processorversion}">
<fileset dir="${test.dir}/exist-distribution-${project.processorversion}" />
</move>
<!-- task setpermissions requries at least ant 1.10.0 -->
<setpermissions permissions="OWNER_READ,OWNER_WRITE,OWNER_EXECUTE,OTHERS_READ,OTHERS_EXECUTE,GROUP_READ,GROUP_EXECUTE">
<file file="${test.dir}/eXist-db-${project.processorversion}/bin/startup.sh"/>
<file file="${exist.root}/bin/startup.sh"/>
</setpermissions>
<get src="https://ci.de.dariah.eu/exist-repo/find.zip?abbrev=${assets.abbrev}&amp;processor=${project.processorversion}" dest="${test.dir}/eXist-db-${project.processorversion}/autodeploy/sade_assets-latest.xar" ignoreerrors="true"/>
<get src="https://ci.de.dariah.eu/exist-repo/find.zip?abbrev=${code-viewer.abbrev}&amp;processor=${project.processorversion}" dest="${test.dir}/eXist-db-${project.processorversion}/autodeploy/codeview-latest.xar" ignoreerrors="true"/>
<get src="https://ci.de.dariah.eu/exist-repo/find.zip?abbrev=sade_assets&amp;processor=${project.processorversion}" dest="${exist.root}/autodeploy/sade_assets-latest.xar" ignoreerrors="true"/>
<get src="https://ci.de.dariah.eu/exist-repo/find.zip?abbrev=cv&amp;processor=${project.processorversion}" dest="${exist.root}/autodeploy/codeview-latest.xar" ignoreerrors="true"/>
<get src="http://exist-db.org/exist/apps/public-repo/find.zip?abbrev=markdown&amp;processor=${project.processorversion}" dest="${test.dir}/eXist-db-${project.processorversion}/autodeploy/markdown-latest.xar" ignoreerrors="true"/>
<get src="http://exist-db.org/exist/apps/public-repo/find.zip?abbrev=functx&amp;processor=${project.processorversion}" dest="${test.dir}/eXist-db-${project.processorversion}/autodeploy/functx-latest.xar" ignoreerrors="true"/>
<get src="http://exist-db.org/exist/apps/public-repo/public/markdown-0.6.xar" dest="${exist.root}/autodeploy/markdown-latest.xar" ignoreerrors="true"/>
<get src="http://exist-db.org/exist/apps/public-repo/public/functx-1.0.1.xar" dest="${exist.root}/autodeploy/functx-latest.xar" ignoreerrors="true"/>
<copy file="${destfile}" todir="${test.dir}/eXist-db-${project.processorversion}/autodeploy" />
<copy file="${destfile}" todir="${exist.root}/autodeploy" />
</target>
</project>
export PREVIOUS_VERSION=$(git describe --tags $(git rev-list --tags --max-count=1) | sed 's/^v//')
export LAST_COMMIT_SHORT_SHA=$(git rev-parse --short HEAD)
ant test
\ No newline at end of file
export PREVIOUS_VERSION=$(git describe --tags $(git rev-list --tags --max-count=1) | sed 's/^v//')
export LAST_COMMIT_SHORT_SHA=$(git rev-parse --short HEAD)
ant
\ No newline at end of file
......@@ -7,7 +7,7 @@
<title>@project.title@</title>
<dependency processor="http://exist-db.org" semver-min="@project.processorversion@"/>
<dependency package="http://exist-db.org/apps/shared"/>
<dependency package="https://sade.textgrid.de/ns/assets@dependency.track@" semver-min="2.0.0"/>
<dependency package="https://sade.textgrid.de/ns/assets" semver-min="2.0.0"/>
<dependency package="http://www.functx.com"/>
<dependency package="http://exist-db.org/apps/markdown"/>
</package>
{
"name": "",
"tag_name": "",
"description": "",
"assets": {
"links": [{
"name": "",
"url": ""
}]
}
}
project.abbrev=SADE
project.name=http://sade.textgrid.de/ns/SADE
dependency.track=
This diff is collapsed.
{
"name": "sade",
"description": "SADE ( Scalable Architecture for Digital Editions) it an application for [eXist-db](https://exist-db.org) that offers a set of useful features to quickly build a website for digital editions basing on [TEI](http://www.tei-c.org)-encoded texts.",
"repository": {
"type": "git",
"url": "git@gitlab.gwdg.de:SADE/SADE.git"
},
"private": true,
"devDependencies": {
"@semantic-release/changelog": "^5.0.1",
"@semantic-release/exec": "^5.0.0",
"@semantic-release/git": "^9.0.0",
"@semantic-release/gitlab-config": "^8.0.0",
"commitizen": "^4.2.2",
"cz-conventional-changelog": "^3.3.0",
"semantic-release": "^17.2.2"
},
"release": {
"branches": [
"main"
],
"plugins": [
"@semantic-release/commit-analyzer",
"@semantic-release/release-notes-generator",
[
"@semantic-release/changelog",
{
"changelogFile": "CHANGELOG.md"
}
],
[
"@semantic-release/gitlab",
{
"gitlabUrl": "https://gitlab.gwdg.de",
"assets": [
{
"path": ".build/sade-*.xar",
"label": "XML Archive File"
},
{
"path": "CHANGELOG.md",
"label": "Changelog"
}
]
}
],
[
"@semantic-release/git",
{
"assets": [
"CHANGELOG.md"
]
}
],
[
"@semantic-release/exec",
{
"prepareCmd": "./prepare ${nextRelease.version}",
"publishCmd": "./publish .build/sade-${nextRelease.version}.xar"
}
]
]
}
}
#!/bin/bash
export RELEASE_VERSION=$1
echo "Building $RELEASE_VERSION ..."
ant
exit $?
#!/bin/bash
export FILE=$1
echo "Uploading $FILE ..."
curl -u ci:${EXIST_UPLOAD_PW} -X POST -F file=@${FILE} https://ci.de.dariah.eu/exist-upload
exit $?
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment