GitLab

This script can set up a bare Debian installation (install OpenSSH-Server and
git first!) to be used with this project.

The proper way to get user name and password is to fetch them from the .env file
and not from the .env.d/ directory because technically, Docker / docker-compose
will look up all variables from the .env file and it could possibly be out of
sync with the directory.

also reorder chmod, cant mod a file before it is created

* seperate the various bootstrap stages into their own files in `bin/`
* re-write `configure` to run each bootstrap stage inside a container
* make host platform dependencies (docker, python, etc) installation optional

sse-server does not support using a different base uri from frontend
presigned urls, and backend comminication channels.

Therefore for local development, we must expose minio on the localhost,
and use /etc/hosts to resolve lookups for the hostname `minio`

this way sse-server can use the hostname `minio` for both frontend
presigned urls and backend communication.

configuration will have to differ for cloud deployments where traffic
is only allowed port on 80/443.

* expose public services through nginx-proxy container
* reconfigure services to commincate with each other over the internal
  docker network when possible
* remove previous nginx installation from ci

- Split up docker-compose.yml -> docker-compose/*.yml
- Turn make sure all container params are stored in the .env file (no
  hardcoded values in docker-compose environment config)
- Rename many environment variables for better namespacing
- remove docker-compose.bootstrap.yml entirely and move the
  functionality to the configure script

* init_realm_admin.py: use registration authority to generate a cpabe
  key for the realm admin user
* create_subject.py: provision a set of subject keys and create a
  corresponding XNAT subject
* create_user.py: use registration authority to add a new user with
  cpabe key in keycloak

* Remove all `DEMO_*` fixtures from .env.tpl
* Add TEST_PROJECT and TEST_KEYCLOAK_USER/PASSWORD variables
* Replace DEMO users with TEST user in realm-export.json.tpl
* Regenerate xnat.sql.tpl without any subject data
* Remove sse, ta and keytray .sql.tpl files, we can test without these
  fixtures
* Modify docker-compose files to reflect these changes
* Tweak tests to work with new fixture environment

* Start adding utility libraries for interacting with the various
  componenents in the stack. So far very incomplete. These will be
  expanded over time and may become the basis of a library for
  provisioning the system

* Start adding tests, using `unittest` from the python standard library.
  This framework was chosen because it has no external dependencies and
  is available out-the-box on any Python installation. We can move to
  something more feature-complete in the future if needed.

* This commit contains mostly basic health-checks
  and a few create/read data tests. These will be expanded over time

* Port the dotenv file to a real shell script. Shell scripts are more
  portable as they don't require the installation extra dependencies.
  They also have proper support for double and single quoting and
  better support for variable expansion, which is not the case for many
  dotenv implementations (I'm looking at you docker-compose)

* Rename many variables, adding the `DEMO_` prefix. This is to make it
  more clear which variables are fixtures for demonstration purposes.

* Add a number of new new variables and corresponding templates. Mostly
  to allow us more customization of the demonstration fixtures when
  bootstrapping

Added `configure` file which sets up the stack on the machine and updated
README how to install stack. Some special secrets (e.g. ABE keypairs) are not
created in this script and need to created manually post configuration.

commit d36a1a81 inadvertantly added empty-lines to the end of everyfile,
rather than just a new line character.

fix this with the following one-liner script:

```
for f in $(git whatchanged --no-commit-id --name-only 29216f52..d36a1a81);
    do sed -i '$d' $f;
done;
```

Git (and probably some other software) does not work well with text files which
do not have an empty line at the end.

* Remove keycloak.sql dump/template
* Update realm-export.json.tpl and re-enable templating
* Add instructions to README for using realm-export.json

The updated realm-export.json contains a number of new users

* member
* owner
* collaborator
* subject
* subjec2

Apparently, it is impossible to tell docker-compose to delete the containers
after their processes exited. Added an instruction to the README.md how to
half-manually delete the dangling containers.

References:
https://serverfault.com/questions/750175/docker-compose-option-to-automaticaly-remove-container-after-run-in-docker-comp
https://stackoverflow.com/questions/47207616/auto-remove-container-with-docker-compose-yml

some files (specifically conf-tlp/xnat.sql.tpl and docker-compose.bootstrap.yml
) included hard-coded magic strings of the local machine's IPv4 address. These
occurences are now replaced with an environment variable defined in
.env.example)