-
Clemens Wolff authored
Previously we were hosting the ok-server, ok-worker, autopy-manager and autopy-worker in AppService for containers. However, we were seeing some issues with this approach as workers were killed without death notifications and rq/Redis was getting confused. After this change, we now deploy the compute (workers and servers) to Azure Kubernetes service. In order to make it easier to orchestrate the setup of the various resources in Azure as well as the Kubernetes deployments, we're using bash scripts. First, we create a bunch of resources using `setup.sh` and then we configure them using `install.sh`. The setup scripts are wrapped in a Docker container so that the environment in which the scripts execute is well known, e.g. Azure CLI is installed, helm is available, kubectl is available, etc. Most of the setup scripts fall-back to ARM templates to create the resources in Azure. The ARM templates can be modified to configure parameters such as resource SKUs, number of VMs to spin up for the cluster, type of VMs to use in the cluster, etc. The install scripts perform operations such as creating the database schema, deploying the application to Kubernetes via Helm and setting up the DNS for the Kubernetes ingress via the Azure CLI so that kube-lego can configure a TLS certificate for the ingress via Let's Encrypt. Note that the ingress to the applications running in the Kubernetes cluster are configured as just one entrypoint so that a single TLS certificate and DNS entry can be used (Azure public IP resource currently only support a single DNS label per IP). To solve routing, we thus need to mount the two sub-applications (OKpy and AutoPY) at sub-urls of the ingress which requires some minor application changes which are submitted in a separate pull request: https://github.com/okpy/ok/pull/1300 The setup scripts can be run either via the "Deploy to Azure" button (in which case the setup Docker container is run in an Azure Container Instance) or via a local `docker run` command. In the latter case, it's advised to also include a volume mount `docker run -v $PWD:/app/secrets` so that the access keys for the resources created during the deployment will be made available to the deployer. Note that in order to skip deploying specific resources (e.g. to re-use existing resources), you can place a file into the secrets folder that contains the access keys for your existing resources in which case the setup scripts will skip deploying that step.
e630eb7e