The Quickest Way to Add New Apps to ShinyProxy

Once your ShinyProxy server is up and running, you need to maintain and update it. Last time we looked at how to update existing apps. This section explains how to add new ones hassle-free.

ShinyProxy serves containerized web applications, like Shiny apps, without limiting the number of concurrent users or application hours. It is free, open-source, and comes with free enterprise features, such as authentication and app-level authorization.

In the previous posts, I introduced ShinyProxy and its configuration, then reviewed how to update existing apps without disrupting the users. But once in a while, you want to add new apps to ShinyProxy. This is when you have to update the configuration and do a few more steps. Let's see what are these steps and how to make the process more efficient.

Prerequisites

You can find the configuration file application.yml for this tutorial in the analythium/shiny-correlation GitHub repository:

To follow along, set up a virtual machine running Docker and ShinyProxy following this post on any cloud provider using Ubuntu 20.04. Or you can use the ShinyProxy 1-click app from the Digitalocean Marketplace:

Create a droplet (virtual machine) using the ShinyProxy marketplace image

Once you know the host URL or IPv4 address, set the $HOST environment variable and log in as root user using ssh:

export HOST="178.128.235.125"

ssh root@$HOST

Steps to add new apps

The ShinyProxy configuration is defined by the application.yml file in the /etc/shinyproxy folder. This file lists the apps, therefore adding new apps involve the following steps:

update the application.yml to include the new apps
pull the new Docker images to the host with docker pull
restart the ShinyProxy service with service shinyproxy restart

Steps 2 and 3 happen on the server. But step 1 can be achieved in a few different ways. Let's see what those are.

Edit configuration on the server

The simplest approach is to edit the YAML file directly on the server. Open the file with nano or vim, remove unwanted applications (I removed both demo apps from the /etc/shinyproxy/application.yml), and add the new apps. I show the final application.yml file here for reference:

proxy:
  title: Shiny Proxy
  logo-url: https://hub.analythium.io/assets/logo/logo.png
  landing-page: /
  favicon-path: favicon.ico
  heartbeat-rate: 10000
  heartbeat-timeout: 60000
  port: 8080
  authentication: simple
  admin-groups: admins
  users:
  - name: admin
    password: password
    groups: admins
  - name: user
    password: password
    groups: users
  docker:
    url: http://localhost:2375
    port-range-start: 20000
  specs:
  - id: cor2d
    display-name: Correlation in 2D
    description: App with 2D kernel density estimate
    container-cmd: ["R", "-e", "shiny::runApp('/home/app')"]
    container-image: analythium/correlation:v1
    logo-url: https://hub.analythium.io/assets/apps/cor2d.png
    access-groups: []
  - id: cor3d
    display-name: Correlation in 3D
    description: App with kernel density estimate with 3D RGL plot
    container-cmd: ["R", "-e", "shiny::runApp('/home/app')"]
    container-image: analythium/correlation:v2
    logo-url: https://hub.analythium.io/assets/apps/cor3d.png
    access-groups: [admins, users]

logging:
  file:
    name: shinyproxy.log

The two new apps are the same as what you saw when discussing how to update existing apps. The access-groups: [] value means that all authenticated users can access the 2D version of the apps.

Pull the new Docker images:

docker pull analythium/correlation:v1
docker pull analythium/correlation:v2

If you are using a private Docker registry, you have to log in. It is best to generate a token with restricted scopes (i.e. pull-only privileges). You can use this token instead of your password, and it is easy to revoke the token any time without having to change your password at all the places you have used it. This is a one-time thing, the username and token will be saved and used later when login is required. Save the token into a file (the double space at the beginning of the line prevents the token to end up in bash history):

  echo your_token > ./token.txt

Now you can pass the token via standard input, use your username and registryname:

cat ./token.txt | docker login --username username --password-stdin registryname

Finally, restart ShinyProxy:

service shinyproxy restart

If you were logged into the ShinyProxy server, now you should see 502 Bad Gateway until ShinyProxy restarts, then you'll be asked to log in again.

This is what will happen with any of the logged-in users on the server. To avoid such disruptions, you can schedule the updates to times when there are no users logged in, or when the disruption is minimal and advertised to users in advance. You can also use Redis in-memory database for session persistence.

The approach described here is a perfectly functional way of updating ShinyProxy configuration. But once the file is changed, it is harder going back unless you saved a backup copy somewhere. This is why using version control could be helpful.

Use git to edit and update the configuration

Edit the application.yml on your local machine and git push the changes to a remote repository. When you first update the configuration on the server, git clone the project you have under version control:

git clone https://github.com/analythium/shiny-correlation.git

Next time, you can just cd into the directory and git pull the changes:

cd shiny-correlation
git pull

Copy the YAML configuration file into the /etc/shinyproxy folder:

cp application.yml /etc/shinyproxy

Pull the new images and restart ShinyProxy as in the previous section:

docker pull analythium/correlation:v1
docker pull analythium/correlation:v2

service shinyproxy restart

This approach gets around the versioning issue we had before. But typing the new image names one by one seems tedious. Let's see if we can find some efficiencies.

Push configuration changes over SSH

I am going to explain a workflow similar to the previous one but instead of pulling changes, I am going to use ssh and scp to transfer the file to the server and run the docker pull and ShinyProxy restart commands.

Edit the application.yml and commit the changes so you have the full history. Once your application.yml is ready, cd into the local directory where the configuration file is located and download the following script file:

curl -s https://raw.githubusercontent.com/analythium/shinyproxy-1-click/master/digitalocean/setup.sh -o setup.sh

The general usage of the freshly downloaded setup.sh is:

bash setup.sh -i ~/.ssh/id_rsa -s root@ip_address -f application.yml

The following command-line arguments need to be passed to the setup.sh script:

-i: path to your ssh key
-s: user name (`root`) and the IP address, e.g. user@178.128.235.125
-f: path and file name of the ShinyProxy config, e.g. /path/to/application-new.yml.

The script then takes care of the steps in the following order:

Copies the updated YAML to the server into the /etc/shinyproxy folder
pulls the Docker images listed in the YAML file: updates the ones already pulled before, and the ones newly added too
restarts the ShinyProxy service

This is the command for our actual YAML file inside the git repository:

bash setup.sh -i ~/.ssh/id_rsa -s root@${HOST} -f application.yml

You can see an output similar to this one:

[INFO] Copying application.yml to host
[INFO] Updating docker images according to application.yaml
v1: Pulling from analythium/correlation
420047682034: Pulling fs layer

...

7bdb658c82cf: Pull complete
Digest: sha256:175663b0be8c97743f7a5bd1960c59d3f09536cd762a0f6f0a5744ee55ddb7ce
Status: Downloaded newer image for analythium/correlation:v1
docker.io/analythium/correlation:v1
v2: Pulling from analythium/correlation
420047682034: Already exists

...

b65c57a60c68: Pull complete
Digest: sha256:4eaa98afe32b88d4cce7ce5436956ecb38e94c776bee97244a8fc28cde8e415f
Status: Downloaded newer image for analythium/correlation:v2
docker.io/analythium/correlation:v2
[INFO] Restarting ShinyProxy
[INFO] Done

The script parses the YAML configuration and finds the container-image entries. This is done by the following script in the background:

#!/bin/bash
db=$(grep 'container-image:' $1 | sed 's/[^:]*://' | sed 's/^[[:space:]]*//g')
while IFS= read -r line; do docker pull $line; done <<< "$db"

Check your ShinyProxy server, you should see the two new apps added:

ShinyProxy listing the 2 new images after the updates

Don't forget to shut down the server if you don't need it anymore.

Conclusions

We reviewed 3 ways of updating ShinyProxy. The 3rd script-based update of the application.yml file is quite efficient if you have all the command line tools handy. Changes can be version-controlled, and you are one line away from an updated ShinyProxy server without navigating files and directories on the host machine. The script even scans the YAML file for all the Docker images that need to be pulled. The only thing to remember is to perform these updates at a time when it causes minimal disruption to users.