Sasmodel Marketplace Deployment

!! Model file Mimetypes !!

At Code Camp 8 (September 2018, Lund) it was deduced that it is not the Apache web server but rather some combination of methods called on the various dependencies Python libraries that power the marketplace that are causing this issue. Therefore, as a workaround, we are now accepting files with a mime type of 'text/plain' onto the site.

However, this is only a stop-gap measure. It was decided that due to the age and increasingly precarious nature of the disks in the danse servers that it would make sense to renew the disks in the danse2 server and slowly migrate active web services from danse to danse2. At which point this issue should be probed further as part of the transition.

Website Deployment

Deployment for the Sasmodel Marketplace is mostly automatic:

When a new commit is pushed to the ​SasView/Sasmodel-Marketplace repo on GitHub, a Travis CI build is started. If the Travis build succeeds, it will push the changes to a bare git repo at danse.chem.utk.edu:/home/git/marketplace.sasview.org. This repo has a post-receive hook that will then checkout the code to /var/www/marketplace.sasview.org.

If you add any new static files to the site (e.g. images, css files, etc), then you will need to follow the instructions in ​README.md to ensure these are served correctly.

The updated website will not be served until sudo service apache2 reload is ran. The server does actually restart every day, so if the changes don't need to be deployed immediately, you don't need to manually reload the server; the changes will take effect after the daily restart.

Automatic Model Updating

The entries on the marketplace for the models included with sasmodels will update automatically.

When a new commit is pushed to ​SasView/sasmodels a Travis CI build will run. If it succeeds, the latest code will be pushed to a bare git repo at danse.chem.utk.edu:/home/git/sasmodels. This repo has a post-receive hook that will checkout a copy of the code to /var/www/marketplace.sasview.org/sasmodels. It will then run the upload_sasmodels.py script in that directory.

The upload_sasmodels.py script will check to see if any of the model files have changed, and if so, update their entries on the marketplace. If any new models have been added, they will be uploaded to the marketplace.
This script does come with a caveat: if any model files are renamed, a new entry will be created for them on the marketplace, instead of updating the existing one.

User Permissions on the Server

Ideally, all files in the /var/www/marketplace.sasview.org directory should belong to the user/group 'www-data'. However, the post-receive hooks in the git repos that write files to this directory are ran by the user 'git', so the owner of the files can get changed.

To fix this issue, the 'git' user has been added to the group 'www-data', and the 'www-data' user has been added to the group 'git'. The owning group has been given read & write access to all files in the directory, so both users should be able to read/write the files freely.

If you need to make any manual changes in the /var/www/marketplace.sasview.org directory, make sure your user is a member of both the 'www-data' and 'git' groups. You can add a user to a group by running sudo adduser [username] [group].
Avoid using sudo when running commands that write to files (eg git pull), as this will set the owner of the files to 'root'.