Redmine with Passenger and Nginx on Ubuntu 16.04
Just in time for Ubuntu 18, here are Ubuntu 16 instructions for getting Redmine up and running. Though better than on Ubuntu 14, this is still a maze of twisty little passages, all alike.
Here are flexible instructions to get an up to date, robust, secure installation going.
- Ubuntu 16.04 LTS - a mature Ubuntu version with long term support
- MySQL - the database
- Ruby - the technology on which Redmine runs, installed using RVM to manage the ruby version and have access to up to date components
- Phusion Passenger- the application server in which to run Redmine
- Nginx - the web server within which Redmine runs
MySQL Configuration
Install MySQL. For this step, relying on the Ubuntu packages is fine.
sudo apt-get install -y mysql-server libmysqlclient-dev
Connect to the mysql service (mysql -p) and create the database and provide access to the redmine user:
mysql -p -u root
CREATE DATABASE redmine CHARACTER SET utf8;
CREATE USER 'redmine'@'localhost' IDENTIFIED BY 'yourpassword';
GRANT ALL PRIVILEGES ON redmine.* TO 'redmine'@'localhost';
If this is just a test system, move onto the Ruby Installation section.
For production systems, a separate data disk should be used rather that storing data on the same partition as the root system. Stop the service and then migrate the data directory.
service mysql stop
Modify the data directory configuration in MySQL:
sudo vim /etc/mysql/mysql.conf.d/mysqld.cnf
#datadir = /var/lib/mysql #old location
datadir = /data/mysql # new location
On install, MySQL 5.7 automatically initiates the data directory. So this must be moved to the new location.
sudo mkdir /data
sudo mv /var/lib/mysql/ /data/mysql/
Update AppArmor, otherwise the mysql process won't start. The MySQL 5.7 and Ubuntu combination have an AppArmor bug that shows when moving the data directory. The error will show in syslog like this:
Jan 12 10:03:12 ubuntu16 kernel: [ 1289.012262] audit: type=1400 audit(1515780192.392:116): apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" name="/proc/8834/status" pid=8834 comm="mysqld" requested_mask="r" denied_mask="r" fsuid=111 ouid=111
Jan 12 10:03:12 ubuntu16 kernel: [ 1289.012897] audit: type=1400 audit(1515780192.396:117): apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" name="/sys/devices/system/node/" pid=8834 comm="mysqld" requested_mask="r" denied_mask="r" fsuid=111 ouid=0
To really geek out, this bug has one of the more sad but funny threads I've read in a long time. They eventually get to the right conclusion and open two bugs, but getting there is a journey. We will patch it ourselves, because the bugs haven't been actually fixed in the Ubuntu release.
To update AppArmor:
sudo vim /etc/apparmor.d/usr.sbin.mysqld
In the "Allow system resource access" section add the following to fix the bug(s):
/sys/devices/system/node/ r,
/sys/devices/system/node/** r,
/proc/** r,
And under the "Allow data dir access" section change the entries from /var/lib/mysql/ to the new directory, /data/mysql/.
# Allow data dir access
# /var/lib/mysql/ r,
# /var/lib/mysql/** rwk,
/data/mysql/ r,
/data/mysql/** rwk,
Reload AppArmor:
sudo systemctl reload apparmor
Restart mysql:
service mysql start
The system should start without error if everything has been done correctly. Run the following command to ensure everything looks okay:
mysql -p -u redmine
Ruby Installation
Many methods exist to install Ruby. Unfortunately, the most convenient using apt-get will leave the system many versions behind, which means many plugins for redmine, and parts of redmine itself, will be unsupported. RVM is a tried and true method to provide clean management of ruby versions.
For whatever reason, these steps must be run as root:
sudo su - #get into root
apt-add-repository -y ppa:rael-gc/rvm
apt-get update
apt-get install rvm
Once done, logout and log back in as root.
exit
sudo su - #log out and log back in
command curl -sSL https://rvm.io/mpapis.asc | gpg --import -
rvmsudo rvm get stable
rvm requirements
rvm install 2.5.0
rvm use 2.5.0 --default
Test that the install looks okay.
ruby -v
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-linux]
Note: once setup, each user of rvm needs to be added to the rvm group.
sudo usermod -a -G rvm username
Phusion Passenger Installation
Again, many options exist for application containers. Phusion has a Passenger-Nginx combo that is straight-forward to install and configure. It does not need to be done as root.
Details of this installation can be found on the Phusion site.
sudo apt-get install -y dirmngr gnupg
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 561F9B9CAC40B2F7
sudo apt-get install -y apt-transport-https ca-certificates
sudo sh -c 'echo deb https://oss-binaries.phusionpassenger.com/apt/passenger xenial main > /etc/apt/sources.list.d/passenger.list'
sudo apt-get update
sudo apt-get install -y nginx-extras passenger
Nginx Configuration
Passenger configuration with nginx has simplified since previous versions.
Enable passenger in the nginx:
sudo vim /etc/nginx/nginx.conf
Uncomment the following line:
include /etc/nginx/passenger.conf;
Restart nginx:
sudo service nginx restart
Confirm everything looks correct
sudo /usr/bin/passenger-config validate-install
sudo /usr/sbin/passenger-memory-stats
The result will be something like this:
/usr/lib/ruby/vendor_ruby/phusion_passenger/locations.ini
Also, confirm the correct location of the correct ruby.
which passenger-config
/usr/bin/passenger-config #use this result to perform the next command
/usr/bin/passenger-config --ruby-command
The result will be something like this:
/usr/local/rvm/gems/ruby-2.5.0/wrappers/ruby
Build out the site - assuming http for now. More configuration will be needed to SSL-ize the system and lock it down. But for now, this will get things correct and running.
sudo cp /etc/nginx/sites-available/default /etc/nginx/sites-available/default.orig
sudo vim /etc/nginx/sites-available/default
Update the root location and add passenger configuration. The assumption is your redmine root location is also on the /data partition. This directory will be created when we add the redmine account.
root /data/redmine/redmine/public/; #installation location
passenger_enabled on; #turn on application container
client_max_body_size 10m; # Max attachement size allowed
Then to prevent a mess of 404 errors, comment out the location entry. Missing this step results in a special level of redmine 404 hell.
#location / {
# First attempt to serve request as file, then
# as directory, then fall back to displaying a 404.
#try_files $uri $uri/ =404;
#}
No need to restart nginx just yet. The system is almost ready for redmine installation.
Create the redmine account
Before doing too much with ruby, create a redmine service account. Note: Ideally the home directory is located where redmine is going to be installed - for production systems this should be on a separate partition.
sudo adduser --system --shell /bin/bash --gecos 'Redmine Administrator' --group --disabled-password --home /data/redmine redmine; sudo usermod -a -G rvm redmine
Give the account sudo privileges (temporarily).
sudo visudo
redmine ALL=(ALL) NOPASSWD:ALL
Welcome to dependency-o-rama
The ruby add-on dependencies next depends (ha ha get it?) on various ruby pieces needed to install ruby components. A minimal list will look something like this:
sudo apt-get install -y build-essential imagemagick libmagickwand-dev
Redmine, remember this was the main point of the article?
Whew. Like a hero that doesn't show up until the third reel of a movie, redmine is finally on the scene. Yes, this is just like Batman vs Superman. There is a lot of build up to the main event, and when you get there it is anticlimactic.
Install the latest redmine
Switch to the redmine user and pull down the latest stable release.
sudo su - redmine # should result in being in the redmine installation directory
wget http://www.redmine.org/releases/redmine-3.4.4.tar.gz
tar xvfz redmine-3.4.4.tar.gz
ln -s redmine-3.4.4 redmine
rm redmine-3.4.4.tar.gz
Configure the mysql connection
Update the production entry with the account connection information.
cd redmine
cp -pR config/database.yml.example config/database.yml
vim config/database.yml
Adding gems - truly outrageous!
Configure the gems - and avoid doing this as root. As with everything in redmine administration, this should be as that fancy redmine service account:
gem install bundler
bundle install --without development test
If versions and such match these instructions, the bundle install should go clean:
Bundle complete! 31 Gemfile dependencies, 55 gems now installed.
Gems in the groups development and test were not installed.
Use `bundle show [gemname]` to see where a bundled gem is installed.
If versions are different or new features are needed, some iteration may be needed to build and install the gems. This seems to be a "normal" task for ruby administrators. Apply google-fu and iterate.
Rake magic
Next run the magic rake commands. All magic comes in three.
bundle exec rake generate_secret_token
RAILS_ENV=production bundle exec rake db:migrate
RAILS_ENV=production bundle exec rake redmine:load_default_data
Now everything should be ready to start. This is done by restarting nginx. Monitor the following logs to ensure things start clean:
tail -f /var/log/nginx/error.log
tail -f /data/redmine/redmine/log/production.log
Clean up
Congratulations, the redmine site should now be up and available.
Remove sudo privileges from the redmine account. Move the site to HTTPS, ideally using the excellent Let's Encrypt service. These instructions can be found here..
Links of interest
General Redmine installation
Always a good place to review the latest information on generic Redmine installations.
https://www.redmine.org/projects/redmine/wiki/RedmineInstall
Using Google Authentication
If using Google Apps or Google Auth is of interest with Redmine, a longstanding plugin has been brought back to life with a patch.
The plugin can be found here:
https://github.com/twinslash/redmine_omniauth_google
But in modern ruby and google land, a patch will be needed. The plugin will install just fine, but it will not save any configuration information, nor will it put the google button on the login page without the patch below.
https://github.com/twinslash/redmine_omniauth_google/pull/42
Though out of date, this article has some pointers on setting up the google side:
https://adminsdiary.wordpress.com/common-installations/redmine-installation-with-google-authentication/
Installing RVM
https://github.com/rvm/ubuntu_rvm
Instead of MySQL, consider Aurora
For some of my more recent Redmine installations, I've been using Amazon Aurora. It is cheaper than MySQL on AWS, compatible with MySQL 5.6, backups are taken care of, and even has regional failover built in. Much better than researching AppArmor patches because you want to simply move a data directory.
Setting up redmine is the same as above, but instead of doing MySQL, connect to an AWS Aurora instance. (You will still need the MySQL client - libmysqlclient-dev):
https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Aurora.Connecting.html
Nginx HTTPS with Let's Encrypt and Redmine
These instructions are based on Ubuntu and use the Let's Encrypt Certificate Authority. With Let's Encrypt, self-signed certificates on a public Linux server should be ancient history.
These instructions assume the following;
- Ubuntu 14.04 LTS - a mature Ubuntu version with long term support
- Nginx - the web server within which Redmine runs
- A running Redmine system. Details of how to setup Redmine on Nginx can be found here
Getting the certificates from Let's Encrypt
As the Let's Encrypt system has just left beta and is still new, using their instructions are best. For example, the certbot command was introduced this week, replacing the "old" letsencrypt-auto command. However, here is how things work as of today.
Install git and download the Let's Encrypt client. On first run, the client will install dependencies and update itself.
apt-get install git
git clone https://github.com/certbot/certbot
cd certbot
# Running for the first time will install needed dependencies.
./certbot-auto --help
After everything is installed, the certificate can be requested and downloaded all in one step. Because the Let's Encrypt client uses port 80 for verification of ownership, anything running on that port needs to be temporarily turned off -- in this case nginx.
service nginx stop
./certbot-auto certonly --standalone -d redmine.yourdomain.com
service nginx start
Note where the certificates are installed. For example:
/etc/letsencrypt/live/redmine.yourdomain.com/
As said, these instructions may change at any time. What follows is the standard method for setting up SSL in general for nginx and Redmine.
Set up the certificates
Generate a set of DH parameters for the Diffie-Hellman handshake. Please don't ask what this is for... just know it is a good thing.
mkdir /etc/nginx/ssl
chmod 700 /etc/nginx/ssl
openssl dhparam 2048 -out /etc/nginx/ssl/dh2048.pem
In /etc/nginx/sites-available create a new file or update the "default" file.
Modify the listen 443 ssl entry.
server_name redmine.yourdomain.com;
ssl_dhparam /etc/nginx/ssl/dh2048.pem;
ssl_certificate /etc/letsencrypt/live/redmine.yourdomain.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/redmine.yourdomain.com/privkey.pem;
NOTE: The ssl_certificate should be set to the fullchain.pem, which includes both the server certificate and the intermediate CA certificates. Setting this just to the cert.pem will work in Chrome, but give a "SEC ERROR UNKNOWN ISSUER" error in Firefox.
Add the redmine details (same as used for port 80).
root /var/data/redmine/public/;
passenger_enabled on;
client_max_body_size 10m;
Comment out or delete any references to 404, if they exist.
#passenger_spawn_method direct;
#location / {
# try_files $uri $uri/ =404;
#}
If this file was newly created, instead of the "default" file, add a soft link to the new file in /etc/nginx/sites-enabled.
Restart nginx and everything should come up on port 443 with a valid certificate.
Lock down the SSL installation
Now that the site is functional, the configuration should be locked down.
# limit HTTPS to the most recent protocol
ssl_protocols TLSv1.2;
# define the list of ciphers used
ssl_prefer_server_ciphers on;
# this list is always in flux, but the list below works at time of writing
ssl_ciphers EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH;
#set the cache session, 10 minutes is the minimum
ssl_session_cache shared:SSL:10m;
In the end, the the file should look something like this:
server {
listen 443 ssl;
server_name redmine.yourdomain.com;
ssl_dhparam /etc/nginx/easy-rsa/keys/dh2048.pem;
ssl_certificate /etc/letsencrypt/live/redmine.yourdomain.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/redmine.yourdomain.com/privkey.pem;
ssl_protocols TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_ciphers EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH;
ssl_session_cache shared:SSL:10m;
root /var/data/redmine/public/;
passenger_enabled on;
client_max_body_size 10m;
#passenger_spawn_method direct;
#location / {
# try_files $uri $uri/ =404;
#}
}
Restart nginx and if desired, run a certificate security test against the system.
Figure: Qualsys scan
Automated renewal
Because the Let's Encrypt philosophy is full automation, the certificates need to be renewed every 90 days. In cron, setup a renewal job to be run every week. For example, at 3:05 AM every Saturday:
05 03 * * 6 /yourpath/cert-renew.sh
This job needs root privileges. Once the certificate gets into the time window where it can be renewed, cron will do the update "magically".
install-path/certbot-auto renew --standalone --pre-hook "service nginx stop" --post-hook "service nginx start"
This can be tested by doing a dry-run:
install-path/certbot/certbot-auto renew --dry-run --standalone --pre-hook "service nginx stop" --post-hook "service nginx start"
Links of interest
Renewal script for Let's Encrypt
https://certbot.eff.org/docs/using.html#renewal
Redmine with Passenger and Nginx on Ubuntu
Have to say this should be easier nowadays, but getting Redmine up and running can still be a maze of twisty little passages, all alike.
Trying to decide what is a good option and what is not requires a solid knowledge of Linux AND some mad Google-Fu. Cobbled from multiple sources, here are flexible instructions to get an up to date, robust, secure installation going.
- Ubuntu 14.04 LTS - a mature Ubuntu version with long term support
- MySQL - the database
- Ruby - the technology on which Redmine runs, installed using RVM to manage the ruby version and have access to up to date components
- Phusion Passenger- the application server in which to run Redmine
- Nginx - the web server within which Redmine runs
MySQL Configuration
Install MySQL. For this step, relying on the Ubuntu packages is fine.
sudo apt-get install -y mysql-server libmysqlclient-dev
For production systems, a separate data disk should be used rather that storing it on the same partition as the root system. This can be changed by modifying the data directory configuration in MySQL:
/etc/mysql/my.cnf
#datadir = /var/lib/mysql #old location
datadir = /data/mysql # new location
If going down this path, set permissions correctly on your new location:
chown mysql:mysql /data/mysql
chmod 700 /data/mysql
Update AppArmor, otherwise the mysql process won't start.
sudo vim /etc/apparmor.d/usr.sbin.mysqld
Restart mysql.
service mysql start
Connect to the mysql service (mysql -p) and create the database and provide access to the redmine user:
mysql -p
CREATE DATABASE redmine CHARACTER SET utf8;
CREATE USER 'redmine'@'localhost' IDENTIFIED BY 'yourpassword';
GRANT ALL PRIVILEGES ON redmine.* TO 'redmine'@'localhost';
Ruby Installation
Many methods exist to install Ruby. Unfortunately, the most convenient using apt-get will leave the system many versions behind, which means many plugins for redmine, and parts of redmine itself, will be unsupported. RVM is a tried and true method and provides clean management of ruby versions.
For whatever reason, these steps seem to work best as root.
sudo su -
gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3
curl -sSL https://get.rvm.io | bash -s stable
exit
Once setup, each user of rvm needs to be added to the rvm group.
sudo usermod -a -G rvm username
Logout and log back in to complete the installation.
rvm requirements
Now choose a ruby version. As guidance, a 2.0 or higher is recommended. RVM has pre-compiled versions, which make installation seamless.
rvm install 2.2.3
rvm use 2.2.3 --default
Then confirm everything looks correct.
ruby -v
ruby 2.2.3p173 (2015-08-18 revision 51636) [x86_64-linux]
Phusion Passenger Installation
Again, many options exist for application containers. Phusion has a Passenger-Nginx combo that is straight-forward to install and configure. It also installs an old version of ruby, but via configuration the package can be pointed to our more recent version.
Details of this installation can be found on the Phusion site.
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 561F9B9CAC40B2F7
sudo sh -c 'echo deb https://oss-binaries.phusionpassenger.com/apt/passenger trusty main > /etc/apt/sources.list.d/passenger.list'
sudo apt-get update
sudo apt-get install nginx-extras passenger
Nginx Configuration
Nginx has to be pointed to the correct ruby and the correct passenger installation.
Ensure passenger is installed correctly and confirm the correct location. This location will be needed for nginx.
/usr/bin/passenger-config validate-install
passenger-config --root
The result will be something like this:
/usr/lib/ruby/vendor_ruby/phusion_passenger/locations.ini
Also, confirm the correct location of the correct ruby.
which passenger-config
/usr/bin/passenger-config #use this result to perform the next command
/usr/bin/passenger-config --ruby-command
The result will be something like this:
/usr/local/rvm/gems/ruby-2.2.3/wrappers/ruby
Update nginx to point to the correct location. Don't forget semi-colons at the end!
sudo vim /etc/nginx/nginx.conf
# passenger_root /usr/lib/ruby/vendor_ruby/phusion_passenger/locations.ini; #old entry
# passenger_ruby /usr/bin/passenger_free_ruby; #old entry
passenger_root /usr/lib/ruby/vendor_ruby/phusion_passenger/locations.ini;
passenger_ruby /usr/local/rvm/gems/ruby-2.2.3/wrappers/ruby;
Restart nginx
sudo service nginx restart
Enable the www directory.
sudo mkdir /var/www
sudo chown -R www-data:www-data /var/www
Build out the site - assuming http for now. More configuration will be needed to SSL-ize the system and lock it down. But for now, this will get things correct and running.
cp /etc/nginx/sites-available/default /etc/nginx/sites-available/default.orig
vim /etc/nginx/sites-available/default
Update the root location and add passenger configuration.
root /data/redmine/redmine/public/; #installation location
passenger_enabled on; #turn on application container
client_max_body_size 10m; # Max attachement size allowed
Then to prevent a mess of 404 errors, comment out the location entry. Missing this step results in a special level of redmine 404 hell.
#location / {
# First attempt to serve request as file, then
# as directory, then fall back to displaying a 404.
#try_files $uri $uri/ =404;
#}
The system is almost ready for redmine installation.
Create the redmine account
Before doing too much with ruby, create a redmine service account. Note: Ideally the home directory located where redmine is going to be installed - for production systems this should be on a separate partition.
sudo adduser --system --shell /bin/bash --gecos 'Redmine Administrator' --group --disabled-password --home /data/redmine redmine; sudo usermod -a -G rvm redmine
Give the account sudo privileges (temporarily).
sudo visudo
redmine ALL=(ALL) NOPASSWD:ALL
Welcome to dependency-o-rama
The ruby add-on dependencies next depends (ha ha get it?) on various ruby pieces needed to install ruby components. A minimal list will look something like this:
sudo apt-get install -y build-essential imagemagick libmagickwand-dev
Redmine, remember this was the main point of the article?
Whew. Like a hero that doesn't show up until reel 3, redmine is finally on the scene.
Install the latest
Switch to the redmine user and pull down the latest stable release.
sudo su - redmine # should result in being in the redmine installation directory
wget http://www.redmine.org/releases/redmine-3.2.1.tar.gz
tar xvfz redmine-3.2.1.tar.gz
ln -s redmine-3.2.1 redmine
rm redmine-3.2.1.tar.gz
Configure the mysql connection
Update the production entry with the account connection information.
cd redmine
cp -pR config/database.yml.example config/database.yml
vim config/database.yml
Adding gems - truly outrageous!
Configure the gems - and avoid doing this as root. As with everything in redmine administration, this should be as that fancy redmine service account:
gem install bundler
bundle install --without development test
If versions and such match these instructions, the bundle install should go clean:
Bundle complete! 30 Gemfile dependencies, 54 gems now installed.
Gems in the groups development and test were not installed.
Use `bundle show [gemname]` to see where a bundled gem is installed.
If versions are different or new features are needed, some iteration may be needed to build and install the gems. This seems to be a "normal" task for ruby administrators. Apply google-fu and iterate.
Rake magic
Next run the magic rake commands. All magic comes in three.
bundle exec rake generate_secret_token
RAILS_ENV=production bundle exec rake db:migrate
RAILS_ENV=production bundle exec rake redmine:load_default_data
Now everything should be ready to start. This is done by restarting nginx. Monitor the following logs to ensure things start clean:
tail -f /var/log/nginx/error.log
tail -f /data/redmine/redmine/log/production.log
Clean up
Congratulations, the redmine site should now be up and available.
Remove sudo privileges from the redmine account. Move the site to SSL, ideally using the excellent Let's Encrypt service. These instructions can be found here..
Links of interest
Plenty of chatter and individual parts here to help troubleshoot and see where much of this has been culled. Enjoy.
Ways to install ruby on Ubuntu
http://stackoverflow.com/questions/26595620/how-to-install-ruby-2-1-4-on-ubuntu-14-04
https://gorails.com/setup/ubuntu/14.04
https://www.digitalocean.com/community/tutorials/how-to-install-rails-and-nginx-with-passenger-on-ubuntu
http://stackoverflow.com/questions/5201689/rmagick-gem-install-cant-find-magick-config
Ways to install Nginx and Passenger
https://www.digitalocean.com/community/tutorials/how-to-deploy-a-rails-app-with-passenger-and-nginx-on-ubuntu-14-04
http://www.redmine.org/projects/redmine/wiki/HowTo_configure_Nginx_to_run_Redmine
Troubleshooting Passenger installations
https://www.phusionpassenger.com/library/config/nginx/reference/#passenger_root
https://www.phusionpassenger.com/library/admin/nginx/troubleshooting/ruby/
https://www.phusionpassenger.com/library/config/nginx/reference/#setting_correct_passenger_ruby_value
Ways to install Redmine
http://www.redmine.org/projects/redmine/wiki/HowTos
http://www.redmine.org/projects/redmine/wiki/RedmineInstall#fn0
https://blog.rudeotter.com/install-redmine-with-nginx-puma-and-mariadbmysql-on-ubuntu-14-04/
http://www.redminecrm.com/boards/4/topics/448-installing-redmine-2-2-passenger-nginx-rvm-on-ubuntu-12-04
https://nidomiro.de/2015/03/installing-redmine-3-0-on-clean-ubuntu-14-04/
http://www.redmine.org/projects/redmine/wiki/HowTo_Install_Redmine_30x_on_Ubuntu_1404_with_Apache2_Phusion_Passenger_MySQL_Subversion_and_Git_%28Gitolite%29#Installing-Ruby
