Forking Docker Discourse for configuration beyond plugins

When I first discovered Docker and was able to get its development version running locally in a couple minutes, I was very optimistic for my ability to hack on this project and deploy it with a couple select updates for our needs with GitClear and Amplenote. However, I was soon to discover to that deploying this Rails app on any cloud provider is no small feat, even for an experienced Rails developer with adequate devops abilities. Below I've included the steps that I identified were necessary to deploy it to Heroku. After around 10 hours of work, I concluded that a direct deploy was not going to be a path of low resistance: see Attempting to install Discourse directly if you're curious why.


At several junctures in the painful process of trying to get a functional fork, I considered the Discourse Approved™️ route of augmenting via plugin. But our reasons for needing a forum couldn't be covered by plugins. So eventually, the best solution was...


linkSwapping to Discourse fork in Docker

Much like I thought directly installing the Rails app to Heroku ought not be too difficult, I wrongly deduced this might only be one or two steps. But the state of documentation on how to pull this off is scattered among ancient forum posts. This was the shortest set of steps I could deduce to get a Docker-based Discourse install to use a forked Discourse repo.


link0. Fork discourse/discourse to your own repo

You should know that Discourse (inasmuch as they are represented by Sam Saffron) has been pretty consistent in the forum posts I've seen that they don't advise this. It was hard to pinpoint their aversion. In part it seemed due to the confused inquiries they have received from previous developers who tried and were frustrated in their attempt to deploy their own version of Discourse. It's unclear to me at this time if there are other reasons.


link1. Setup a cloud server, pull the Discourse Docker repo & follow the Discourse cloud install instructions

This should include cloning docker_discourse.git into a directory on your cloud server, and running their their usual ./discourse-setup script where you'll provide email credentials and a domain to be used. As I learned, you definitely don't want to run discourse-setup more than five times total or you'll get rated limited on LetsEncrypt certificates to your domain for a week. 😰


link2. Backup and edit projectroot/containers/app.yml

Running discourse-setup will generate a YAML configuration file, app.yml, in the containers directory. This file allows the simplest opportunity to change the repo that's used for the Discourse install. As suggested by this helpful old forum post, you can add the following new block in the hooks section of the app.yml file:

hooks:
after_code:
- exec:
cd: $home
cmd:
- git remote remove origin
- git remote add origin https://github.com/your-org/discourse.git
- git fetch
- git remote set-branches --add origin main develop
- git branch -u origin/develop HEAD
- git reset --hard origin/develop
... exec for plugin code usually follows here


Initially when I tried to change the app.yml file it didn't work, so I also found an Alternate approach to changing Docker Discourse location that can replace steps #1 and #2 if these steps don't work and you are OK with forking the discourse_docker repo.


link3. Change your discourse fork and merge it to the tests-passed branch

In your Docker fork, update a file to prove that your fork is being used, such as app/views/finish_installation.index.html.erb. I recommend making all changes from a branch, since you'll want to apply your changes on top of the existing tests-passed branch that is used by Docker to install Discourse. If you work in main, you'll be creating some risk that untested changes from discourse/discourse's main branch will be included in your build.


After you push changes to your development branch, you'll want to switch to the tests-passed branch, merge the develop branch, and push the merged tests-passed branch to origin.


link4. Rebuild the container

Back in the project root directory, you can now run ./launcher rebuild app and the updated app.yml file (or updated web.template.yml if you used the Alternate Approach) should incorporate your forked repo as the one that is used to build the served Discourse install.


Todo: How to incorporate newly pushed changes in fork after having built container? Does whole container really have to get torn down and rebuilt?


linkWorking with the Docker install

Once you have Docker up and running, these are a few commands that help to debug issues.


linkLogging into Docker

To log in to the Docker install, which can useful for debugging:


docker ps and then docker exec -it containerpid /bin/bash


Or from project directory when app is running:


./launcher enter app


linkEntering Rails console

To enter Rails console, you need to be the discourse user. This should work if you're logged into the Docker container as root:


$ su - discourse
$ cd /var/www/discourse
$ RAILS_ENV=production bundle exec rails c


linkRestarting Unicorn

ps aux | grep "unicorn master"
kill -HUP [pid of process]


Seems to take about a minute, during which time you might see a 502 response. There's probably a more graceful restart option available?


linkAlternate approach: Attempt to install Discourse directly

LinuxBabe offers the most modern & comprehensive guide I could locate for undertaking the task. It's 10+ steps and might take an hour or two if everything goes right?


I'm also including my own notes for installing via Heroku, which has some of its own quirks. I eventually abandoned my attempt to get the deploy working when I continually ran into a deploy-specific error with Uglifier claiming it could not handle the { syntax. The guides claimed that changing config.assets.js_compressor = Uglifier.new(harmony: true) would cure my woes. It did get the command to work when I ran assets:precompile in production environment locally, but when I tried to push to Heroku, it continued to fail. I later came to believe this might have been because the Uglifier call used to compress ruby needed to have the harmony: true param added to it.


linkPrereqs

Postgres (supposedly postgres 13, not verified)

Redis

Node

Brotli


linkInstallation

linkSet environment variables

Undocumented environment variables are necessary:

DISCOURSE_DB_USERNAME
DISCOURSE_DB_PASSWORD
DISCOURSE_DB_HOST
DISCOURSE_DB_NAME
DISCOURSE_REDIS_HOST
DISCOURSE_REDIS_USERNAME
DISCOURSE_REDIS_PASSWORD

Variables are documented in lower case form at discourse_default.conf.


linkConfigure Postgres

This guide was what I used.


linkSet up Brotli

Option #1 is to not use Brotli by commenting it out in assets.rake. Haven't tested this path yet.


linkSetup up buildpacks

Option #2 is to install it as a prereq. Per Heroku help docs, to add packages one needs to use the experimental buildpack

heroku buildpacks:add --index 1 heroku-community/apt

Which gives access to any package listed at https://packages.ubuntu.com/ by adding their name to an Aptfile added to the project root:

# ./Aptfile
brotli

This means you're going to be using two buildpacks, one for the heroku-community/apt, one for heroku/ruby.


linkDebug assets:precompile configuration

RAILS_ENV=production DISCOURSE_DB_HOST= DISCOURSE_DB_USERNAME= DISCOURSE_DB_NAME= DISCOURSE_DB_PASSWORD= DISCOURSE_REDIS_HOST= DISCOURSE_REDIS_PASSWORD= DISCOURSE_REDIS_PORT= rake assets:precompile


linkProcfile

The file Procfile should be placed in the root directory of the file

web: bundle exec puma -C config/puma.rb
worker: bundle exec sidekiq -e $RAILS_ENV


linkUnsolved mysteries for direct install

How to adapt to Heroku's threat that they will arbitrarily swap db & redis credentials used for DISCOURSE_REDIS_HOST and DISCOURSE_DB_HOST?



How to specify credentials to be used for S3? (these seem to be part of Discourse admin settings?)



Setup SMTP (should just be adding appropriate ENV vars)