Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .rubocop.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,5 +8,7 @@ AllCops:
NewCops: enable

Metrics/BlockLength:
Exclude:
- Rakefile
ExcludedMethods:
- route
28 changes: 14 additions & 14 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ ENV RACK_ENV=production
ENV PATH="/app/bin:${PATH}"

HEALTHCHECK --interval=30m --timeout=60s --start-period=5s \
CMD curl -f http://localhost:3000/health_check.txt || exit 1
CMD curl -f http://${HEALTH_CHECK_USERNAME}:${HEALTH_CHECK_PASSWORD}@localhost:3000/health_check.txt || exit 1

RUN apk add --no-cache \
'git=~2' \
Expand All @@ -23,26 +23,26 @@ ARG UID=991
ARG GID=991

RUN mkdir /app \
&& addgroup --gid "$GID" "$USER" \
&& adduser \
--disabled-password \
--gecos "" \
--home "/app" \
--ingroup "$USER" \
--no-create-home \
--uid "$UID" \
"$USER" \
&& chown "$USER":"$USER" -R /app
&& addgroup --gid "$GID" "$USER" \
&& adduser \
--disabled-password \
--gecos "" \
--home "/app" \
--ingroup "$USER" \
--no-create-home \
--uid "$UID" \
"$USER" \
&& chown "$USER":"$USER" -R /app

WORKDIR /app

USER html2rss

COPY --chown=html2rss:html2rss Gemfile Gemfile.lock ./
RUN gem install bundler:'<3' \
&& bundle config set --local without 'development test' \
&& bundle install --retry=5 --jobs=7 \
&& bundle binstubs bundler html2rss
&& bundle config set --local without 'development test' \
&& bundle install --retry=5 --jobs=7 \
&& bundle binstubs bundler html2rss

COPY --chown=html2rss:html2rss . .

Expand Down
46 changes: 28 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,8 @@ services:
read_only: true
environment:
- RACK_ENV=production
- HEALTH_CHECK_USERNAME=health
- HEALTH_CHECK_PASSWORD=please-set-YOUR-OWN-veeeeeery-l0ng-aNd-h4rd-to-gue55-Passw0rd!
watchtower:
image: containrrr/watchtower
volumes:
Expand All @@ -71,13 +73,14 @@ html2rss-web comes with many feed configs out of the box. [See file list of all

To use a config from there, build the URL like this:

The _feed config_ you'd like to use:
`lib/html2rss/configs/domainname.tld/whatever.yml`
`‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌^^^^^^^^^^^^^^^^^^^^^^^^^^^`
Build the URL of the _feed config_ you'd like to use like this:

The corresponding URL:
`http://localhost:3000/domainname.tld/whatever.rss`
`‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ^^^^^^^^^^^^^^^^^^^^^^^^^^^`
| | |
| -----------------------: | :---------------------------- |
| `lib/html2rss/configs/` | `domainname.tld/whatever.yml` |
| Would becomes this URL: | |
| `http://localhost:3000/` | `domainname.tld/whatever.rss` |
| | `^^^^^^^^^^^^^^^^^^^^^^^^^^^` |

## How to build your RSS feeds

Expand Down Expand Up @@ -112,24 +115,31 @@ If you're going to host a public instance, _please please please_:

### Supported ENV variables

| Name | Description |
| ------------------------------ | ---------------------- |
| `PORT` | default: 3000 |
| `RACK_ENV` | default: 'development' |
| `RACK_TIMEOUT_SERVICE_TIMEOUT` | default: 15 |
| `WEB_CONCURRENCY` | default: 2 |
| `WEB_MAX_THREADS` | default: 5 |
| Name | Description |
| ------------------------------ | -------------------------------- |
| `PORT` | default: 3000 |
| `RACK_ENV` | default: 'development' |
| `RACK_TIMEOUT_SERVICE_TIMEOUT` | default: 15 |
| `WEB_CONCURRENCY` | default: 2 |
| `WEB_MAX_THREADS` | default: 5 |
| `HEALTH_CHECK_USERNAME` | default: auto generated on start |
| `HEALTH_CHECK_PASSWORD` | default: auto generated on start |

### Runtime monitoring via `GET /health_check.txt`

It is recommended to setup a monitoring of the `/health_check.txt` endpoint. With that, you can be notified when one of _your own_ configs break.
It is recommended to setup a monitoring of the `/health_check.txt` endpoint. With that, you can find out when one of _your own_ configs break. The endpoint uses HTTP Basic authentication.

The `GET /health_check.txt` endpoint responds with:
First, set username and password via these environment variables: `HEALTH_CHECK_USERNAME` and `HEALTH_CHECK_PASSWORD`. If these are not set, html2rss-web will generate a new random username and password on _each_ start.

- if the feeds are generatable: it will respond with `success` .
- otherwise: it states the broken config names.
An authenticated `GET /health_check.txt` request will be responded with:

[UptimeRobot's free plan](https://uptimerobot.com/) is sufficent for basic monitoring every 5 minutes. Create a monitor of type _Keyword_ with this information:
- if the feeds are generatable: `success`.
- otherwise: the names of the broken configs.

To get notified when one of your configs breaks, setup a monitoring of this endpoint.

[UptimeRobot's free plan](https://uptimerobot.com/) is sufficent for basic monitoring (every 5 minutes).
Create a monitor of type _Keyword_ with this information and make it aware of your username and password:

![A screenshot showing the Keyword Monitor: a name, the instance's URL to /health_check.txt and an interval.](docs/uptimerobot_monitor.jpg)

Expand Down
49 changes: 44 additions & 5 deletions Rakefile
Original file line number Diff line number Diff line change
@@ -1,28 +1,67 @@
# frozen_string_literal: true

##
# Helper methods used during :test run
module Output
module_function

def describe(msg)
puts ''
puts '*' * 80
puts "* #{msg}"
puts '*' * 80
puts ''
end

def wait(seconds, message:)
print message if message

seconds.times do |i|
putc ' '
putc (i + 1).to_s
sleep 1
end
puts '. Time is up.'
end
end

task default: %w[test]

desc 'Build and run docker image/container, and send requests to it'

task :test do
current_dir = ENV.fetch('GITHUB_WORKSPACE', __dir__)

Output.describe 'Building and running'
sh 'docker build -t gilcreator/html2rss-web -f Dockerfile .'
sh ['docker run',
'-d',
'-p 3000:3000',
'--env PUMA_LOG_CONFIG=1',
'--env HEALTH_CHECK_USERNAME=username',
'--env HEALTH_CHECK_PASSWORD=password',
"--mount type=bind,source=#{current_dir}/config,target=/app/config",
'--name html2rss-web-test',
'gilcreator/html2rss-web'].join(' ')

# wait for container to run and accept connections
sleep 5
sh 'docker ps -a'
Output.wait 5, message: 'Waiting for container to start:'

Output.describe 'Listing docker containers matching html2rss-web-test filter'
sh 'docker ps -a --filter name=html2rss-web-test'

Output.describe 'Generating feed from a html2rss-configs config'
sh 'curl -f http://127.0.0.1:3000/github.com/releases.rss\?username=html2rss\&repository=html2rss || exit 1'

Output.describe 'Generating example feed from feeds.yml'
sh 'curl -f http://127.0.0.1:3000/example.rss || exit 1'

Output.describe 'Authenticated request to GET /health_check.txt'
sh 'curl -f http://username:password@127.0.0.1:3000/health_check.txt || exit 1'

sh 'curl -f http://127.0.0.1:3000/github.com/releases.rss\?username=nuxt\&repository=nuxt.js || exit 1'
sh 'curl -f http://127.0.0.1:3000/health_check.txt || exit 1'
Output.describe 'Print output of `html2rss help`'
sh 'docker exec html2rss-web-test html2rss help'
ensure
Output.describe 'Cleaning up'
sh 'docker logs --tail all html2rss-web-test'
sh 'docker stop html2rss-web-test'
sh 'docker rm html2rss-web-test'
Expand Down
9 changes: 8 additions & 1 deletion app.rb
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@
require_relative './app/local_config'
require_relative './app/html2rss_facade'

require_relative 'roda/roda_plugins/basic_auth'

module App
##
# This app uses html2rss and serves the feeds via HTTP.
Expand Down Expand Up @@ -70,6 +72,7 @@ class App < Roda
plugin :public
plugin :render, escape: true, layout: 'layout'
plugin :typecast_params
plugin :basic_auth

route do |r|
path = RequestPath.new(request)
Expand All @@ -83,7 +86,11 @@ class App < Roda
r.get 'health_check.txt' do |_|
HttpCache.expires_now(response)

HealthCheck.run
with_basic_auth(realm: HealthCheck,
username: HealthCheck::Auth.username,
password: HealthCheck::Auth.password) do
HealthCheck.run
end
end

# Route for feeds from the local feeds.yml
Expand Down
22 changes: 21 additions & 1 deletion app/health_check.rb
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,28 @@

module App
##
# Checks if the local configs generate valid RSS feeds.
# Checks if the local configs are generatable.
module HealthCheck
##
# Contains logic to obtain username and password to be used with HealthCheck endpoint.
class Auth
def self.username
@username ||= ENV.delete('HEALTH_CHECK_USERNAME') do
SecureRandom.base64(32).tap do |string|
puts "HEALTH_CHECK_USERNAME env var. missing! Please set it. Using generated value instead: #{string}"
end
end
end

def self.password
@password ||= ENV.delete('HEALTH_CHECK_PASSWORD') do
SecureRandom.base64(32).tap do |string|
puts "HEALTH_CHECK_PASSWORD env var. missing! Please set it. Using generated value instead: #{string}"
end
end
end
end

module_function

##
Expand Down
49 changes: 49 additions & 0 deletions roda/roda_plugins/basic_auth.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# frozen_string_literal: true

require 'roda'
require 'openssl'

class Roda
##
# Roda's plugin namespace
module RodaPlugins
##
# Basic Auth plugin's namespace
module BasicAuth
def self.authorize(username, password, auth)
given_user, given_password = auth.credentials

secure_compare(username, given_user) & secure_compare(password, given_password)
end

def self.secure_compare(left, right)
left.bytesize == right.bytesize && OpenSSL.fixed_length_secure_compare(left, right)
end

##
# Methods here become instance methods in the roda application.
module InstanceMethods
def with_basic_auth(realm:, username:, password:)
raise ArgumentError, 'realm must not be a blank string' if realm.to_s.strip == ''

response.headers['WWW-Authenticate'] = "Basic realm=#{realm}"

auth = Rack::Auth::Basic::Request.new(env)

if auth.provided? && Roda::RodaPlugins::BasicAuth.authorize(username, password, auth)
yield if block_given?
else
unauthorized
end
end

def unauthorized
response.status = 401
request.halt response.finish
end
end
end

register_plugin(:basic_auth, BasicAuth)
end
end
24 changes: 24 additions & 0 deletions spec/app/health_check/auth_spec.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# frozen_string_literal: true

require 'spec_helper'
require_relative '../../../app/health_check'

RSpec.describe App::HealthCheck::Auth do
before do
allow(ENV).to receive(:delete).with(any_args).and_call_original
end

describe '.username' do
it 'deletes the ENV var', :aggregate_failures do
expect(described_class.username).to be_a String
expect(ENV).to have_received(:delete).with('HEALTH_CHECK_USERNAME').once
end
end

describe '.password' do
it 'deletes the ENV var', :aggregate_failures do
expect(described_class.password).to be_a String
expect(ENV).to have_received(:delete).with('HEALTH_CHECK_PASSWORD').once
end
end
end
55 changes: 55 additions & 0 deletions spec/roda/roda_plugins/basic_auth_spec.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# frozen_string_literal: true

require 'spec_helper'
require 'rack'
require_relative '../../../roda/roda_plugins/basic_auth'

RSpec.describe Roda::RodaPlugins::BasicAuth do
before do
allow(Roda::RodaPlugins).to receive(:register_plugin).with(:basic_auth, described_class)
end

describe '.authorize(username, password, auth)' do
context 'with correct credentials' do
it {
username = 'foo'
password = 'bar'
auth = instance_double(Rack::Auth::Basic::Request, credentials: %w[foo bar])

expect(described_class.authorize(username, password, auth)).to be true
}
end

context 'with wrong credentials' do
it {
username = ''
password = ''
auth = instance_double(Rack::Auth::Basic::Request, credentials: %w[foo bar])

expect(described_class.authorize(username, password, auth)).to be false
}
end
end

describe '.secure_compare(left, right)' do
context 'with left being same as right' do
let(:left) { 'something-asdf' }
let(:right) { 'something-asdf' }

it 'uses OpenSSL.fixed_length_secure_compare', :aggregate_failures do
allow(OpenSSL).to receive(:fixed_length_secure_compare).with(left, right).and_call_original

expect(described_class.secure_compare(left, right)).to be true

expect(OpenSSL).to have_received(:fixed_length_secure_compare).with(left, right)
end
end

context 'with left being different from right' do
it 'returns false', :aggregate_failures do
expect(described_class.secure_compare('left', 'right')).to be false
expect(described_class.secure_compare('lefty', 'right')).to be false
end
end
end
end