Configuring WordPress to work behind an Application Load Balancer (ALB) in AWS

Leave a reply

When putting WordPress behind an ALB that has SSL configured it might result in a configuration where the ALB uses SSL but WordPress communicates with the ALB over regular HTTP.

This can cause WordPress to server HTTP (non-ssl) CSS and JavaScript resources and/or fail in other ways.

The solution is to check the X-Forwarded-Proto header that ALB sets and let WordPress know whether it should treat the incoming request as an SSL request or not.

Put this code at the top of wp-config.php to do exactly that:

// Get true SSL status from AWS load balancer
if(isset($_SERVER['HTTP_X_FORWARDED_PROTO']) && $_SERVER['HTTP_X_FORWARDED_PROTO'] === 'https') {
  $_SERVER['HTTPS'] = '1';
}

Cache warming on low-traffic WordPress sites

Leave a reply

If you have a WordPress site that relies on time-based static page caching (like W3 Total Cache), you may notice that the low default cache timeouts (1-5 minutes) aren’t really working well for sites that receive a low amount of traffic.

In these cases, cache warming provides a great option to make sure every visitor gets a fast TTFB response.

I recently wrote a PHP script to enable cache warming based on a sitemap. (Which both Yoast WordPress SEO and Google XML Sitemaps provide.)

You can find the script and setup instructions by clicking on this link.

Caching strategy for low-traffic WordPress deployments

Let’s talk about caching strategy! Sites fall into one of two categories:

Low amount of content (~20-30 unique posts or pages)

Use a long cache expiration time (24 hours or more), but make sure that the cache is flushed when any content changes. (This is usually configurable in the cache plugin.)

Schedule crawls generously, as crawling will be fast as long as cache hasn’t expired. Crawling every 15 minutes or so is no problem.

Large amounts of content (> 100 posts or pages)

If you have a lot of infrequently accessed pages, the problem gets harder to manage. Crawling hundreds of pages will take a toll on your server, racking up high CPU usage. The only option here is to cache indefinitely, to only flush parts of the cache that are affected by a change, or cache for even longer periods of time (over 1 week).

For caching, I recommend the free Cache Enabler.

Schedule crawles less frequently, once per 1-4 hours. Keep track of the time it takes to run a crawl. If your web host enforces low PHP timeout the crawler might get killed before crawling all the pages.

Automatic dynamic DNS updater using EdgeRouter / EdgeOS and Internet.bs

Leave a reply

EdgeOS already supports Python (as of 1.10.9), so let’s write a short Python script. Replace with your username, password and domain on line 13 in the script.

import urllib2
import datetime
import os

print 'IP Updater - ' + datetime.datetime.now().isoformat()

f = os.popen('/sbin/ifconfig eth0 | grep "inet\ addr" | cut -d: -f2 | cut -d" " -f1')
ip=f.read()

print 'IP detected as: ' + ip

if(ip):
  sendOne = urllib2.urlopen('http://dyndns.topdns.com/update?hostname=my-domain.com&username=user&password=password&myip=' + ip).read()
  print sendOne

Schedule it to run recurrently:

touch /var/log/ibs-update.log
crontab -e

Add the following line:

0 * * * * python /home/ubnt/ibs-update.py >> /var/log/ibs-update.log

Note: /var/log is mounted in-memory on EdgeOS, so it’s not going to introduce wear-and-tear on the flash memory.

vnstat on EdgeRouter – historical bandwidth monitoring and graphical dashboard tutorial

4 Replies

This post will show you how to install vnstat and vnstati on the EdgeRouter for bandwidth monitoring, as well as how to create a graphical dashboardwith historical bandwidth data.

Installation

Add non-free sources to APT:

EdgeOS 1.X

configure
set system package repository wheezy components 'main contrib non-free' 
set system package repository wheezy distribution wheezy 
set system package repository wheezy url http://archive.debian.org/debian
commit ; save
sudo apt-get update

EdgeOS 2.X

configure
set system package repository stretch components 'main contrib non-free' 
set system package repository stretch distribution stretch
set system package repository stretch url http://http.us.debian.org/debian
commit ; save
sudo apt-get update

Install the packages

sudo apt-get install vnstat vnstati

vnstat configuration

Edit the config file in /etc/vnstat.conf to be like the following. This will make sure your bandwidth data will survive a firmware update. (However, you’ll have to reinstall and reconfigure vnstat / vnstati after a firmware update).

# vnStat 1.11 config file
##

# default interface
Interface "eth0"

# location of the database directory
DatabaseDir "/var/lib/vnstat"

# locale (LC_ALL) ("-" = use system locale)
Locale "-"

# on which day should months change
MonthRotate 1

# date output formats for -d, -m, -t and -w
# see 'man date' for control codes
DayFormat    "%x"
MonthFormat  "%b '%y"
TopFormat    "%x"

# characters used for visuals
RXCharacter       "%"
TXCharacter       ":"
RXHourCharacter   "r"
TXHourCharacter   "t"

# how units are prefixed when traffic is shown
# 0 = IEC standard prefixes (KiB/MiB/GiB/TiB)
# 1 = old style binary prefixes (KB/MB/GB/TB)
UnitMode 0

# output style
# 0 = minimal & narrow, 1 = bar column visible
# 2 = same as 1 except rate in summary and weekly
# 3 = rate column visible
OutputStyle 3

# used rate unit (0 = bytes, 1 = bits)
RateUnit 1

# maximum bandwidth (Mbit) for all interfaces, 0 = disable feature
# (unless interface specific limit is given)
MaxBandwidth 1000

# interface specific limits
#  example 8Mbit limit for eth0 (remove # to activate):
#MaxBWeth0 8

# how many seconds should sampling for -tr take by default
Sampletime 5

# default query mode
# 0 = normal, 1 = days, 2 = months, 3 = top10
# 4 = dumpdb, 5 = short, 6 = weeks, 7 = hours
QueryMode 0

# filesystem disk space check (1 = enabled, 0 = disabled)
CheckDiskSpace 1

# database file locking (1 = enabled, 0 = disabled)
UseFileLocking 1

# how much the boot time can variate between updates (seconds)
BootVariation 15

# log days without traffic to daily list (1 = enabled, 0 = disabled)
TrafficlessDays 1


# vnstatd
##

# how often (in seconds) interface data is updated
UpdateInterval 30

# how often (in seconds) interface status changes are checked
PollInterval 5

# how often (in minutes) data is saved to file
SaveInterval 60

# how often (in minutes) data is saved when all interface are offline
OfflineSaveInterval 60

# force data save when interface status changes (1 = enabled, 0 = disabled)
SaveOnStatusChange 0

# enable / disable logging (0 = disabled, 1 = logfile, 2 = syslog)
UseLogging 1

# file used for logging if UseLogging is set to 1
LogFile "/var/log/vnstat.log"

# file used as daemon pid / lock file
PidFile "/var/run/vnstat.pid"


# vnstati
##

# title timestamp format
HeaderFormat "%x %H:%M"

# show hours with rate (1 = enabled, 0 = disabled)
HourlyRate 1

# show rate in summary (1 = enabled, 0 = disabled)
SummaryRate 1

# layout of summary (1 = with monthly, 0 = without monthly)
SummaryLayout 1

# transparent background (1 = enabled, 0 = disabled)
TransparentBg 0

# image colors
CBackground     "FFFFFF"
CEdge           "AEAEAE"
CHeader         "606060"
CHeaderTitle    "FFFFFF"
CHeaderDate     "FFFFFF"
CText           "000000"
CLine           "B0B0B0"
CLineL          "-"
CRx             "92CF00"
CTx             "606060"
CRxD            "-"
CTxD            "-"

Using vnstat

Monthly bandwidth

vnstat -m -i eth0

Daily bandwidth

vnstat -d -i eth0

Live bandwidth usage

vnstat -l -i eth0

Configure vnstati to generate images for bandwidth dashboard

Create the file /var/lib/vnstat/vnstati-update.sh with the content:

#!/bin/bash
vnstati -s -i eth0 -o /var/www/htdocs/media/vnstat-summary.png
vnstati -h -i eth0 -o /var/www/htdocs/media/vnstat-hourly.png
vnstati -m -i eth0 -o /var/www/htdocs/media/vnstat-monthly.png
vnstati -d -i eth0 -o /var/www/htdocs/media/vnstat-daily.png

Make it executable:

chmod +x /var/lib/vnstat/vnstati-update.sh

Schedule the script to run every hour to keep the images up to date:

crontab -e

Add the line:

0 * * * * /var/lib/vnstat/vnstati-update.sh

Set up a dashboard

Create the file /var/www/htdocs/media/dashboard.html

Add the content below. Replace 192.168.10.1 with your router IP.

<!DOCTYPE html>
<html>
<head>
    <meta charset="UTF-8">
    <title>BW Dashboard</title>
    <style>
        body {
            text-align: center;
        }
    </style>
</head>

<body>
<h3>Overview</h3>
<img src="https://192.168.10.1/media/vnstat-summary.png">

<h3>24 hour</h3>
<img src="https://192.168.10.1/media/vnstat-hourly.png">

<h3>Monthly</h3>
<img src="https://192.168.10.1/media/vnstat-monthly.png">

<h4>Monthly</h4>
<img src="https://192.168.10.1/media/vnstat-daily.png">
</body>
</html>

Now you can visit your dashboard at https://192.168.10.1/media/dashboard.html (replace with your router IP).

Cleanup

sudo apt-get clean && sudo apt-get autoclean && sudo apt-get autoremove && rm /var/cache/apt/pkgcache.bin /var/cache/apt/srcpkgcache.bin

More reading

https://community.ubnt.com/t5/EdgeRouter/data-usage-monitoring-using-vnstat-cli-only-HOWTO/td-p/1061213

Visualize disk usage on linux with ncdu

Leave a reply

ncdu is a great tool for visualizing your disk usage. It’s similar to software like WinDirStat for Windows and Disk Inventory X for Mac OSX.

Install it on your distro using your package manager and then run the command from any folder.

ncdu

The results will look something like this:


Screenshot from Wikipedia

Fix 404 errors when running apt-get update on Debian Wheezy

4 Replies

If you are getting errors similar to the ones below, keep reading for a fix.

Err http://http.us.debian.org wheezy/main mipsel Packages
  404  Not Found [IP: 64.50.233.100 80]
Err http://http.us.debian.org wheezy/contrib mipsel Packages
  404  Not Found [IP: 64.50.233.100 80]
Err http://http.us.debian.org wheezy/non-free mipsel Packages
  404  Not Found [IP: 64.50.233.100 80]
W: Failed to fetch http://http.us.debian.org/debian/dists/wheezy/main/binary-mipsel/Packages  404  Not Found [IP: 64.50.233.100 80]

W: Failed to fetch http://http.us.debian.org/debian/dists/wheezy/contrib/binary-mipsel/Packages  404  Not Found [IP: 64.50.233.100 80]

W: Failed to fetch http://http.us.debian.org/debian/dists/wheezy/non-free/binary-mipsel/Packages  404  Not Found [IP: 64.50.233.100 80]

E: Some index files failed to download. They have been ignored, or old ones used instead.

For normal servers

Edit /etc/apt/sources.list and replace the current servers in the file with http://archive.debian.org/debian.

Example – before

...
deb http://http.us.debian.org/debian wheezy main contrib non-free

Example – after

...
deb http://archive.debian.org/debian wheezy main contrib non-free

For Ubiquity EdgeOS routers

SSH into the console and write:

config
set system package repository wheezy url http://archive.debian.org/debian
commit ; save
apt-get update

Enable logging of DNS queries in Unbound DNS resolver

Leave a reply

In order to enable logging in the Unbound DNS resolver, you have to add the following lines to your /etc/unbound/unbound.conf configuration file:

server:
    chroot: ""
    logfile: /var/log/unbound.log
    verbosity: 1
    log-queries: yes
    ...

Then, create the file and make sure it’s owned by the unbound process:

touch /var/log/unbound.log
chown unbound:unbound /var/log/unbound.log

Finally, restart Unbound:

/etc/init.d/unbound restart

Now you should be able to see the log:

tail -f /var/log/unbound.log
[1553775590] unbound[32655:0] info: 127.0.0.1 googlemail.l.google.com. A IN
[1553775609] unbound[32655:0] info: 127.0.0.1 acp-ss-ew1.adobe.io. A IN
[1553775695] unbound[32655:0] info: 127.0.0.1 clients4.google.com. A IN
...

The reason you have to add chroot: "" is because by default unbound runs in a chroot and can’t write to /var/log.

This post was tested on OpenWRT.

Redeploy all Convox apps in a rack using CLI and set RedirectHttps=No for selected apps

Leave a reply

Convox recently started redirecting http to https but allows you to keep this behaviour by setting the apps param RedirectHttps=No. This script automates redeploying all applications in your current rack and lets you apply the RedirectHttps parameter to certain apps. (UPGRADE_HTTP_APPS)

Save as upgrade.php and run using php upgrade.php. (Authenticated Convox CLI is required)

<?php

define('RACK', 'org/rack');
define('UPGRADE_HTTP_APPS', ['my-http-app', 'my-second-http-app']);

exec('convox switch ' . RACK, $ret);

$appsCommandResult = [];
exec('convox apps', $appsCommandResult);

$apps = [];

// Get list of apps
foreach($appsCommandResult as $result) {
  $matches = null;
  preg_match('/([\w-_]*).*/', $result, $matches);

  if(isset($matches[1]) && $matches[1] !== 'APP') {
    $apps[$matches[1]] = '';
  }
}

// For each app, find active release
foreach($apps as $app => $release) {
  $releasesCommandResult = [];
  exec('convox releases -a ' . $app, $releasesCommandResult);

  foreach($releasesCommandResult as $result) {
    $matches = null;
    preg_match('/([\w-_]*).*active.*/', $result, $matches);

    if(isset($matches[1])) {
      $apps[$app] = $matches[1];
    }
  }
}

foreach($apps as $app => $release) {
  //Promote each app
  $promoteCommand = "convox releases promote {$release} -a {$app} --wait";
  echo $promoteCommand;
  echo "\n";
  $deployCommandResult = '';
  exec($promoteCommand, $deployCommandResult);
  var_dump($deployCommandResult);

  //If in UPGRADE_HTTP_APPS, set RedirectHttps=No
  if(in_array($app, UPGRADE_HTTP_APPS)) {
    $noHttpCommand = "convox apps params set RedirectHttps=No -a {$app} --wait";
    echo $noHttpCommand;
    echo "\n";
    $noHttpCommandResult = '';
    exec($noHttpCommand, $noHttpCommandResult);
    var_dump($noHttpCommandResult);
  }
}