How to cache the wp_oembed_get() function

The wp_oembed_get() function is not cached by default. Unfortunately, many themes (like Elegant Themes Divi) still use it which leads to your page making many background requests to YouTube and similar video embedding sites. This slows down your pageloads since WordPress has to wait for YouTube to response before showing your page.

If you’re not able to fix your theme, you can add the code below into an mu-plugin or into your themes functions.php file. It will cache embeds that are using wp_oembed_get(). Please note that embeds that fail to resolve (for example: deleted youtube video) will still not be cached. This is due to the way wp_oembed_get() works internally and is not trivial to fix.

<?php
/**
 * Cache wp_oembed_get()
 * Version: 1.0
 */

define('CUSTOM_OEMBED_CACHE_KEY', 'coc_');

function _wp_custom_oembed_cache_key($url, $args) {
  $args_serialized = serialize($args);
  return CUSTOM_OEMBED_CACHE_KEY . md5("{$url}-{$args_serialized}");
}

/**
 * This function caches the result
 */
add_filter('oembed_result', function($data, $url, $args) {
  // Cache result
  set_transient(_wp_custom_oembed_cache_key($url, $args), $data, DAY_IN_SECONDS);
  return $data;
}, 999, 3);

/**
 * This function serves the cached result
 */
add_filter('pre_oembed_result', function($result, $url, $args) {

  // Clean out empty oembed calls caused by Divi
  if(trim($url) === '') {
    return '';
  }

  // Return cached result if available
  if($cached_result = get_transient(_wp_custom_oembed_cache_key($url, $args))) {
    return $cached_result;
  }

  return $result;
}, 2, 3);

Configure how often WordPress sends recovery mode emails about errors

By default, WordPress sends a maximum of one email per day to the site administrator if a fatal error is discovered. (Since WordPress 5.2)

To configure this, use the following snippet:

add_filter('recovery_mode_email_rate_limit', function($rate) {
  return HOUR_IN_SECONDS;
});

Instead of using one of the predefined constants like HOUR_IN_SECONDS, you can use any integer value, like 600, which would correspond to every 10 minutes.

Sending emails to multiple users when WordPress goes into recovery mode

WordPress 5.2 pioneered a new “Recovery mode” feature that can notify site administrators by email if their site experiences a fatal error.

By default, the email is sent to either an email configured under the RECOVERY_MODE_EMAIL constant or the user that created the website (admin_email option).

If you want this email to multiple addresses, you can use the snippet below in your themes functions.php or in a plugin. Configure the extra address on line 6.

add_filter('recovery_mode_email', function($email, $url) {
  $email_array = [];
  $email_array[] = $email['to'];

  // Adds another email
  $email_array[] = 'your-email@gmail.com';

  $email['to'] = $email_array;

  return $email;
}, 10, 2);

Configuring WordPress to work behind an Application Load Balancer (ALB) in AWS

When putting WordPress behind an ALB that has SSL configured it might result in a configuration where the ALB uses SSL but WordPress communicates with the ALB over regular HTTP.

This can cause WordPress to server HTTP (non-ssl) CSS and JavaScript resources and/or fail in other ways.

The solution is to check the X-Forwarded-Proto header that ALB sets and let WordPress know whether it should treat the incoming request as an SSL request or not.

Put this code at the top of wp-config.php to do exactly that:

// Get true SSL status from AWS load balancer
if(isset($_SERVER['HTTP_X_FORWARDED_PROTO']) && $_SERVER['HTTP_X_FORWARDED_PROTO'] === 'https') {
  $_SERVER['HTTPS'] = '1';
}

Cache warming on low-traffic WordPress sites

If you have a WordPress site that relies on time-based static page caching (like W3 Total Cache), you may notice that the low default cache timeouts (1-5 minutes) aren’t really working well for sites that receive a low amount of traffic.

In these cases, cache warming provides a great option to make sure every visitor gets a fast TTFB response.

I recently wrote a PHP script to enable cache warming based on a sitemap. (Which both Yoast WordPress SEO and Google XML Sitemaps provide.)

You can find the script and setup instructions by clicking on this link.

Caching strategy for low-traffic WordPress deployments

Let’s talk about caching strategy! Sites fall into one of two categories:

Low amount of content (~20-30 unique posts or pages)

Use a long cache expiration time (24 hours or more), but make sure that the cache is flushed when any content changes. (This is usually configurable in the cache plugin.)

Schedule crawls generously, as crawling will be fast as long as cache hasn’t expired. Crawling every 15 minutes or so is no problem.

Large amounts of content (> 100 posts or pages)

If you have a lot of infrequently accessed pages, the problem gets harder to manage. Crawling hundreds of pages will take a toll on your server, racking up high CPU usage. The only option here is to cache indefinitely, to only flush parts of the cache that are affected by a change, or cache for even longer periods of time (over 1 week).

For caching, I recommend the free Cache Enabler.

Schedule crawles less frequently, once per 1-4 hours. Keep track of the time it takes to run a crawl. If your web host enforces low PHP timeout the crawler might get killed before crawling all the pages.

Automatic dynamic DNS updater using EdgeRouter / EdgeOS and Internet.bs

EdgeOS already supports Python (as of 1.10.9), so let’s write a short Python script. Replace with your username, password and domain on line 13 in the script.

import urllib2
import datetime
import os

print 'IP Updater - ' + datetime.datetime.now().isoformat()

f = os.popen('/sbin/ifconfig eth0 | grep "inet\ addr" | cut -d: -f2 | cut -d" " -f1')
ip=f.read()

print 'IP detected as: ' + ip

if(ip):
  sendOne = urllib2.urlopen('http://dyndns.topdns.com/update?hostname=my-domain.com&username=user&password=password&myip=' + ip).read()
  print sendOne

Schedule it to run recurrently:

touch /var/log/ibs-update.log
crontab -e

Add the following line:

0 * * * * python /home/ubnt/ibs-update.py >> /var/log/ibs-update.log

Note: /var/log is mounted in-memory on EdgeOS, so it’s not going to introduce wear-and-tear on the flash memory.

vnstat on EdgeRouter – historical bandwidth monitoring and graphical dashboard tutorial

This post will show you how to install vnstat and vnstati on the EdgeRouter for bandwidth monitoring, as well as how to create a graphical dashboardwith historical bandwidth data.

Installation

Add non-free sources to APT:

EdgeOS 1.X

configure
set system package repository wheezy components 'main contrib non-free' 
set system package repository wheezy distribution wheezy 
set system package repository wheezy url http://archive.debian.org/debian
commit ; save
sudo apt-get update

EdgeOS 2.X

configure
set system package repository stretch components 'main contrib non-free' 
set system package repository stretch distribution stretch
set system package repository stretch url http://http.us.debian.org/debian
commit ; save
sudo apt-get update

Install the packages

sudo apt-get install vnstat vnstati

vnstat configuration

Edit the config file in /etc/vnstat.conf to be like the following. This will make sure your bandwidth data will survive a firmware update. (However, you’ll have to reinstall and reconfigure vnstat / vnstati after a firmware update).

# vnStat 1.11 config file
##

# default interface
Interface "eth0"

# location of the database directory
DatabaseDir "/var/lib/vnstat"

# locale (LC_ALL) ("-" = use system locale)
Locale "-"

# on which day should months change
MonthRotate 1

# date output formats for -d, -m, -t and -w
# see 'man date' for control codes
DayFormat    "%x"
MonthFormat  "%b '%y"
TopFormat    "%x"

# characters used for visuals
RXCharacter       "%"
TXCharacter       ":"
RXHourCharacter   "r"
TXHourCharacter   "t"

# how units are prefixed when traffic is shown
# 0 = IEC standard prefixes (KiB/MiB/GiB/TiB)
# 1 = old style binary prefixes (KB/MB/GB/TB)
UnitMode 0

# output style
# 0 = minimal & narrow, 1 = bar column visible
# 2 = same as 1 except rate in summary and weekly
# 3 = rate column visible
OutputStyle 3

# used rate unit (0 = bytes, 1 = bits)
RateUnit 1

# maximum bandwidth (Mbit) for all interfaces, 0 = disable feature
# (unless interface specific limit is given)
MaxBandwidth 1000

# interface specific limits
#  example 8Mbit limit for eth0 (remove # to activate):
#MaxBWeth0 8

# how many seconds should sampling for -tr take by default
Sampletime 5

# default query mode
# 0 = normal, 1 = days, 2 = months, 3 = top10
# 4 = dumpdb, 5 = short, 6 = weeks, 7 = hours
QueryMode 0

# filesystem disk space check (1 = enabled, 0 = disabled)
CheckDiskSpace 1

# database file locking (1 = enabled, 0 = disabled)
UseFileLocking 1

# how much the boot time can variate between updates (seconds)
BootVariation 15

# log days without traffic to daily list (1 = enabled, 0 = disabled)
TrafficlessDays 1


# vnstatd
##

# how often (in seconds) interface data is updated
UpdateInterval 30

# how often (in seconds) interface status changes are checked
PollInterval 5

# how often (in minutes) data is saved to file
SaveInterval 60

# how often (in minutes) data is saved when all interface are offline
OfflineSaveInterval 60

# force data save when interface status changes (1 = enabled, 0 = disabled)
SaveOnStatusChange 0

# enable / disable logging (0 = disabled, 1 = logfile, 2 = syslog)
UseLogging 1

# file used for logging if UseLogging is set to 1
LogFile "/var/log/vnstat.log"

# file used as daemon pid / lock file
PidFile "/var/run/vnstat.pid"


# vnstati
##

# title timestamp format
HeaderFormat "%x %H:%M"

# show hours with rate (1 = enabled, 0 = disabled)
HourlyRate 1

# show rate in summary (1 = enabled, 0 = disabled)
SummaryRate 1

# layout of summary (1 = with monthly, 0 = without monthly)
SummaryLayout 1

# transparent background (1 = enabled, 0 = disabled)
TransparentBg 0

# image colors
CBackground     "FFFFFF"
CEdge           "AEAEAE"
CHeader         "606060"
CHeaderTitle    "FFFFFF"
CHeaderDate     "FFFFFF"
CText           "000000"
CLine           "B0B0B0"
CLineL          "-"
CRx             "92CF00"
CTx             "606060"
CRxD            "-"
CTxD            "-"

Using vnstat

Monthly bandwidth

vnstat -m -i eth0

Daily bandwidth

vnstat -d -i eth0

Live bandwidth usage

vnstat -l -i eth0

Configure vnstati to generate images for bandwidth dashboard

Create the file /var/lib/vnstat/vnstati-update.sh with the content:

#!/bin/bash
vnstati -s -i eth0 -o /var/www/htdocs/media/vnstat-summary.png
vnstati -h -i eth0 -o /var/www/htdocs/media/vnstat-hourly.png
vnstati -m -i eth0 -o /var/www/htdocs/media/vnstat-monthly.png
vnstati -d -i eth0 -o /var/www/htdocs/media/vnstat-daily.png

Make it executable:

chmod +x /var/lib/vnstat/vnstati-update.sh

Schedule the script to run every hour to keep the images up to date:

crontab -e

Add the line:

0 * * * * /var/lib/vnstat/vnstati-update.sh

Set up a dashboard

Create the file /var/www/htdocs/media/dashboard.html

Add the content below. Replace 192.168.10.1 with your router IP.

<!DOCTYPE html>
<html>
<head>
    <meta charset="UTF-8">
    <title>BW Dashboard</title>
    <style>
        body {
            text-align: center;
        }
    </style>
</head>

<body>
<h3>Overview</h3>
<img src="https://192.168.10.1/media/vnstat-summary.png">

<h3>24 hour</h3>
<img src="https://192.168.10.1/media/vnstat-hourly.png">

<h3>Monthly</h3>
<img src="https://192.168.10.1/media/vnstat-monthly.png">

<h4>Monthly</h4>
<img src="https://192.168.10.1/media/vnstat-daily.png">
</body>
</html>

Now you can visit your dashboard at https://192.168.10.1/media/dashboard.html (replace with your router IP).

Cleanup

sudo apt-get clean && sudo apt-get autoclean && sudo apt-get autoremove && rm /var/cache/apt/pkgcache.bin /var/cache/apt/srcpkgcache.bin

More reading

https://community.ubnt.com/t5/EdgeRouter/data-usage-monitoring-using-vnstat-cli-only-HOWTO/td-p/1061213