Automatic dynamic DNS updater using EdgeRouter / EdgeOS and Internet.bs

Leave a reply

EdgeOS already supports Python (as of 1.10.9), so let’s write a short Python script. Replace with your username, password and domain on line 13 in the script.

import urllib2
import datetime
import os

print 'IP Updater - ' + datetime.datetime.now().isoformat()

f = os.popen('/sbin/ifconfig eth0 | grep "inet\ addr" | cut -d: -f2 | cut -d" " -f1')
ip=f.read()

print 'IP detected as: ' + ip

if(ip):
  sendOne = urllib2.urlopen('http://dyndns.topdns.com/update?hostname=my-domain.com&username=user&password=password&myip=' + ip).read()
  print sendOne

Schedule it to run recurrently:

touch /var/log/ibs-update.log
crontab -e

Add the following line:

0 * * * * python /home/ubnt/ibs-update.py >> /var/log/ibs-update.log

Note: /var/log is mounted in-memory on EdgeOS, so it’s not going to introduce wear-and-tear on the flash memory.

vnstat on EdgeRouter – historical bandwidth monitoring and graphical dashboard tutorial

3 Replies

This post will show you how to install vnstat and vnstati on the EdgeRouter for bandwidth monitoring, as well as how to create a graphical dashboardwith historical bandwidth data.

Installation

Add non-free sources to APT:

EdgeOS 1.X

configure
set system package repository wheezy components 'main contrib non-free' 
set system package repository wheezy distribution wheezy 
set system package repository wheezy url http://archive.debian.org/debian
commit ; save
sudo apt-get update

EdgeOS 2.X

configure
set system package repository stretch components 'main contrib non-free' 
set system package repository stretch distribution stretch
set system package repository stretch url http://http.us.debian.org/debian
commit ; save
sudo apt-get update

Install the packages

sudo apt-get install vnstat vnstati

vnstat configuration

Edit the config file in /etc/vnstat.conf to be like the following. This will make sure your bandwidth data will survive a firmware update. (However, you’ll have to reinstall and reconfigure vnstat / vnstati after a firmware update).

# vnStat 1.11 config file
##

# default interface
Interface "eth0"

# location of the database directory
DatabaseDir "/var/lib/vnstat"

# locale (LC_ALL) ("-" = use system locale)
Locale "-"

# on which day should months change
MonthRotate 1

# date output formats for -d, -m, -t and -w
# see 'man date' for control codes
DayFormat    "%x"
MonthFormat  "%b '%y"
TopFormat    "%x"

# characters used for visuals
RXCharacter       "%"
TXCharacter       ":"
RXHourCharacter   "r"
TXHourCharacter   "t"

# how units are prefixed when traffic is shown
# 0 = IEC standard prefixes (KiB/MiB/GiB/TiB)
# 1 = old style binary prefixes (KB/MB/GB/TB)
UnitMode 0

# output style
# 0 = minimal & narrow, 1 = bar column visible
# 2 = same as 1 except rate in summary and weekly
# 3 = rate column visible
OutputStyle 3

# used rate unit (0 = bytes, 1 = bits)
RateUnit 1

# maximum bandwidth (Mbit) for all interfaces, 0 = disable feature
# (unless interface specific limit is given)
MaxBandwidth 1000

# interface specific limits
#  example 8Mbit limit for eth0 (remove # to activate):
#MaxBWeth0 8

# how many seconds should sampling for -tr take by default
Sampletime 5

# default query mode
# 0 = normal, 1 = days, 2 = months, 3 = top10
# 4 = dumpdb, 5 = short, 6 = weeks, 7 = hours
QueryMode 0

# filesystem disk space check (1 = enabled, 0 = disabled)
CheckDiskSpace 1

# database file locking (1 = enabled, 0 = disabled)
UseFileLocking 1

# how much the boot time can variate between updates (seconds)
BootVariation 15

# log days without traffic to daily list (1 = enabled, 0 = disabled)
TrafficlessDays 1


# vnstatd
##

# how often (in seconds) interface data is updated
UpdateInterval 30

# how often (in seconds) interface status changes are checked
PollInterval 5

# how often (in minutes) data is saved to file
SaveInterval 60

# how often (in minutes) data is saved when all interface are offline
OfflineSaveInterval 60

# force data save when interface status changes (1 = enabled, 0 = disabled)
SaveOnStatusChange 0

# enable / disable logging (0 = disabled, 1 = logfile, 2 = syslog)
UseLogging 1

# file used for logging if UseLogging is set to 1
LogFile "/var/log/vnstat.log"

# file used as daemon pid / lock file
PidFile "/var/run/vnstat.pid"


# vnstati
##

# title timestamp format
HeaderFormat "%x %H:%M"

# show hours with rate (1 = enabled, 0 = disabled)
HourlyRate 1

# show rate in summary (1 = enabled, 0 = disabled)
SummaryRate 1

# layout of summary (1 = with monthly, 0 = without monthly)
SummaryLayout 1

# transparent background (1 = enabled, 0 = disabled)
TransparentBg 0

# image colors
CBackground     "FFFFFF"
CEdge           "AEAEAE"
CHeader         "606060"
CHeaderTitle    "FFFFFF"
CHeaderDate     "FFFFFF"
CText           "000000"
CLine           "B0B0B0"
CLineL          "-"
CRx             "92CF00"
CTx             "606060"
CRxD            "-"
CTxD            "-"

Using vnstat

Monthly bandwidth

vnstat -m -i eth0

Daily bandwidth

vnstat -d -i eth0

Live bandwidth usage

vnstat -l -i eth0

Configure vnstati to generate images for bandwidth dashboard

Create the file /var/lib/vnstat/vnstati-update.sh with the content:

#!/bin/bash
vnstati -s -i eth0 -o /var/www/htdocs/media/vnstat-summary.png
vnstati -h -i eth0 -o /var/www/htdocs/media/vnstat-hourly.png
vnstati -m -i eth0 -o /var/www/htdocs/media/vnstat-monthly.png
vnstati -d -i eth0 -o /var/www/htdocs/media/vnstat-daily.png

Make it executable:

chmod +x /var/lib/vnstat/vnstati-update.sh

Schedule the script to run every hour to keep the images up to date:

crontab -e

Add the line:

0 * * * * /var/lib/vnstat/vnstati-update.sh

Set up a dashboard

Create the file /var/www/htdocs/media/dashboard.html

Add the content below. Replace 192.168.10.1 with your router IP.

<!DOCTYPE html>
<html>
<head>
    <meta charset="UTF-8">
    <title>BW Dashboard</title>
    <style>
        body {
            text-align: center;
        }
    </style>
</head>

<body>
<h3>Overview</h3>
<img src="https://192.168.10.1/media/vnstat-summary.png">

<h3>24 hour</h3>
<img src="https://192.168.10.1/media/vnstat-hourly.png">

<h3>Monthly</h3>
<img src="https://192.168.10.1/media/vnstat-monthly.png">

<h4>Monthly</h4>
<img src="https://192.168.10.1/media/vnstat-daily.png">
</body>
</html>

Now you can visit your dashboard at https://192.168.10.1/media/dashboard.html (replace with your router IP).

Cleanup

sudo apt-get clean && sudo apt-get autoclean && sudo apt-get autoremove && rm /var/cache/apt/pkgcache.bin /var/cache/apt/srcpkgcache.bin

More reading

https://community.ubnt.com/t5/EdgeRouter/data-usage-monitoring-using-vnstat-cli-only-HOWTO/td-p/1061213

Visualize disk usage on linux with ncdu

Leave a reply

ncdu is a great tool for visualizing your disk usage. It’s similar to software like WinDirStat for Windows and Disk Inventory X for Mac OSX.

Install it on your distro using your package manager and then run the command from any folder.

ncdu

The results will look something like this:


Screenshot from Wikipedia

Fix 404 errors when running apt-get update on Debian Wheezy

4 Replies

If you are getting errors similar to the ones below, keep reading for a fix.

Err http://http.us.debian.org wheezy/main mipsel Packages
  404  Not Found [IP: 64.50.233.100 80]
Err http://http.us.debian.org wheezy/contrib mipsel Packages
  404  Not Found [IP: 64.50.233.100 80]
Err http://http.us.debian.org wheezy/non-free mipsel Packages
  404  Not Found [IP: 64.50.233.100 80]
W: Failed to fetch http://http.us.debian.org/debian/dists/wheezy/main/binary-mipsel/Packages  404  Not Found [IP: 64.50.233.100 80]

W: Failed to fetch http://http.us.debian.org/debian/dists/wheezy/contrib/binary-mipsel/Packages  404  Not Found [IP: 64.50.233.100 80]

W: Failed to fetch http://http.us.debian.org/debian/dists/wheezy/non-free/binary-mipsel/Packages  404  Not Found [IP: 64.50.233.100 80]

E: Some index files failed to download. They have been ignored, or old ones used instead.

For normal servers

Edit /etc/apt/sources.list and replace the current servers in the file with http://archive.debian.org/debian.

Example – before

...
deb http://http.us.debian.org/debian wheezy main contrib non-free

Example – after

...
deb http://archive.debian.org/debian wheezy main contrib non-free

For Ubiquity EdgeOS routers

SSH into the console and write:

config
set system package repository wheezy url http://archive.debian.org/debian
commit ; save
apt-get update

Enable logging of DNS queries in Unbound DNS resolver

Leave a reply

In order to enable logging in the Unbound DNS resolver, you have to add the following lines to your /etc/unbound/unbound.conf configuration file:

server:
    chroot: ""
    logfile: /var/log/unbound.log
    verbosity: 1
    log-queries: yes
    ...

Then, create the file and make sure it’s owned by the unbound process:

touch /var/log/unbound.log
chown unbound:unbound /var/log/unbound.log

Finally, restart Unbound:

/etc/init.d/unbound restart

Now you should be able to see the log:

tail -f /var/log/unbound.log
[1553775590] unbound[32655:0] info: 127.0.0.1 googlemail.l.google.com. A IN
[1553775609] unbound[32655:0] info: 127.0.0.1 acp-ss-ew1.adobe.io. A IN
[1553775695] unbound[32655:0] info: 127.0.0.1 clients4.google.com. A IN
...

The reason you have to add chroot: "" is because by default unbound runs in a chroot and can’t write to /var/log.

This post was tested on OpenWRT.

Redeploy all Convox apps in a rack using CLI and set RedirectHttps=No for selected apps

Leave a reply

Convox recently started redirecting http to https but allows you to keep this behaviour by setting the apps param RedirectHttps=No. This script automates redeploying all applications in your current rack and lets you apply the RedirectHttps parameter to certain apps. (UPGRADE_HTTP_APPS)

Save as upgrade.php and run using php upgrade.php. (Authenticated Convox CLI is required)

<?php

define('RACK', 'org/rack');
define('UPGRADE_HTTP_APPS', ['my-http-app', 'my-second-http-app']);

exec('convox switch ' . RACK, $ret);

$appsCommandResult = [];
exec('convox apps', $appsCommandResult);

$apps = [];

// Get list of apps
foreach($appsCommandResult as $result) {
  $matches = null;
  preg_match('/([\w-_]*).*/', $result, $matches);

  if(isset($matches[1]) && $matches[1] !== 'APP') {
    $apps[$matches[1]] = '';
  }
}

// For each app, find active release
foreach($apps as $app => $release) {
  $releasesCommandResult = [];
  exec('convox releases -a ' . $app, $releasesCommandResult);

  foreach($releasesCommandResult as $result) {
    $matches = null;
    preg_match('/([\w-_]*).*active.*/', $result, $matches);

    if(isset($matches[1])) {
      $apps[$app] = $matches[1];
    }
  }
}

foreach($apps as $app => $release) {
  //Promote each app
  $promoteCommand = "convox releases promote {$release} -a {$app} --wait";
  echo $promoteCommand;
  echo "\n";
  $deployCommandResult = '';
  exec($promoteCommand, $deployCommandResult);
  var_dump($deployCommandResult);

  //If in UPGRADE_HTTP_APPS, set RedirectHttps=No
  if(in_array($app, UPGRADE_HTTP_APPS)) {
    $noHttpCommand = "convox apps params set RedirectHttps=No -a {$app} --wait";
    echo $noHttpCommand;
    echo "\n";
    $noHttpCommandResult = '';
    exec($noHttpCommand, $noHttpCommandResult);
    var_dump($noHttpCommandResult);
  }
}

Restrict viewing of uploaded attachments to logged-in users using WordPress and Nginx

2 Replies

Even if you lock down your WordPress site so outside visitors can’t see it, all uploaded attachment such as media, images and PDFs are available to anyone as long as they know the direct link to those files, or can find it by other means such as search engines. Sometimes that’s not acceptable and you want to make sure only logged in users can see uploaded media files.

This approach works by proxying all files in /wp-content/uploads through a script (dl-file.php) which should be placed in the root of your WordPress installation. The script checks whether the user is logged in before serving the file. If the user is not logged in a redirect is performed so that the user can log in before viewing the file.

Because this approach uses the same check as WordPress itself, it should be considered a very safe way of protecting attachments. You could also extend the proxy script to check for different access levels (for example if you have a membership site) and much more!

The Nginx config

# Hotlink protection
location ~ ^/wp-content/uploads/(.*) {
    try_files /dl-file.php =403;
    include fastcgi_params;
    fastcgi_pass php7;
}

This config should be placed in a server block and the example is for servers running EasyEngine, but it should work with minor modification on any php-fpm server.

The proxy script

<?php
/*
 * dl-file.php
 *
 * Protect uploaded files with login.
 *
 * Based on http://wordpress.stackexchange.com/questions/37144/protect-wordpress-uploads-if-user-is-not-logged-in
 * 
 * @author hakre <http://hakre.wordpress.com/> / khromov
 * @license GPL-3.0+
 * @registry SPDX
 */

require_once('wp-load.php');

//Check if we are logged in before attempting to serve the file
is_user_logged_in() || auth_redirect();

//FIXME ... simplify this
list($basedir) = array_values(array_intersect_key(wp_upload_dir(), array('basedir' => 1)))+array(NULL);

//Check if file exists
$getFile = isset($_GET[ 'file' ]) ? $_GET[ 'file' ] : $_SERVER['REQUEST_URI'];

//Normalize file path
$getFile = str_replace('..', '', $getFile);
$getFile = str_replace('wp-content/uploads/', '', $getFile);

//This provides a notice in the log
trigger_error('Loading protected file ' . $getFile);

$file =  trailingslashit($basedir) . $getFile;

//If file is missing, 404 it
if (!$basedir || !is_file($file)) {
    status_header(404);
    die('404 &#8212; File not found.');
}

$mime = wp_check_filetype($file);
if( false === $mime[ 'type' ] && function_exists( 'mime_content_type' ) )
    $mime[ 'type' ] = mime_content_type( $file );

if( $mime[ 'type' ] )
    $mimetype = $mime[ 'type' ];
else
    $mimetype = 'image/' . substr( $file, strrpos( $file, '.' ) + 1 );

header( 'Content-Type: ' . $mimetype ); // always send this

// If we made it this far, just serve the file
readfile( $file );

Similar StackOverflow question

Photo by Micah Williams on Unsplash

Fix broken / overlapping Instagram embed for WordPress

Leave a reply

At some point recently (February 2018), Instagram broke their oEmbed implementation which causes JavaScript errors and embeds that overlap each other. Use the snippet below to mitigate this issue and hopefully Instagram will fix it in the future. The snippet makes sure only one

/**
 * Remove Instagram embed.js script on each embed
 */
add_filter('embed_oembed_html', function($html, $url, $attr, $post_id) {
  $regex =    '/<script.*instagram\.com\/embed.js.*\s?script>/U';
  $regex_2 =  '/<script.*platform\.instagram\.com\/.*\/embeds\.js.*script>/U';

  if(preg_match($regex, $html) || preg_match($regex_2, $html)) {
    add_filter('kh_has_instagram_embed', '__return_true');

    $html = preg_replace($regex, '', $html);
    $html = preg_replace($regex_2, '', $html);

    return $html;
  }

  return $html;
}, 100, 4);

/**
 * Enqueue the embed.js script once at the bottom of the page, if at least one Instagram embed is enqueued
 */
add_filter('wp_footer', function() {
  if(apply_filters('kh_has_instagram_embed', false)) :
    ?>
      <script async defer src="//www.instagram.com/embed.js"></script>
    <?php
  endif;
}, 999);

Shorthand caching pattern in PHP

Leave a reply

Here’s a short and simple caching pattern you can use to cache an expensive function call:

if(!$result = Cache::get('expensive-function-cache-key')) {
  $result = expensive_function();
  Cache::set('expensive-function-cache-key', $result);
}

//The result variable will always contain the proper value, whether the call was cached or not.
echo $result;