NZBGet MySQL Extension Script

I wrote a simple plugin for NZBGet that inserts download history into a MySQL database. Having my download list in a database makes operations on the data easier than groking text logs.

I wanted to display my most recent downloads on a Grafana Dashboard:
Table of downloads

To use simply copy the Mysql-Log.py script into your NZBGet/scripts directory. In NZBGet, select settings and set the hostname of your MySQL instance.

You can download the plugin on github

Read More

Comments

Using curl with the Pihole API

I couldn’t find any quick references about accessing the Pihole API so I created this page.

Pihole is a great app for blocking internet advertising that was originally designed to be run on a raspberry pi. It blocks known advertisers’ domains at the DNS level by effectively null routing requests destined to serve ads. It can be run on VMs, Raspberry Pis and bare metal servers.

Here are the steps to access the pihole’s rest api. I’m using curl in this example, but you can integrate it with OpenHAB or any other system that can talk REST.

Step 1: Obtaining the web password

Most of the useful API endpoints the pihole provides wisely require authentication. After searching around the net I found that I could pass &token=A_VALID_SESSION_TOKEN to authenticate to the pihole for a session. Unfortunately, this is a temporary auth token and wasn’t suitable for my needs. After more digging, I found the gem I needed: &auth=WEBPASSWORD. Sounds great, but where do I obtain this password? You simply log onto your pihole instance or server and run:

1
2
sudo cat /etc/pihole/setupVars.conf | grep PASSWORD
17d1ce58eb420998d69dfd2850e7c639706715335d78447f78723489df78234

Read More

Comments

NZBGet ElasticSearch Plugin

I wrote a simple plugin for NZBGet that inserts download history into an ElasticSearch cluster (or node). It uses API calls rather than parsing filesystem logs. I wanted a quick way to just insert the data so I created this script.

Simply copy the ESLog.py script into your NZBGet/scripts directory. In NZBGet, select settings and set the hostname of your ElasticSearch instance.

You can download the plugin on github

Read More

Comments

Speeding up Ansible Playbook runs

Ansible is a great tool for configuration management but because of the way it’s designed a common complaint is that it’s not as fast as other tools like Salt, Chef or Puppet. This is because Ansible doesn’t have an agent that listens (although it can) on a host and uses a different type of deployment methodology that is based on SSH. This post isn’t about the pros and cons of each tool, but rather about ways to improve upon Ansible’s default configuration values. By default Ansible ships with very conservative default values. This is smart in my opinion because it offers greater compatibility out-of-the-box. Here I highlight some safe adjustments that can be made to the default configuration for improved performance (speed!)

Real World Playbook Test

For this test I’m using a real-world playbook that I use in my homelab when provisioning a new CentOS VM. It configures some basic things (hostname, ssh keys, etc), installs common packages/utilities and tunes some OS configurations.

Read More

Comments

Pfsense WAN with Cellular LTE/4G

Using pfsense for WAN Redundancy with Cellular LTE/4G

I work from home a lot and I need reliable internet connectivity to workplace. As a result, I decided to implement a failover WAN connection into my firewall. I use pfSense as my firewall which combines power and flexibility with easy of configuration.

The Hardware: NetGear LB2120 LTE Modem

While researching options, I decided on the Netgear LB2120 4G LTE modem since it has Gigabit port(s) and can be used with external antennae.

[ PICTURE OF MODEM ]

Read More

Comments

Dynamic DNS via Digital Ocean API

This is a quick tutorial about setting up a simple shell script that runs periodically to update a DNS record on the internet with the IP address of your network. If you’ve ever wanted to have myhome.mydomain.com always updated with your home IP address, this is one way you can do it. There are paid and free services that offer this functionality, but I choose to use this method because it gives me ultimate flexibility over my domain.
This tutorial assumes you have your own Top Level Domain (TLD) and that you’ve configured it to use Digital Ocean as the authoritative DNS for it. Here, I’ll use “example.com” as the top level domain.

Generate a Personal Access Token

Log in to Digital Ocean and click on API from the top Menu. Click the Generate New Token button and give it a name. The name is not parsed, it’s just for your own reference. I like to give it something descriptive to my application. For this example, I’ll use MyDNS. Be sure to check the box next to Write so we can update the records using this API Key.

Once you have the API Key, save it somewhere for reference. It’s one of the 3 variables we’ll need to update in the update_do_dns.sh script.

The API Key will look something like this:
fe3aeda96b7wer8wer1e6bb5erae528sdf3a6120dfrf7e492bwer6343fsdf

Read More

Comments

Quieting a Dell R710

I have a Dell R710 rev. II that I use in my home office lab (homelab) running ESXi 6.5. The R710 sits in my office where my girl and I spend some days when we work from home. Normally the hum of the R710 fans isn’t terribly bothersome - the 5 fans it houses run at around 3,800 RPM each. The noise is definitely noticeable so I did a little bit of digging into ways I could quiet it down. After looking into replacing the fans with quieter ones I found that I could override the system control of the fans and silence them that way. While I have to monitor the onboard temperatures more closely when disabled, I’ve found little downside to doing so when I’m in there. Here’s how to do it:

The commands used below assume default username / password of root / calvin. Hopefully you’ve changed the default password so substitute yours where applicable.

Read More

Comments

A simple Ansible playbook for updating multiple Pihole DNS

I wrote a very simple little playbook for updating my local DNS records for my piholes. For me it’s easier than manually ssh’ing onto each node and editing a file and restarting the service. Here’s the playbook:

update_dns.yml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
#!/usr/bin/env ansible-playbook
---
- hosts: ns-01, ns-02
gather_facts: yes
sudo: yes
tasks:
- name: TASK | Copy dnsmasq config for cbnet
template: src=templates/02-localnet.conf.j2 dest=/etc/dnsmasq.d/02-localnet.conf force=yes
- name: TASK | Copy updated dns file
template: src=templates/localnet.list.j2 dest=/etc/pihole/localnet.list force=yes
- name: TASK | Restart dnsmasq
service:
name: dnsmasq
state: restarted

This playbook adds a DNSmasq config file for my local network and copies a template file (dnsmasq include file for my local network) and restarts DNSmasq. Here is the template (sample):

Read More

Comments

A Dashboard for Pihole Stats

Pihole + Grafana + InfluxDB Dashboard

Grafana Dashboard
I wanted to add the metrics from my ad-blocker, the great Pihole to my executive dashboard. To create the dashboard I used Grafana to display the graphs and InfluxDB a the time-series backend database. I use a simple python script to get the metrics from pihole and record them in influxdb.
Grafana makes it easy to render them into a user friendly dashboard.

Installing Grafana and Influxdb is beyond the scope of this blog post but here is the scipt that I use to get the data from pihole and insert it into Influx.

After you’re getting data in your influx db you’ll have to create a grafana dashboard.

Read More

Comments

Here's how they voted

Credit: truefalseequivalence @ reddit

Internet Freedom

Senate Vote for Net Neutrality

Read More

Comments

pfSense graphs in Grafana

Using Grafana with pfSense

I put this guide together using information from various other blogs. This is current as of 2018 and using pfSense 2.4.2. For this tutorial, you’ll need your IP or hostname of your influxdb data source and your username and password.

The data flow is as follows:
pfSense -> Telegraf (gather metrics) -> InfluxDB (store metrics) -> Grafana (render graphs)

Read More

Comments

Using Ansible to build a high availablity Nzbget usenet downloader

I’m limited to about 80MB/s on downloads on my VPC at Digital Ocean, but I run Nzbget for downloading large files from usenet. It doesn’t take long to download at all, but out of curiosity I wanted to see if I could parallelize this and download multiple files at the same. I use Sonarr for searching usenet for freely distributable training videos which then sends them to NZBget for downloading. Since Sonarr can send multiple files to nzbget which get queued up, I figured I can reduce the queue by downloading them at the same time.

Using Ansible and Terraform (devops automation tools), I can spin up VPC on demand, provision them, configure them as nzbget download nodes and then destroy the instances when complete.

Read More

Comments

Record and playback terminal sessions with Showterm

Showterm

I just found a neat tool that will let you record a bash session for playback / site linking. It’s called Showterm. Adding the playback video is as simple as adding an iframe to your page:

1
<iframe src="http://showterm.io/7b5f8d42ba021511e627e" width="640" height="480"></iframe>

or pasting the url:

1
http://showterm.io/7b5f8d42ba021511e627e

Here’s a sample:

Comments

Building an executive dashboard with Grafana

Grafana + InfluxDB + scripts = Awesome

I have many interests and some of them have metrics that are useful or fun to watch. For example, I have investment in Bitcoin so it’s nice to be able to keep an eye on it periodically.
I decided to create a graphical “at a glance” dashboard for myself. I chose Grafana as the user interface / front end and InfluxDB a the time-series backend database to store the metrics. I use various scripts and applets to populate the data into Influx and Grafana makes it easy to
render them into a user friendly dashboard.

Some of the metrics I monitor are Pihole stats, the price of bitcoin, how many IPs get banned from my webservers and my network throughput.

Here’s my dashboard:

Grafana Dashboard

Read More

Comments

Using Ansible to build a high availablity Sabnzbd usenet downloader

I’m limited to about 40MB/s on downloads on my VPC at Digital Ocean, but I run Sabnzbd for downloading large files from usenet. It doesn’t take long to download at all, but out of curiosity I wanted to see if I could parallelize this and download multiple files at the same. I use Sonarr for searching usenet for freely distributable training videos which then sends them to SABnzbd for downloading. Since Sonarr can send multiple files to sabnzbd which get queued up, I figured I can reduce the queue by downloading them at the same time.

Using Ansible and Terraform (devops automation tools), I can spin up VPC on demand, provision them, configure them as sabnzbd download nodes and then destroy the instances when complete.

The instances all run the same sabnzbd config and the instances use haproxy for round-robin distribution. I will probably change this to Consul, but I just wanted something quick so I used a basic haproxy config.

Read More

Comments