Record and playback terminal sessions with Showterm

Showterm

I just found a neat tool that will let you record a bash session for playback / site linking. It’s called Showterm. Adding the playback video is as simple as adding an iframe to your page:

1
<iframe src="http://showterm.io/7b5f8d42ba021511e627e" width="640" height="480"></iframe>

or pasting the url:

1
http://showterm.io/7b5f8d42ba021511e627e

Here’s a sample:

Comments

Building an executive dashboard with Grafana

Grafana + InfluxDB + scripts = Awesome

I have many interests and some of them have metrics that are useful or fun to watch. For example, I have investment in Bitcoin so it’s nice to be able to keep an eye on it periodically.
I decided to create a graphical “at a glance” dashboard for myself. I chose Grafana as the user interface / front end and InfluxDB a the time-series backend database to store the metrics. I use various scripts and applets to populate the data into Influx and Grafana makes it easy to
render them into a user friendly dashboard.

Some of the metrics I monitor are Pihole stats, the price of bitcoin, how many IPs get banned from my webservers and my network throughput.

Here’s my dashboard:

Grafana Dashboard

Technology used:



Grafana
InfluxDBPiHole
Comments

Using Ansible to build a high availablity Sabnzbd usenet downloader

I’m limited to about 40MB/s on downloads on my VPC at Digital Ocean, but I run Sabnzbd for downloading large files from usenet. It doesn’t take long to download at all, but out of curiosity I wanted to see if I could parallelize this and download multiple files at the same. I use Sonarr for searching usenet for freely distributable training videos which then sends them to SABnzbd for downloading. Since Sonarr can send multiple files to sabnzbd which get queued up, I figured I can reduce the queue by downloading them at the same time.

Using Ansible and Terraform (devops automation tools), I can spin up VPC on demand, provision them, configure them as sabnzbd download nodes and then destroy the instances when complete.

The instances all run the same sabnzbd config and the instances use haproxy for round-robin distribution. I will probably change this to Consul, but I just wanted something quick so I used a basic haproxy config.

Terraform builds 4 sabnzbd, 1 haproxy, and 1 ELK instance. It configures a VIP which I point Sonarr to. Here’s the terraform config that builds a sabnzbd server:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
resource "digitalocean_droplet" "sab1" {
image = "centos-7-x64"
name = "sab1"
tags = ["sab"]
region = "nyc1"
size = "2gb"
private_networking = true
ssh_keys = [
"${var.ssh_fingerprint}"
]
connection {
user = "root"
type = "ssh"
private_key = "${file(var.pvt_key)}"
timeout = "2m"
}
provisioner "remote-exec" {
inline = [
"export PATH=$PATH:/usr/bin",
"sudo yum -y install epel-release"
]
}
}

Here is a terraform play to provision 6 new hosts (1 Elasticsearch, 1 HAproxy and 4 Sabnzbd):

Click here to watch terraform build the instances

Here is the ansible playbook that configuress the sabnzbd instances:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
#!/usr/bin/env ansible-playbook
---
- hosts: all
gather_facts: yes
become: yes
tasks:
- name: TASK | Create usenet group
group:
name: usenet
state: present
- name: TASK | Create usenet user
user:
name: usenet
shell: /bin/bash
groups: usenet
append: yes
password: [ removed ]
update_password: on_create
- name: TASK | Create /apps directory
file:
path: /apps
state: directory
owner: usenet
group: usenet
mode: 0755
- name: TASK | Create /apps/data/.sabnzbd directory
file:
path: /apps/data/.sabnzbd
state: directory
owner: usenet
group: usenet
mode: 0755
- name: TASK | Install EPEL Repo
action: >
yum name=epel-release state=present update_cache=yes
- name: TASK | Install packages
action: >
yum name={{ item }} state=present update_cache=yes
with_items:
- vim
- git
- mlocate
- par2cmdline
- p7zip
- unzip
- tar
- gcc
- python-feedparser
- python-configobj
- python-cheetah
- python-dbus
- python-devel
- screen
- vim
- htop
- iftop
- bind-utils
- tree
- jq
- telnet
- lsof
- tcpdump
- nload
- name: TASK | Install unrar
yum:
name: ftp://rpmfind.net/linux/dag/redhat/el7/en/x86_64/dag/RPMS/unrar-5.0.3-1.el7.rf.x86_64.rpm
state: present
- name: TASK | Install yenc
yum:
name: http://archives.fedoraproject.org/pub/archive/fedora/linux/releases/22/Everything/x86_64/os/Packages/p/python-yenc-0.4.0-4.fc22.x86_64.rpm
state: present
- name: TASK | Clone SAB repo
git:
repo: 'https://github.com/sabnzbd/sabnzbd.git'
dest: /apps/sabnzbd
version: master
clone: yes
update: yes
- name: TASK | Change owner of SAB to usenet
file:
path: /apps
state: directory
owner: usenet
group: usenet
mode: 0755
- name: TASK | Install systemd unit file for SAB
template: src="templates/sabnzbd.service.j2" dest="/etc/systemd/system/sabnzbd.service" mode=0644
- name: TASK | Copy SAB config
template: src="templates/sabnzbd_config.ini.j2" dest="/apps/data/.sabnzbd/sabnzbd_config.ini" mode=0644
- name: TASK | Enable and start SAB service
systemd:
name: sabnzbd
enabled: yes
state: started
- name: TASK | Install filebeat
yum:
name: https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-5.6.2-x86_64.rpm
state: present
update_cache: yes
- name: TASK | Copy filebeat config file
template: src="templates/filebeat.yml.j2" dest="/etc/filebeat/filebeat.yml" mode=0644
- name: TASK | Enable and start filebeat
systemd:
name: filebeat
enabled: yes
state: restarted
- name: TASK | Start updatedb for locate
shell: "updatedb >/dev/null 2>&1 &"

Technology used:

TerraformAnsibleSabnzbdElasticsearchSonarr
Comments

System info bash script

I put together a quick bash shell script to view system info at a glance. I know there are existing tools for this like inxi, but I wanted to put something together I can copypasta. This is specific to RHEL, Centos and Sci Linux but it can be easily adapted for other distros.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
#!/bin/bash
# run as root
#echo "================================ Services: ================================"
#for i in `journalctl -F _SYSTEMD_UNIT | grep -v session | sed -e "s/.service//g"`; do echo "========= $i ========="; ps aux | grep -v grep | grep $i; done
echo "================================ Service restarts today: ====================================="
output=$(journalctl --since today | grep -v slice | egrep "Starting|Stopping" | grep -v systemd)
echo "$output"
echo "================================ User logins today: =========================================="
output=$(journalctl --since today | grep -v slice | grep pam_unix)
echo "$output"
echo "================================ Non core processes: ========================================="
pstree -n | grep -v lvmeta | grep -v systemd | grep -v rhn | grep -v dbus-daemon | grep -v crond | grep -v agetty | grep -v oddjobd | grep -v vmtoolsd | grep -v tuned | grep -v rsyslog | grep -v irqbalance | grep -v ntpd | grep -v sssd | grep -v qmgr | grep -v pickup | grep -v puppet | grep -v mcollectived | grep -v dhclient | grep -v sshd | grep -v nrpe | grep -v collectd | grep -v xinetd | grep -v consul | grep -v rpcbind | grep -v grep
echo "================================ Open ports: ================================================="
sudo netstat -ntpl | grep -v ssh | grep -v nrpe | grep -v master | grep -v "rpc" | grep -v Active | grep -v Proto | grep -v xinetd | grep -v systemd | sed -e "s/:::/0.0.0.0:/g" | tr -s " " | cut -f4,7- -d " "
echo "================================ Open Sockets: ==============================================="
sudo systemctl list-sockets
echo "================================ Uptime: ====================================================="
uptime
echo "================================ Users: ======================================================"
who
echo "================================ Volume Info: ================================================“
lvdisplay | grep "LV Name"
vgdisplay | grep "VG Name"
Comments

Inxi - a utility for viewing system information

Inxi

inxi is a super handy system info utility. These days I typically work with ephemeral instances / microservers, so I just dispose of infrastructure that flakes out. Occassionally I’ll need to see what’s up with a box so I’ve put together some common invocations of inxi below for reference:

Common Invocations

General system info:

1
2
3
4
5
6
7
8
9
10
11
12
13
inxi -c 5 -b
System: Host: mgmt-01.tools.atl.primedia.com Kernel: 3.10.0-229.el7.x86_64 x86_64 bits: 64 Console: tty 4
Distro: Scientific Linux release 7.1 (Nitrogen)
Machine: Device: vmware System: VMware product: VMware Virtual Platform serial: VMware-42 01 09 7b d7 cf ec cd-a6 9a 79 d9 35 f5 0c 95
Mobo: Intel model: 440BX Desktop Reference Platform serial: N/A BIOS: Phoenix v: 6.00 date: 09/17/2015
CPU(s): 2 Single core Intel Xeon E5-2698 v3s (-HT-SMP-) speed: 2294 MHz (max)
Graphics: Card: VMware SVGA II Adapter
Display Server: N/A driver: vmwgfx tty size: 180x50 Advanced Data: N/A for root out of X
Network: Card: VMware VMXNET3 Ethernet Controller driver: vmxnet3
Drives: HDD Total Size: 96.6GB (20.1% used)
Info: Processes: 132 Uptime: 439 days Memory: 398.0/3792.0MB Init: systemd runlevel: 3
Client: Shell (bash) inxi: 2.3.39

Full system info:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
inxi -Fi
System: Host: mgmt-01.tools.atl.primedia.com Kernel: 3.10.0-229.el7.x86_64 x86_64 bits: 64 Console: tty 4
Distro: Scientific Linux release 7.1 (Nitrogen)
Machine: Device: vmware System: VMware product: VMware Virtual Platform serial: VMware-42 01 09 7b d7 cf ec cd-a6 9a 79 d9 35 f5 0c 95
Mobo: Intel model: 440BX Desktop Reference Platform serial: N/A BIOS: Phoenix v: 6.00 date: 09/17/2015
CPU(s): 2 Single core Intel Xeon E5-2698 v3s (-HT-SMP-) cache: 81920 KB
clock speeds: max: 2294 MHz 1: 2294 MHz 2: 2294 MHz
Graphics: Card: VMware SVGA II Adapter
Display Server: N/A driver: vmwgfx tty size: 180x50 Advanced Data: N/A for root out of X
Network: Card: VMware VMXNET3 Ethernet Controller driver: vmxnet3
IF: eth0 state: up speed: 10000 Mbps duplex: full mac: 00:50:56:81:59:7e
WAN IP: 216.52.38.4
IF: eth0 ip-v4: 172.24.107.75 ip-v6-link: fe80::250:56ff:fe81:597e
Drives: HDD Total Size: 96.6GB (20.1% used)
ID-1: /dev/sda model: Virtual_disk size: 21.5GB
ID-2: /dev/sdb model: Virtual_disk size: 75.2GB
Partition: ID-1: / size: 88G used: 17G (19%) fs: xfs dev: /dev/dm-1
ID-2: /boot size: 497M used: 89M (18%) fs: xfs dev: /dev/sda1
ID-3: swap-1 size: 2.15GB used: 0.10GB (5%) fs: swap dev: /dev/dm-0
RAID: No RAID devices: /proc/mdstat, md_mod kernel module present
Sensors: None detected - is lm-sensors installed and configured?
Info: Processes: 132 Uptime: 439 days Memory: 397.8/3792.0MB Init: systemd runlevel: 3
Client: Shell (bash) inxi: 2.3.39

View repos:

1
inxi -r

Repos: Active yum sources in file: /etc/yum.repos.d/spacewalk-client.repo
spacewalk-client ~ http://yum.spacewalkproject.org/2.2-client/RHEL/7/$basearch/
```

The inxi code base is (here)[https://github.com/smxi/inxi].

Comments

A systemD unit file for Hexo

Hexo is a simple, lightweight node blog framework. It didn’t include a SystemD Unit file, so I created one:

Hexo SystemD Unit File

/lib/system/systemd/hexo.service

1
2
3
4
5
6
7
8
9
10
11
12
13
[Service]
WorkingDirectory=/home/[yourdirectory]/blog
ExecStart=/bin/hexo server -p80
Restart=always
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=hexo
User=root
Group=root
Environment=NODE_ENV=production
[Install]
WantedBy=multi-user.target

Obviously, you’ll want to replace [yourdirectory] with the location of your Hexo blog.

Start Hexo

1
$ sudo systemctl start hexo

View status of Hexo

1
2
3
4
5
6
7
8
9
10
11
12
$ sudo systemctl status hexo
● hexo.service
Loaded: loaded (/usr/lib/systemd/system/hexo.service; disabled; vendor preset: disabled)
Active: active (running) since Sat 2017-10-07 02:27:36 UTC; 7min ago
Main PID: 8997 (hexo)
CGroup: /system.slice/hexo.service
└─8997 hexo
Oct 07 02:27:36 centos-2gb-nyc1-01 systemd[1]: Started hexo.service.
Oct 07 02:27:36 centos-2gb-nyc1-01 systemd[1]: Starting hexo.service...
Oct 07 02:27:38 centos-2gb-nyc1-01 hexo[8997]: INFO Start processing
Oct 07 02:27:38 centos-2gb-nyc1-01 hexo[8997]: INFO Hexo is running at http://localhost:80/. Press Ctrl+C to stop.

I recommend Digital Ocean for great cloud hosting.

Comments

Comma.ai Panda

Comma is shpping the Panda interface again. I received mine last week. It’s a wifi enabled OBDII interface. The Panda is more than just a reader though. It decodes and enables writing to the Media canbus. It’s going to make for some fun projects. Using Cabana I should be able to create a Database file of the Kia Optima’s CANbus protocol.

Panda

Panda
Panda in the 2016 Optima
Panda closeup

Comments

Building a homebrew generator

A few years ago I built my own generator. I cobbled it together with some parts I had laying around the house. I found a free lawnmower on craigslist and I used the engine from it as a power source for the generator. I coupled the engine to an automobile alternator and a power inverter. The result is a cheap AC power generator. It only produces a few hundred watts of power, but it’s enough to power lights or emergency equipment in the event of power failure. I did this project for fun, not as a replacement for a real backup power generator. Here are some pictures:
The generator
The generator
Wood coupler panel
Belt and pulleys

Comments

Building an Ethanol Still

A few years ago, when gas prices were exorbitant, I started to build my own Ethanol Still. In light of the global energy crisis, I figured that being able to produce my own fuel was a useful endeavor. Creating ethanol fuel requires distilling corn (or other vegetables/fruits) and extracting its potential liquid energy. I did some research on the web and discovered the Charles 803.
I bought plans and began building the still. I’ve put this project on hold for now but perhaps one day I’ll complete it. Until then, here are some pictures of the project:

Laying out the copper pipe
Reducer coupling between reflux and boiler
Cutting the main copper pipe
Drilling out the boiler
Drilling out the boiler
Boiler section
Boiler section
Boiler section
The ethanol still
The ethanol still
The reflux column

Comments

How high gas prices paid for my new (used) car

The stage is set:

I drive my VW Jetta exactly 50 miles a day to and from work. When gas prices started going up in the Summer of 2006, I decided to figure out exactly how much fuel was costing me each workday. So, one morning on the way to work, I stopped at the gas station nearest my home. I filled my tank and reset my trip counter to zero. I recorded the price per gallon ($3 something a gallon!). Then, I drove directly to work, worked, and drove back to the same gas station. I pulled up to the same pump I filled up at that morning. I took note of how many miles I had driven, and sure enough it read almost exactly 50 miles. Next, I refilled my gas tank. It cost me $7. I had burned $7 dollars worth of fuel just going to work and back! That came as quite a shock. I knew gas prices were high, but I never thought that I was spending that much each day. $3 a Gallon?!? Somethings gotta give! After doing some homework on the web (fueleconomy.gov), et al., I decided to buy a much more fuel efficient car (and a soon to be FREE CAR).

Enter the $1,000 commuter car: The 1995 Ford Aspire

1995 Ford Aspire

40 + MPG of pure road worthy goodness. This was to be my new commuter car. It’s safer than a motorcycle (it has airbags). It was designed to be affordable, and it was: I only paid $1,000 cash for it. As a bonus, the owner was a mechanic whom had just just rebuilt the miniscule 1.3L engine and had it purring like a kitten.

Perfect!!! A cheap car, with a good engine and high MPG rating. I decided to tidy the car up a bit, so I ripped out all the seats except for the drivers’ (afterall, I wasn’t going to be toting anyone around it; hell I doubt anyone would even want to be seen it). It even has ICY cold A/C. I installed a car alarm (without the siren) for the remote doorlocks. I ripped out the carpet and put subwoofer carpet all around. It has a neat contour to it. I don’t need pretty, I need functional and ultra fuel efficient.

But how did you get it for free?

After much calculation, I bought the Aspire and decided to see how much money it was costing me to drive to work each day. I performed the same routine as with my Volkswagen. I even used the same pump at the same station.
I drove the 50 miles as normal and stopped back at the gas station on my way home. I took a deep breath and topped off the gas tank.

It cost me two dollars. Yes, only $2 to go to work and back!?!

I was paying $7 A DAY the week before. I was stunned!
That’s a savings of $5 each day that I work. I work 20 days a month, so that’s $100/month I saved on gas! And all I have to do is get used to driving this little car. Wait a second, considering that this car was only $1,000 dollars, if I save $100 per month, it will pay for itself in 10 months.
Oh, it paid for itself alright – with dividends!

1: It uses the cheapest gas and less of it.

It’s a great feeling getting 300+ miles for about $20 dollars (in 2006)!

2: Patriotism!

It’s a great feeling pulling into a gas station and driving right past the Hummers that are stopped at the pump. I love it when I catch them glancing over at my little microcar and then see them looking back the $$$$$$$ on their pump.

3: My other cars maintain their value.

My Volkswagen Jetta has about 60,000 miles on it. Putting a few hundred miles a week on it was certainly going to affect it’s value drastically in a short amount of time. Now, she just sits in the garage looking pretty. I also don’t need to pay for the car wash each month (there’s another $20 I save). Since I drive it much less, I put less wear-and-tear on the car. Less maintenance = less money out of my pocket = more money in the bank. Probably a few hundred dollars a year.

4: Easy to maintain.

Parts for this car are cheap. New tires on our BMW cost over $1,000. New tires on my Jetta cost around $500. New tires for the Aspire cost about $20 each.

5: Cheaper car insurance.

I now get a multi-car discount. My car insurance actually came DOWN when I added this car to my policy. I didn’t expect that at all. Nice!

6: The benefit of having multiple cars.

I remember when I only had one car and it needed repair. I was stuck. I couldn’t go anywhere – it was my only car and it was in the shop. That sucked. Owning more than 1 car is a great stress reliever. Got a nail in the tire of the Jetta? Throw the tire into the trunk of the Aspire, drop it off and pick it up later after it’s been fixed. No more sitting in tire shop lobby hell watching Martha Stewart and having to choose between Southern Living or Fishing Today magazines.

7: It’s a Micro Utility Vehicle.

After ripping out the rear seats and passenger seat, I was left with quite a bit of room in the car. So much in fact, that it’s like having a Micro SUV. I can load up lumber from Home Depot (remember theres no passenger seat so I can just push 2×4’s in all the way to the engine firewall. I don’t have to worry about cutting/scratching leather seats, scuffing the dashbaord, etc.

8: It’s retro, tiny and fun to drive!

It’s incredibly fun to drive a small car and it’s retro old. I get funny looks from some people as if it’s a concept car.

9: It makes me appreciate my other cars.

On the weekends when I take out my other cars, I can really appreciate the creature comforts: Leather seats, woodgrain dashes, sunroofs, horsepower etc. I used to take them for granted, but now I really appreciate just how nice my other cars are (relatively).

10: Goodbye Speeding Tickets.

Okay, so this one’s a bit of a stretch. Last year, I got a $300 speeding ticket in my Jetta. I’ve never had a speeding ticket in my Aspire, nor do I think I ever will. I know when I’m doing 55 in my Aspire. My foot is to the floor and I’m getting passed by UHaul trucks. In my Jetta I’ll get up to 70 without even knowing it. Since I’m in my Aspire 90% of the time, that’s only 10% of the time that I have consciously avoid speeding.

It paid for itself in less than a year by:

  • Using less fuel – a LOT LESS.
    $100/month less.
  • Preserves my other cars’ values and saves them on maintenance. At least a few $100/year.
  • Lowers my car insurance premiums (multi-car discount): $80/year
  • Is much more affordable to fix, repair, maintain – No expensive dealership repairs here baby! I can replace parts on this car using cheap junkyard parts. $15 for an alternator!?? I’ll take two!

Summary:


$1,400 total investment in the car
$100/month gas savings for 36 months = $3,600

Net gain: $2,200 versus driving my Jetta to work for the same time period.

Efficiency is the new cool. Feel free to ask questions or opine below.

Comments