My Home Network – Part 2

Hello again, this is the second in a series of my home network posts. In this edition, we’re going to be flashing another router with OpenWrt. You can read part one here.

I decided it was time to leave all the old hardware behind and move to brand new stuff. I was having problems with the ZYXEL VMG8924-B10A as the main router, it kept cutting out and was causing short minute outages that ultimately I think was due to the system running out of ram and botnets trying to break into the thing. So I decided it was time to jump ship and move to something a little bit more enterprise. At the same time, I decided now would also be a good time to leave the Netgear WNR3500L V2 to one side. Despite it serving us good for many years, its routing features won’t be necessary for the upgrade as I want to have a good crack at managed switching.

The new network

The new network is composed with the following,

  • A Ubiquiti EdgeRouter X
  • A Netgear ProSafe GS108T
  • A new to me MR33

I wanted to do away with having two routers on the network, it wasn’t neccisary and caused some of the upstairs equipment to be inaccessible from downstairs, the new configuration would mean that all management would be done through the EdgeRouter X which would be much cleaner and hopefully faster.

The ProSafe GS108T was a garage find, I wasn’t using it for anything and I hadn’t really implemented proper VLAN tagging before so I thought now was a better time than ever to get into it.

Network Topology
My new home network diagram, ignore the fact its all Ubiquiti gear, its the iconography I had loaded at the time. And I couldn’t quickly find icons for Netgear.

I also received a new to me MR33 complete with Meraki OS. Unfortunately their licencing of their hardware is not something that aligns with my principals on hardware so before we accept any software agreements I thought it best to do as always and flash OpenWRT to the thing and say goodbye to Meraki. Flashing OpenWRT onto it was no easy feat but I made sure the thing never saw access to the internet and eventually I was in.

[aidan@aidan-ld mr33]$ ssh root@192.168.1.1


BusyBox v1.28.3 () built-in shell (ash)

  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 -----------------------------------------------------
 OpenWrt 18.06.1, r7258-5eb055306f
 -----------------------------------------------------
=== WARNING! =====================================
There is no root password defined on this device!
Use the "passwd" command to set up a new password
in order to prevent unauthorized SSH logins.
--------------------------------------------------
root@OpenWrt:~# 
root@OpenWrt:~# exit
Connection to 192.168.1.1 closed.
[aidan@aidan-ld mr33]$ 

A couple of configuration changes later and we’ve got a dumb ap ready for deployment.

I really wanted to play with subnets this time, before we had two but this time I thought go hard or go home, so in total there are five subnets on my home network now, that’s enough IP space on the 192 network for 1200 devices. I’ve refrained from using the 10.0.0.0/16 network class simply because I use some VPNs with the same IP space so I thought best to just keep it simple. I have to say the Ubiquiti EdgeRouter X was a little bit new to me but I like the interface, PoE was a particularly nice touch.

For the Netgear switch the plan is to take a trunked VLAN ethernet cable and have the VLAN20 on the first 7 ports and leave the remaining 8th port for the MR18. Setting it up was quite easy using the online wizard however the visualisation thingy was clearly older than useful because it seemed to use some java applet that chrome did not like.

A Little Conclusion

I like what I’ve got set up now and will probably leave it for some time. The ZYXEL VMG8924-B10A is destined for the bin but the Netgear WNR3500L V2 I’ll keep for now, it can do VLANs and probably would have worked fine, I just wanted to use a managed switch.

The new network is great. I have a printer which its driver doesn’t seem to like cross-lan communication because every time I print something, it prints fine but the software client continually reports that the communication with the printer failed. I put that down to poor software testing I guess. Wouldn’t be the first time someone had printer woes.

EdgeRouter X Dashboard with VLANS
EdgeRouter Dashboard
SubnetPurpose
192.168.1.0/24Downstairs Wired (eth1)
192.168.2.0/24Upstairs Wired (eth2.20)
192.168.3.0/24Downstairs Wireless AP Clients (eth3)
192.168.4.0/24Upstairs Wireless AP Clients (eth2.40)
192.168.5.0/244th port on router, only a printer attached.(eth4)
Subnets (eth0 is WAN)

Thats all folks. Aidan.

Scraping Canvas (LMS)

Because my time at university is ending I thought it best to archive the canvas pages available to me for later reference should I not be able to access canvas later if they change platforms or disable my account. I should probably add this is for archival purposes and I will not be able to share the data I was able to collect. Thankfully I was able to get the whole thing going in a few minutes and downloading took a lot longer.

The first snippet I got from here, didn’t complete the first time, it seemed some image was causing issues so I moved to another gist, at this rate we could be done in half an hour ūüėä.

Unfortunately it also borked out on a similar place,

FileNotFoundError

I think it is because there’s something missing or I don’t have access to it. But the real problem is that its downloading content for a course I didn’t care about because I was enrolled in it but it’s full of junk I’m not interested in, so we can remove it by using the second scrapers code and specifying the course id’s which I had to manually go through, there was about 15 of them but it didn’t take too long. Which gave me the full command.

F:\Downloads\canvas>python canvas.py https://canvas.hull.ac.uk/ 4738~DUI9Nha9weSuemu1M2qsmhljoBcQtR0zghXTs3QA7ECHDHQkpsgBQ9RllbaEwySf output 52497,56148,56149,52493,54499,54452,54456,53441,52496,22257,22274,22276,22277,22278,22279,22280,50664,50656,22275,50652

The access token you can see above should be expired by now. You can do it yourself by downloading the same file and installing python3, pathvalidate and pycanvas. You need to generate a security token from /profile/settings and you can get the course id by clicking on the course like this /courses/56149. When you generate a new token you should receive an email about it.

Canvas online with our starred modules displated.

I decided to make a small adaptation to catch the FileNotFoundError and went off to the races. It took over an hour so I decided it was best to leave it running overnight, when I returned in the morning I had 116 errors (failed downloads) and the rest is the course content!

Our Canvas Modules saved to Windows File Explorer.

Unfortunately I don’t seem to have the submissions for each of these courses so I needed to manually download them aswell and then our archive was completed.

Thanks for reading.

How Much Does My Car Cost Per Mile?

I thought its time I put my GNUCash Data to good use and worked out how much I spent on fuel. I loaded up a simple Cash Flow bar chart in GNUCash and selected my expenses column for Car>Petrol and voila.

Monthly petrol costs since 2017, highest at £167, lowest £10. Apparently I did not buy any petrol in July РAlso checked this. Total expenses from 01/01/2017-01/01/2020 £2,659.79.

If you are astute you may have also noticed there are additional ‘fixed’ (varies annually/monthly) costs to running a car such as tax, insurance, maintenance and depreciation but I have chosen to eliminate these costs because I would like to explore the benefits of buying another car. We can use this graph based on the last 3 years to estimate this car, my 1998-1999 Vauxhall Corsa costs ¬£73.88 monthly (excluding this month from the data)

My Car

So How far does £74/month get you?

Good Question, I’ve always used my car when I’ve needed it. I have little reason not to use it. I’ve used it to commute to University, School and Work and on days out. It’s my main mode of transport is what I’m trying to say.

Unfortunately, I don’t keep the history of my car’s odometer, however, I can use the mot history of my car to estimate the ¬£/per mile. Using the mot history, which has the date my car was taken in for MOT for two different dates recorded in the MOT history, one in 2017 and one in 2019, we can determine in that time I did 19,205 miles so roughly 6.66k miles a year.

We can then use this 19,205 miles, which have around 1 year, 11 months between them to get (74*23) ¬£1,702 expenditure during that time, which compared with the actual data gives ¬£2,051.23 (over ¬£300 diff, 21%) gives us a fairly low confidence, however we can use this to estimate my cars cost per mile on fuel alone is around ¬£0.106 per mile. 10p per mile (2051.23/19205) or 740 miles per month, give or take 20%. That’s 24 miles per day!

Inside a Western Digital Blue Hard Drive

I thought I’d share pictures I took when I took apart a 250GB dead hard drive.

Rest in pieces my WD2500AAKX

I got this hard drive as part of a Dell Optiplex 780 and used it as a server for my internal network. It worked great until it wouldn’t boot. I checked on it and sure enough, it was stuck in ubuntu server boot recovery. I tried to recover it but I think I did more damage than good. I decided to move to a Windows computer and tried to recover the data with Recuva which didn’t do anything because it couldn’t pick up the disk, so then I moved to TestDisk which was able to see the drive and partitions but never got past profiling the disk. So then I decided to take it apart.

The hard drive in the Dell Optiplex 780 covered in dust
The hard drive in the Dell Optiplex 780

First I unscrewed all the screws, there is another screw holding the read/write head under the label.

Hard drive and hard drive mainboard
Front of hard drive and hard drive mainboard

After that I took it apart a little more, it has one platter internally and one big old magnet which I kept.

WD2500AAKX internals with platter and read write head exposed
Well, I’ve let the magic smoke out now.

Interestingly there seems to be a metal piece on the bottom and side of the hard drive which I think is for easy destruction. CrystalDiskInfo said it had 29202 hours on it and 2875 power ons, nearly exactly the same as my ST2000DM001-1CH164 T2B hard drive. The smart data also had warnings for its Reallocated Sectors Count.

It’s in the bin now. Thanks for reading.

Tracking My Position all the Time

I have a long standing interest in monitoring my daily doings, I previously thought about putting a GPS tracker in my car, a good idea for both tracking my position and also checking to see if it has gone walkies (is stolen). I decided against doing this a while ago due to exams and lack of time to set the whole thing up. But I did decide to have a crack at this idea later with my phone.

The way I chose to go about this power hungry task was to use an app like IFTTT to record my position to a spreadsheet. But then I heard about owntracks, a purpose-built app for tracking your position. After setting up the owntracks server, MQTT and owntracks recorder. I now have a pretty good way of determining where I am at one time or another and have been using it for about 6 months.

My phone is a OnePlus 3 and as I said the app is fairly power hungry. Currently owntracks uses ~39% of my battery’s capacity based on 1 day 1 hour uptime. Although this seems fairly high I have accepted it as a fair compromise untill I can finish another GPS project I have in the works.

Here’s my experience of such a setup so far,

  • Battery usage increased significantly to a point where some users may not find it acceptable.
  • I had to turn off battery optimization and advanced battery optimization or else the app would stop.
  • The app does not always capture my location all of the time, sometimes it may be a few hours before it reports a significant change despite being in move monitoring mode or significant changes mode.
  • The performance and recording of positions in the app isn’t as good if the app does not have internet access all the time, so if data is switched off in my phone, the recording of the position isn’t as good as if it were on all the time, sometimes there can be large gaps where there was no internet access where it drops out.
  • My use case was for later analysis of the data, not realtime.
  • Owntracks recorder is not very featured but it does the job.
Oneplus3 Battery usage with owntracks

Exporting GnuCash Data to PowerBi

Some things are better if you do them yourself. I mainly did this project to keep a running ledger of the changes that I would need to keep track of the account balance data.

GnuCash is great, but when I export my accounts data the CSV file isn’t easily translated with power-query automatically. I decided that because I need to keep track of my transactions, this problem was best suited with a little program to calculate my net inflows and outflows using a python program, I also decided halfway through the project that I wanted to hook it up to my graphing backend, a silly idea – but a fun one to see my spends go up and down. I decided that I’d publish it here so that in the future I would find it a lot easier.

Traditionally if you wanted to export GnuCash data from CSV to PowerBi, you’d be better off using their inbuilt power query, however, I wanted to implement a ledger system, something that I don’t think can be accomplished directly in PowerBi without some scripting, and in the future I want to be able to change platforms if I need to because I no longer have a licence for PowerBi or want to use something else like excel or free equivalents like Google Sheets. I reckon that if done properly Google Scripts could make everything run automagically from an upload, but I don’t have my reports ready yet for that to happen.

# Extracted from here https://github.com/aidancrane/GnuCash-CSV2CSV-for-PowerBi/blob/master/convert_to_powerBI.py

import csv
from re import sub
from decimal import Decimal
import time


# If we only want the transactions values, set this to true and they will print to console, they do not save to accounts_out.csv!
numbersOnly = False

# Anyone thats not me will want this to be false, this is used to show the transaction data on a live graph I use for scratching around with.
smoothieChartEmulation = False
sessionCookie = "39494goodluckguessingthispartlololol213232expiresanyway"

# Leave this here so that Notepad++ and Atom auto-suggest it.
# Date,Account Name,Number,Description,Notes,Memo,Category,Type,Action,Reconcile,To With Sym,From With Sym,To Num.,From Num.,To Rate/Price,From Rate/Price

if (smoothieChartEmulation):
     import requests

# Negative Numbers are bracketed when exported from GNUCash so we need to fix that for the float data type.
def convert_if_negative(number):
    returnNumber = str(number)
    if ("," in returnNumber):
        if ("(" in returnNumber):
            returnNumber = returnNumber[1:]
            returnNumber = returnNumber[:-1]
            returnNumber = returnNumber.replace(",", "")
            returnNumber = 0 - round(float(returnNumber), 2)
        returnNumber = str(returnNumber).replace(",", "")
    if ("(" in returnNumber):
        returnNumber = Decimal(sub(r'[^\d.]', '', returnNumber))
        return (0 - round(float(returnNumber), 2))
    return returnNumber

# open accounts.cvs, our exported file.
with open("accounts.csv", "r") as csvIn:
    reader = csv.DictReader(csvIn)
    entries = []

    runningTotal = float(0)
    
	# Save
    for row in reader:
        if (row["Account Name"] == ""):
            pass
        else:
            runningTotal = runningTotal + float(convert_if_negative(row["To Num."]))
            if (numbersOnly):
                print(str(round(runningTotal, 2)))
            else:
                if (smoothieChartEmulation):
                    payload = {'random_graph': runningTotal}
                    r = requests.get('https://dash.infinityflame.co.uk/dash/flex.php', params=payload, cookies={'PHPSESSID': sessionCookie})
                print(str(convert_if_negative(row["To Num."])) + " Description: "+ row["Description"] + " Account Balance: " + str(round(runningTotal, 2)))
                entries.append([row["Date"],row["Description"],row["Category"],str(convert_if_negative(row["To Num."])),str(round(runningTotal, 2))])

# Save what we care about to our new csv for power BI   
with open('accounts_out.csv', mode='w', newline='') as csvOut:
    titles = ["Date","Description","Destination","Transaction","Account Balance"]
    writer = csv.writer(csvOut, delimiter=',', quotechar='"', quoting=csv.QUOTE_MINIMAL)
    writer.writerow(titles)
    for transaction in entries:
        writer.writerow(transaction)

I used python because it’s quick to debug in Atom and I know it well. The way this snippet is used is to export an account from GnuCash, then use this program to output the final CSV to be taken in by PowerBi, I’ve chained these steps together. The final result is stored in accounts_out.csv.

Thanks!

Opening up my Tile Mate

Bluetooth trackers are a great addition to anyone’s cluttered key chain, Last year I got a Tile Mate as a present. A nifty little key-chain Bluetooth connected sounder for when I loose my keys, or when I loose my phone.

So far given one years use I have maybe used it seriously to find my phone 2 or 3 times, it works very well for it. Anyway now that I’ve had a years use, the Tile app decided they wanted more money as the battery was getting low. Interestingly enough in the newest versions of the Tile Mate the battery is replaceable, however in this version (replaceable or not) the method to open the case is rather destructive.

The Tile Mate Main board, front and back case housing.

Forgive the image, it was rather late at night, As you can see there was a replaceable CR2025, so changing the battery was as simple as replacing it with a battery I had spare. I wonder what ’62’ means?

I couldn’t get the data-matrix to scan on my phone app unfortunately, but the board is very pretty and its clear a lot of thought has gone into keeping the size small. I like the little Pogo pins on the right for the piezo on the lid and the little pcb antenna. Also featured, the timing crystal oscillator, microprocessor and what looks like a version number for the board.

Anyway, then I put it all back together nice and snug inside its now fairly tarnished case and used some 5-second-fix glue to glue it back together and jobs-a-gooden, almost. It seems like the message on the app still persists.

Anyways, I’m still happy with it, it’s served its time and I think I’ve broken it a little too much to replace it again. In the future perhaps with the latest version, tile could offer some sort of slot so that the battery can be replaced a little less destructively (or maybe I did it wrong, oops). Nevertheless I don’t advise doing it yourself. TTFN, Aidan.

Cya later, Mate.

Gigabit and More on my Home Network

In a recent pursuit to have the fastest speeds possible at home, I have started to upgrade and replace a large section of my home network.

Ensuring That My Network Runs Fast at its Core

Barring the routing technologies at home the most crucial part of my network is my server, an old Dell Optiplex 780 salvaged from an office clear-out it now spends its days hosting my media, files and applications. One of the first things I did was replace all of the cabling around my network. Most being CAT5 the fastest speeds possible were only 100Mbps, this was a noticeable bottleneck when I checked iPerf3 as the switch its connected to, the router and my PC were all gigabit connected and clearly an oversight on my part. Replacing the cable from it to the switch instantly jumped the speed to gigabit. Yay!

Getting Rid of Old Hardware

Another decision I made was to remove some of my old hardware that was no longer up to the task of Gigabit+ speed. That for me meant swapping out a dumb switch capable of 100Mbps max for a new 8 port gigabit one, oddly I bought the 100Mbps one for £10 and sold it for £14 on Ebay so I guess theres money in old switches.

I then replaced its cables too and viola, 1 Gbps throughput on the whole wired network!

What does the future hold?

After looking at transfer speeds between my server and PC, I think the next step is to replace the Optiplex and my PC, as they are now at the point where their age is starting to be a performance problem, although they remain somewhat performant at their daily duties they are both starting to age poorly with the increasing demand from myself for hosting my projects.

I also want to try SFTP+ but I don’t yet have the SSDs or Hard-drives to necessitate it.

My Meraki Home Network

Cisco Meraki MR18
Cisco Meraki MR18

If you have ever wanted to use WiFi reliably in your own home, most homelabbers and computer enthusiasts will eventually learn that using consumer equipment to achieve fast speeds and low ping can be a route that many take in vain, whilst mesh networking appliances look promising for home applications, the ever vast expanding market of power-line networking adapters and WiFi range extenders have lead consumers astray and often leave them unhappy with their network performance due to patchy and inconsistent¬†behavior caused by devices looking for the ‘strongest’ signal despite¬†it not necessarily being the fastest or most optimal.

Having an ISP issued router has never put me off fiddling before, having being issued a Netgear WNR3500L V2 it wasn’t long before I decided to flash Tomato on it and leave Netgear Genie hopefully forever.

Having moved on with my life I quickly became unhappy with the performance of the WiFi and decided it needed sprucing up a little bit, running two routers on two sides of the house were becoming tiresome due to patchiness and being connected to one side of the house while at the other requiring manual adjustment. It was time to buy something enterprise to set-and-forget. Alas enterprise comes with a hefty price tag.

Rather than going mainstream and buying something like Ubiquiti or MikroTiK, popular routers with the home lab community. I decided to take a different approach to save a buck. Having flashed firmware on many devices in the past, such as Arduino, ESP8266s or even other routers I decided to get my hands on the enterprise CISCO Meraki MR18, a subscription model router with a hefty price tag at the current time of £308.38 for a three-year license. Would I be paying that? Absolutely not.

I picked one up off eBay for a fairly high market price of¬†¬£17.99 which came with a wall bracket and no plug or license. Why so cheap compared to the¬†¬£308.38 price tag? Well, it turns out Meraki are giving them away free to anyone who sits through their webinar, and once the license runs out it’s my understanding that they essentially¬†become paperweights. Once it arrived I fired it up just to see what it used to be called and see if I could get a hit on WiGLE’s database to see where this thing came from, unfortunately, 0 hits means my best guess is that it came from somewhere around Aylesbury, UK. The location of the Ebay E-Recycler the SSID was generic¬†“Guest Access” and “Staff Internet”.

Once I cleaned up the router it looked brand new, the power cable I got from my drawer of many cables and began to study the thing, the brushed aluminum backing had the screws hidden behind rubber feet, an easy adversary I had the thing open in no time.

Meraki MR18 opened
Big RF Shields ūüôā

I particularly like the bendy lightpipe for the RGB leds.

Bendy Light Pipe

The UART pins were neatly placed and populated with pins at the top of the AP and easy to flash with.

I decided to opt for flashing OpenWRT on to it, basically because at the time they were the only ones supporting this particular device and it would only really be acting as an Access Point, with my Tomato router doing all the hard work.

Thanks for reading. Enjoy this picture of my neighbors cat.

Tabitha the cat