1 Nov 2024

New Router: BananaPi R3 - Part 3 - Configuration

After being subjected to numerous mean glares from the wife and accusations of "breaking the internet", I think I've got it all configured now...

https://www.k3can.us/garfield.gif

Ironically, the more "exotic" configuration, like the multiple VPN and VLAN interfaces were pretty simple to set up and worked without much fuss. The part that had me pulling my hair and banging my head against the desk was just trying to get the wifi working on 2.4GHz and 5GHz at the same time... Something wireless routers have been doing flawlessly for over a decade. After enough troubleshooting, googling, and experimenting, though, I had a working router.

I installed AdGuardHome for dns and ad blocking, but kept dnsmasq for DHCP and local rDNS/PRT requests. dsnmasq's DHCP options directs all clients to AGH, and AGH fowards any PTR requests it recessives to dsnmasq.

Next, I installed a Crowdsec bouncer and linked it to my Local Crowdsec Engine. Now, when a scenario is triggered, instead of banning the offending address at the server, it will be blocked at the edge router level instead. 

 

Lastly I installed and configured SQM (Smart Queue Management), which controls the flow of traffic through the router to the WAN interface. Without this management, the WAN interface buffer can get "bogged down" with heavy traffic loads and cause other devices to experience high latency or even lose their connection entirely. SQM performs automatic network scheduling, active queue management, traffic shaping, rate limiting, and QoS prioritization.

For a comparison, I used waveform to test latency under load.

Before SQM:

====== RESULTS SUMMARY ====== 	
Bufferbloat Grade	C
	
====== RESULTS SUMMARY ====== 	
Mean Unloaded Latency (ms)	51.34
Increase In Mean Latency During Download Test (ms)	76.01
Increase In Mean During Upload Test (ms)	8.69

After SQM:

====== RESULTS SUMMARY ====== 	
Bufferbloat Grade	A
	
====== RESULTS SUMMARY ====== 	
Mean Unloaded Latency (ms)	38.92
Increase In Mean Latency During Download Test (ms)	12.75
Increase In Mean During Upload Test (ms)	1.5

I have to say, I'm pretty happy with the results!
Going from a grade C with a 76 ms increase to a grade A with only a 12.75ms is a pretty substantial difference. This does increase the load on the CPU, but with the BPI R3s quad core processor, I expect that I'll still have plenty of overhead.

Overall, I think I'm happy with the configuration and the BPI R3 itself.

27 Oct 2024

New Router: BananaPi R3 - Part 2 - Flashing

Part 1 is here.

Now that the router is assembled, the next step is to decide where to flash the firmware. As I mentioned in the last post, this device offers a handful of options. Firmware can be flashed to the NOR, NAND, eMMC, or simply run from the SD card. From what I've read, it's not possible to boot from an m.2 card, though. That option is only for mass storage.

After a bit of reading, my decision was ultimately to install to all four!  Sort of...

Image showing the leads connected the UART connector and the DIP switches
The DIP switches and the leads connecting to the UART

My plan is to install a "clean" OpenWRT image to both the NOR and NAND.  The NAND image will be fully configured into a working image, and then copied to the eMMC. The eMMC will then be the primary boot device going forward.  If there's a problem with the primary image in the future, I would then have a cascading series of recovery images available. At the flip of a switch, I can revert to the known working image in the NAND, and if that fails, then I can fallback to the perfectly clean image in the NOR.

..And I do mean "at the flip of a switch". Due to the way that the bootable storage options are connected, only 2 of the 4 can be accessed at a time. Switching between NOR/NAND and SD/eMMC requires powering off the BPI and toggling a series of 4 dip switches, as seen on the official graphic below:

https://wiki.banana-pi.org/images/thumb/4/4c/BPI-R3-Jumper.png/320x107x320px-BPI-R3-Jumper.png.pagespeed.ic.Qyd9EK01n9.png

Switches A and B determine which device the BPI will attempt to boot from. Switch C sets whether the NOR or NAND will be connected and switch D does  the same for the SD and eMMC. To copy a image from the SD card to the NOR, for example, switch D must be high (1) to access the SD card, and switch C must be low (0) to access the NOR.  Since switches A and B set the boot device independent of which devices are actually connected, it would seem that you could set them to an impossible state to render the device unbootable, like 1110, or 1001. 

To accomplish my desired install state, I had to first write the OpenWRT image to the SD card on a PC, and then insert it into the RPI. With the the switches to 1101, I could write the image from the SD card to the NOR, then flip the switches (with the BPI powered off) to 1111 to copy the image to the NAND. Lastly, I can remove the SD card and reboot with the switches in 1010 to boot from the NAND. Then I'll configure the BPI into my fully configured state. This is the step I'm currently working on. I have it about 80% configured, but will need to actually install it in my network before I can complete and test the remaining 20%.  Once it is installed, tested, and fully configured, I'll  copy the NAND to eMMC, before finally setting the switches to 0110 and booting from the eMMC for ongoing use.

Unfortunately, I haven't had a good opportunity to take my network offline to install the new router, so the last bit of configuration might need to wait a little while...

 

22 Oct 2024

New Router: BananaPi R3 - Part 1 - Hardware

I've been using a consumer router from from 2016 (with OpenWRT hacked onto it) all the way here in 2024, and felt that it might finally be time for an upgrade. I settled on a BananaPi R3 because it was a reasonable price and seemed like it would be a fun project.

Here's the bare board as received:

You can see most of the physical features in this photo, including a USB3 port, two SFP ports, 5 RJ45 ports, an m.2 slot for a cellular modem, and a 26-pin GPIO header. On the bottom, there's also a m.2 slot intended for nvme storage, as well as slots for a micro SD and a micro SIM.  The CPU is a quad-core ARM chip paired with 2gb of RAM, and there's a handful of flash chips, providing NAND, NOR and eMMC.   Quite a lot of options!

My plan is to install OpenWRT to the NAND storage. I suspect the nvme might be useful if I wanted to run a small file server or something, but that's not in the plan for now.

 

The first step I took in assembly was to apply some thermal pads to the chips and then attach a cooler and fan.

The thermal pads are "OwlTree" brand, but I don't have any specific preference to them, I just happen to already have them on-hand from a previous project. The CPU is a 0.5mm pad applied, and I applied 1.5mm pads to the remaining chips.

Thermal pad applied to CPU

After applying  pads to all of the chips, I attached the cooler and plugged in the fan.

The next step was to install the board into the case. I went with the official BPI-R3 case. The quality is surprisingly nice and looks great once assembled. After installing the board I then installed the pigtails for the eight (yes, eight) antennas and applied some basic cable management.

Board installed into case and coax attached and routed to antenna ports.  

Now, I can't finish putting the case together quite yet, since I'll need access to the UART pins to install Openwrt to the NAND flash. The  UART header can be seen on the right side of this photo, but there is no way to access it once the case is assembled.

But, that's enough for today. I'll post an update once I make some progress towards getting OpenWRT flashed.