-
Notifications
You must be signed in to change notification settings - Fork 0
/
Changelog
70 lines (49 loc) · 7.92 KB
/
Changelog
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
# Changelog
## 3-12-2022
A quad port SFP+ card is unnecessary.
- I won't be needing independent 10 gigabit connections for each VM.
- The dual port NIC I have supports SR-IOV in case I want to pass through each port to separate VMs.
- The motherboard I have has two onboard 10 gigabit NICs providing me a total of four 10 gig ports.
## 3-14-2022
It seems like the folks at iXsystems really want TrueNAS Scale to be an appliance which is not a great fit for my intended use case on the primary NAS. I'm going to stick with Core on the primary for the following reasons:
- The `apt` package manager is disabled from the command line.
- Clustering features using `gluster` requires a paid license to TrueCommand
- Clustering features don't yet have a GUI.
- If I want to make a cluster I'll need a third box, and that's not currently in the budget.
## 3-15-2022
While the primary NAS may need some bells and whistles the backup NAS can be a simple appliance. I can sell the Mini and replace it with a Synology NAS using the same drives. Also installed a 2U mounting plate in the rack for the Pi cluster.
## 3-16-2022
Current desktop machine died. Repeated notices I was running out of space(which were at odds with output from `df -h`) were followed by a hard lock and reboot into the `initramfs` console. Instead of attempting to repair partitions I opted to replace the OS drive altogether with a larger 1TB SSD. I had also been planning a CPU upgrade to a 5700G so I could turn it into a VFIO box using Looking Glass. Boot was not successful after that was done. Restoring original components did not improve the situation, so now it appears the motherboard is the culprit. Replacing the board and trying again.
## 3-18-2022
Cooling loop for the workstation is nearly complete and ready to be filled. Due to the immediate 90 degree bend on the fill line it's difficult to fill the loop without spills that cause all kind of annoying and time consuming cleanup jobs. I'm waiting on a few extra fittings that I can connect to the fill and drain ports in order to make filling and draining easier. All the fittings I have are for hard lines, and being able to use a soft line while filling or draining the loop will make things much easier.
## 3-19-2022
Only one of the two 1TB SSDs currently used in Stockpile's SSD pool will make its way into the workstation. That will give me 7TB of NVMe storage plus an additional ~2TB SATA storage for virtual machine disks. I'd probably need to add a SATA power splitter to make the fourth SATA SSD fit otherwise the cabling would get super annoying, and it wouldn't really provide that much more additional storage. I'd rather try to sell it, so into the ebay box it shall go.
## 3-21-2022
We have boot! Bedlam is alive. Considering doing the Pi KVM thing since I have one 2GB Pi4 which I couldn't fit in the rack. The 2U Pi rackmount kit I got came with a bad mounting tray, and I couldn't be bothered to RMA it or 3d print a replacement. It'll probably get turned into either a tiny HTPC, or a Pi KVM for bedlam.
## 3-27-2022
Rather than virtualizing FreeNAS I'm going to be keeping Stockpile around as a separate system. It's the more enterprisey way to do things, and I didn't like how the drives were obstructing airflow from the front radiator. I have a feeling lack of airflow through the case is part of what killed the motherboard in the desktop. I'll be swapping the 10TB SAS drives I bought for the workstation into it, and replacing the 8TB SATA drives it's using currently. I'll sell the 8TB drives, and maybe set up a 2x10gig link aggregation for the NAS. In the unlikely event that I regularly saturate a single 10gig link having the link aggregation will prevent any other concurrent transfers from being impacted.
This leaves the Fujitsu HBA without a home. Stockpile already has a SAS controller that I flashed to IT mode when I first built the system, and I don't have any other use for it, so that will also be going into the ebay box.
The VFIO machine will use both of the 1TB SSDs currently in Stockpile. I have three 1TB m.2 drives leftover from the previous iteration of the workstation, and had planned to use one of those in the desktop/VFIO machine as its OS drive. I think the better move is to sell all three of those, and the 4x M.2 PCIe card that came with the Aours Xtreme. It's unlikely I'll need more than 2TB of local storage on this machine just for host OS, VM, and game storage.
## 4-3-2022
After a crazy week at work I'm deciding against re-building the old desktop machine and having a separate machine dedicated to VFIO shenanigans. The laptop from System76 is doing fine as a daily driver. I'll be returning the parts I bought for it's cooling loop, and adding the rest to the list of stuff to sell. I may decide to keep a small SSD pool in Stockpile depending on how the performance of a 2x1TB mirror of SATA SSDs stacks up against the 8x10TB RAIDZ1 SAS drives.
## 5-1-2022
Drive swap into Stockpile is done. I've now got eight 8TB SATA drives sitting around with the rest of the spare spare parts to sell. I also spent some time finally labeling my patch panel so I can stop chasing wires down every time I work on the rack. I bought a CRS326-24S+2Q+RM from Mikrotik as well which I'll be using to replace the 16+4 10gig switch from Ubiquiti. I already have a few of Mikrotik's other switches, and I'm not fond of Ubiquiti's decision to sue the journalist who reported on their(somewhat) recent data breach. I'll start with the switch, and then as time and budget allows I'll replace their APs also. I never made the jump to wifo6 since everything in my homelab is wired, so wireless isn't as much of a priority. Documentation on Mikrotik stuff isn't great, and I'm not sure how applicable experience with their hardware is going to be in terms of networking in general, but it's a decent and relatively inexpensive way to get started.
This also gives a chance to build out decent vlans for storage, containers, and management which is something I never really got around to previously.
## 5-5-2022
Lots of network updates.
- Moved all connections away from XG16 switch to the Mikrotik CRS326
- Installed the 40gig Chelsio cards in the workstation and NAS.
- Removed one of the two APs. I only need one, so the other is going to get sold.
- Created VLANS for IoT, storage, containers, and management.
-- Moved 40gbe interface on Stockpile to storage vlan
-- Moved IPMI, and web portals to management vlans
-- Moved AP, 3d printer, TV, and other devices to the IoT vlan
- Created link aggregations:
-- 2x10gig from CRS326 to CRS305
-- 3x10gig from CRS326 to CRS328
-- 2x10gig from CRS326 to Stockpile on container VLAN
While doing all of this, the PoE switch seems to have died. It was acting strange and dropping connections to its management portal. Finally I lost access to the management portal altogether even after a factory reset.
## 5-21-2022
Turns out the PoE switch isn't dead. It just has weird reset behavior coupled with the fact that I changed its fallback IP and then forgot I had changed it. I've seen people mess with switch configuration using Ansible, but I should probably start elsewhere with Ansible once I finally get around to it.
Having multiple different link aggregations on the same switch is giving me the business also. I keep getting broadcast storms at random times as if it's forgetting that the interfaces in the LAG are in fact bonded together. For now it's not so much of a problem, but I would like to get that working eventually. The Mikrotik switches seem decent for the price, but the more oddballs I find the more likely it seems that experience gained here may not be directly applicable elsewhere. It'll do fine though for the deeper dive into networking I've had planned for quite some time now. The concepts should be the same it's just the mechanics of it which are different.
In addition to that, I'm also working on setting up a simple site using Hugo as a kind of portfolio/blog where I'll be writing these updates.