Migration "complete", bgp.tools last night moved from a 1/4 cab to a 1/2 cab. I stuck my old phone to the other side to take a timelapse of the processes, and it caught a few things. Including a mildly scary moment where the top U storage box's rails failed and nearly crushed me... OH well! There are definitely other lessons to follow from this migration, but I think I am getting better at this (not that I am planning on doing this again for at least a full year I hope)
one honk maybe more
jamesog@mastodon.soc..
replied 26 Feb 2024 18:09 +0000
in reply to: https://benjojo.co.uk/u/benjojo/h/kzc6n9rjX6Sk8c4DVs
@benjojo Me and a colleague were once re-racking one of these (Sun V490, 5U, ~60kg) when it fell off the rails and just missed both our feet
benjojo
replied 26 Feb 2024 18:24 +0000
in reply to: https://mastodon.social/users/jamesog/statuses/111999123115277384
@jamesog Impressed it didn't keep going through the floor, that gives off the vibe of "unstoppable object"
benjojo
replied 26 Feb 2024 13:59 +0000
in reply to: https://benjojo.co.uk/u/benjojo/h/kzc6n9rjX6Sk8c4DVs
Another really good call I made before unracking everything was labelling things like the server blades where they are suppose to be... with easy to understand arrows rather than numbers. hard to argue with yourself when the arrow on the NIC is pointing the wrong way. Disks also got clear slot labels too. I don't think ceph would care if I reordered all of the drives, but I do NOT want to find out!
benjojo
replied 26 Feb 2024 14:01 +0000
in reply to: https://benjojo.co.uk/u/benjojo/h/8h3Rk2bXRZvRBmnR2T
Also a moments silence for this generic Apple keyboard that ate the full weight of a storage chassis landing on it, and was likely the only reason the rails and case was not bent up. Dunno if it still works, I uh "straightened" it out afterwards. Who knows.
k@layer8.space
replied 26 Feb 2024 14:06 +0000
in reply to: https://benjojo.co.uk/u/benjojo/h/P7nlhD532hRCrzSPCr
Lioh@social.anoxinon..
replied 26 Feb 2024 14:38 +0000
in reply to: https://benjojo.co.uk/u/benjojo/h/P7nlhD532hRCrzSPCr
@benjojo at least it's only a keyboard and not a disk. Had disks with similar notes on it lying around for quite some time. Until I finally decided to throw them away 😱
wrmsr@peering.social
replied 27 Feb 2024 17:34 +0000
in reply to: https://benjojo.co.uk/u/benjojo/h/P7nlhD532hRCrzSPCr
@benjojo I had that happen to me aswell several years ago... but the full storage chassis crashed onto the hardware firewall ~6U below it, leaving a severe bump for the rest of that firewall's lifetime in its chassis... luckily I've never received any complaints about hardware stopped working 😅
resingm@infosec.exch..
replied 26 Feb 2024 13:58 +0000
in reply to: https://benjojo.co.uk/u/benjojo/h/kzc6n9rjX6Sk8c4DVs
benbe@social.chaotik..
replied 26 Feb 2024 14:28 +0000
in reply to: https://benjojo.co.uk/u/benjojo/h/kzc6n9rjX6Sk8c4DVs
In the vast and complex ecosystem of the data center, a remarkable event unfolds. A group of servers from a cramped cabinet is about to embark on a dangerous journey. They power down and unplug, ready to go. One after another hops out and heads to their new home. Exhausted, they reach their destination in the new cabinet and slide in. Reconnected they power up, humming peacefully. Migration is complete. Nothing reminds of the events that just took place. </DavidAttenboroughVoice>
benjojo
replied 26 Feb 2024 14:05 +0000
in reply to: https://bsd.network/users/ledeuns/statuses/111998133149523617
@ledeuns The only really spicy moment was that storage box. In hindsight I really should have had someone else there just for that machine alone. I did already know it was mounted a little bit sketchy. OH well. The new position is a lot safer for both future me, and the machine itself (less distance to fall if it does go wrong during unracking)