While doing some capacity shuffling today I wondered if bgp.tools had gotten to the point where there is no single motherboard that could run the whole operation. The answer is... almost You can buy 2S AMD 9005 machines (2x 128 C parts) that have 48 DIMM slots (48 * 128G DDR5 = 6TB~) that would fit what normally does the whole half cab into one machine.. The downside is that this one box costs roughly 280,000£, so I will take my setup thank you very much
mcr314@todon.nl
replied 01 Jul 2026 14:56 +0000
in reply to: https://benjojo.co.uk/u/benjojo/h/4QMXTn7qvM8d2H4d2p
@benjojo where does the ram all go? I get that you have many BGP sessions up, with DFZ feeds... Surely that part does not need to be on a single machine?
benjojo
replied 01 Jul 2026 15:10 +0000
in reply to: https://todon.nl/users/mcr314/statuses/116845301767994066
penguin42@mastodon.o..
replied 01 Jul 2026 15:40 +0000
in reply to: https://benjojo.co.uk/u/benjojo/h/4QMXTn7qvM8d2H4d2p
@benjojo The Lenovo SR950 V3 is an 8 socket Xeon (two strap together for 8S) which claims it can do 32TB RAM and 480 cores; I guess the people who use them for SAP can afford it. (Although they're paranoid about ever having to reboot them since the time it takes to shuffle 32TB of data back into RAM is too long for their service).
benjojo
replied 01 Jul 2026 15:46 +0000
in reply to: https://mastodon.org.uk/users/penguin42/statuses/116845473893334311
@penguin42 Yeah the 8S machines are also a considerably more magic, See https://community.hpe.com/t5/servers-systems-the-right/defying-the-law-of-diminishing-returns-hpe-superdome-flex/ba-p/7012970 I suspect this system is utterly haunted, also at some point you start to have so many DIMMs etc that you are basically asking for problems (yes, I know there is RAS etc, but I don't think Intel's version is bomb proof enough in the same way other non x86 systems managed to get it)
penguin42@mastodon.o..
replied 01 Jul 2026 15:56 +0000
in reply to: https://benjojo.co.uk/u/benjojo/h/3kv228P677L2f3By5N
@benjojo Yeh; I don't think the 8 sockets are *that* weird - I think they're mostly standard design with some glue for the link between the two boxes; not as crazy as the Superdomes. I had been thinking of getting a few gen old 4 socket; they're pretty cheap second hand, and well if you can find 4GB DIMMs - they are still cheap if you can find them, and just fill it up.
benjojo
replied 01 Jul 2026 15:58 +0000
in reply to: https://mastodon.org.uk/users/penguin42/statuses/116845537448972015
@penguin42 yeah the problem with 8S though is the performance sucks if you ever make the mistake of intra NUMA memory access, and not just "oh that kinda hurts" in the way it is on 2S systems... It really sucks
benjojo
replied 01 Jul 2026 14:48 +0000
in reply to: https://secluded.ch/users/jeroen/statuses/116845246643778553
@jeroen I've had the complete opposite in my career so far, one big-ass-box can run laps around the availability/integrity of clusters of machines, it's a question if you would rather take one big "bang" outage every ~5 years or tiny outages every couple of months
mcr314@todon.nl
replied 01 Jul 2026 14:59 +0000
in reply to: https://benjojo.co.uk/u/benjojo/h/CXFjh7yYJwDrGLCThM
benjojo
replied 01 Jul 2026 15:13 +0000
in reply to: https://todon.nl/users/mcr314/statuses/116845313037461269
@mcr314 @jeroen I guess I'm generally a bit absolutist (when I can be) in this regarding that I don't actually see active/standby as a good option and I will try as hard as I can to avoid it, active/active or you may as well go restoring a backup, since the number of times I've seen such standby operations go wrong or cause extra work/surprises etc, or just not work correctly when the standby needs to become active
FritzAdalis@infosec...
replied 01 Jul 2026 16:41 +0000
in reply to: https://benjojo.co.uk/u/benjojo/h/dVnB34pr1Wvj3fjbpd
benjojo
replied 01 Jul 2026 16:47 +0000
in reply to: https://infosec.exchange/users/FritzAdalis/statuses/116845713843769652
@FritzAdalis @mcr314 @jeroen Yeah but that is a huge commitment right? I understand that not everything can be active/active but _surely_ most systems can be these days, or people can accept downtime for rare "shit's fucked" situations. Though if it's something like a EHR system, then yeah I can see how this might be a more complex take...