@benjojo ok, sure.
(I do really like the amount of vector graphics in SuperMicro documentation though)

@benjojo @feuerrot if the answer is 8 socket systems, the question is wrong 😹

@manawyrm @benjojo @feuerrot I imagine maybe AI stuff... even then not sure this is necessary

@kouett @benjojo @feuerrot @manawyrm If you want large scale memory footprints for in memory databases going this big is often the only option. @work we have a few usecases for this but could lukly solve it with just a shit ton of SSDs as swap.

These boxes are mostly a direct competitor for Power based scale-up systems used for SAP and related large scale databases with the uptime requirments. And having only one box to manage is often espeically in the older style enterprise a big plus

@kouett @benjojo @feuerrot @manawyrm Now with CXL based memory expantion on the horizon i'm not seen a real market anymore for this tbh. Given with CXL or related stuff you can easly cram a a bunch of memory into one system as long as it fits in the address space.

@erikk @kouett @benjojo @feuerrot Exactly. Use CXL and/or fix your applications.

Even dual socket is already a reliability nightmare in practice (at scale), having 8 points of failure (* memory channels, etc.) — yeesh.

@manawyrm @kouett @benjojo @feuerrot fixing you application is often not really possible or rally expensive to do, or just not yet a solved problem. Often then just trowing money at boxes like this is the cheapest solution, and the failure modes of these boxes are way more predictible then large scale distributed systems.

Like most large scale destributed fail just as often, if you see the failure rates most large GPU clusters are dealing with....

@manawyrm @kouett @benjojo @feuerrot oh and with CXL you run in an other filter, the bloom filters ( or other cache coherency structures) are often to small so if you go to a dual socket or other distributed design you will still see a performance impact.

@erikk @kouett @benjojo @feuerrot @manawyrm It's only SAP isn't it???

@feuerrot @benjojo "we have a mainframe at home"

@benjojo you can go ever higher 🙃 But also whyy??? I guess this is just really one strange usecase some has for it

@benjojo https://buy.hpe.com/us/en/compute/mission-critical-x86-servers/compute-scale-up-servers/compute-scale-up-servers/hpe-compute-scale-up-server-3200/p/1014774076 those boxes go up to 32 Sockets (or 16 for 4th gen). The limiting factor on the 8S is only the cache coherency and snooping, but intel does offer the specs so you can build / design your own custom asics to offload this. A bit like NVlink switches.

@benjojo You run RAID on your dimms in those setups, so its fine if one fails or trips, you can just take one offline and swap one back in. Similarly with the CPUs those are all hot swappable and can to auto recovery.

@benjojo But compared to what IBM can do these are all rookie numbers, during hotchips they had a theoritical 4000 socket deployment 😨

@benjojo @erikk Superdome…

@erikk @benjojo that thing has a shield in front of it. Be ready with your spear

@benjojo that's a biblically acurate server right there.

@benjojo Lenovo also do something like this still, I think as a pair of 4U(5U?) chassis that go together.

@benjojo Looks like a slightly beefier version of some Sun (SPARC) kit I used to run back in the day

@jamesog @benjojo A T5440?

@Tenzer @benjojo Bigger! A pair of M4000s.

@benjojo Those machines were pretty fun. The OOB was probably the best I've used. IIRC it was based on Linux/ARM. And a bit like a mainframe you could do either hardware partitioning or use its own hypervisor for logical partitioning.

@benjojo I think my favourite thing is the lack of handles.
Don’t try moving it without taking all the PCI cards and drives out of it