Feel like in general of running the bgp.tools business over a time I have made pretty good hardware purchasing decisions, I would however make one exception to that. If at any point you are buying SSDs with the intention of using them for more than bare idle workloads, for the love of god spend the 2x price multiplier on enterprise drives. I'm officially over replacing Crucial MX500s in production, I finally pulled the trigger on replacing all of them left because they consume so much time just keeping up with them slowly all burning out at slightly different rates, even though I decided to not buy anymore over 9 months ago Life is to short to stand around in a data hall waiting for a mdraid to resync
beasts@social.mythic..
replied 29 Sep 2024 15:29 +0000
in reply to: https://benjojo.co.uk/u/benjojo/h/Y9fF3M74gN4Jhxzy5j
benjojo
replied 29 Sep 2024 15:43 +0000
in reply to: https://social.mythic-beasts.com/users/beasts/statuses/113221549911552671
kasperd@westergaard...
replied 29 Sep 2024 19:56 +0000
in reply to: https://social.mythic-beasts.com/users/beasts/statuses/113221549911552671
(@benjojo@benjojo.co.uk @beasts@social.mythic-beasts.com) You've definitely got a relevant point. But that metric can be harder to find and even harder to verify.
beasts@social.mythic..
replied 29 Sep 2024 21:15 +0000
in reply to: https://westergaard.social/objects/55449c72-d80b-4d9c-b2e5-abbece493c31
0x47df@duckpon.de
replied 29 Sep 2024 16:37 +0000
in reply to: https://benjojo.co.uk/u/benjojo/h/Y9fF3M74gN4Jhxzy5j
@benjojo i found this out the hard way with my gotosocial instance yesterday. apparently there are something in the region of 220tb written to the used SSD inside it. it can write well for about a week before a reboot to flush the write queue has to happen. somehow no SMART failure, nor reallocated blocks. but wow it has become absurdly slow.
benjojo
replied 29 Sep 2024 16:43 +0000
in reply to: https://duckpon.de/users/0x47df/statuses/01J8ZAA3B29ZJEHBHHX7HKBWJ5
@0x47df yeah that sounds about right, I have a alert when a drive goes above to 80% of its life span. The MX500's I am now aggressively phasing where going up by 1% a week in most cases, the new enterprise drives have yet to pass their first %... and yet only cost about 2x what a MX500 would
0x47df@duckpon.de
replied 29 Sep 2024 16:43 +0000
in reply to: https://benjojo.co.uk/u/benjojo/h/3DlJ7N29SvZ66T4l4x
@benjojo well the smart data on this disk swears blind it's got plenty of life left, but the perf stinks of otherwise. i guess the life was calculated against a desktop workload. i have learned my lesson also i guess ;;
jakob@mastodon.chaos..
replied 29 Sep 2024 16:55 +0000
in reply to: https://benjojo.co.uk/u/benjojo/h/Y9fF3M74gN4Jhxzy5j
@benjojo I've made good experience so far by looking for drives with TLC or lower and DRAM Cache. Apacer and Samsung with quite a lot of drive writes and a few years of operation seem to hold up good. But yes the moment you have the money - especially as a business - enterprise drives are worth it. Just the power loss protection alone can make the difference between just booting again and having to restore a backup.
albonycal@fosstodon...
replied 29 Sep 2024 18:18 +0000
in reply to: https://benjojo.co.uk/u/benjojo/h/Y9fF3M74gN4Jhxzy5j
@benjojo I have Crucial BX500 running on my server. They have their own custom S.M.A.R.T attribute for this > It is a measure of how much of the drive’s projected lifetime is remaining at any point in time. When SSD is new it will report “100”, and when its specified lifetime has been reached “0,” Mine is at 50% right now
``` 202 Percent_Lifetime_Remain 50```
it does not mean that the drive is going to fail when that counter reaches zero, only that your SSD may need to be replaced soon.
benjojo
replied 29 Sep 2024 18:37 +0000
in reply to: https://fosstodon.org/users/albonycal/statuses/113222214968039435
@albonycal I know, however I am not interested in dealing with actively broken SSDs, especially consumer grade ones, who sometimes return random garbage, partial pages, or block/stall other things
albonycal@fosstodon...
replied 29 Sep 2024 18:47 +0000
in reply to: https://benjojo.co.uk/u/benjojo/h/5FggS31N494Jw412lX
benjojo
replied 29 Sep 2024 18:51 +0000
in reply to: https://fosstodon.org/users/albonycal/statuses/113222330893077528
jeff@noxon.cc
replied 29 Sep 2024 18:36 +0000
in reply to: https://benjojo.co.uk/u/benjojo/h/Y9fF3M74gN4Jhxzy5j
@benjojo The MX500 in particular seems to wear out incredibly fast. What enterprise SATA disks do you recommend? I need to replace some MX500s soon, but most enterprise models aren’t SATA these days.
benjojo
replied 29 Sep 2024 18:40 +0000
in reply to: https://noxon.cc/users/jeff/statuses/113222288396514996
@jeff I bought a load of Samsung "MZ7L3960HCJR" to replace them, also known as Samsung PM893, they have been faster, more consistent, and the hardest hit one has only just hit it's first % of lifetime after 100+ days The only annoying thing about weening off the consumer drives for the enterprise drives is that the enterprise ones tend to be 480G/960G, where the consumer ones are 500G/1T, meaning if you want to easily do a RAID swap upgrade you have to double your sizes
jeff@noxon.cc
replied 29 Sep 2024 18:43 +0000
in reply to: https://benjojo.co.uk/u/benjojo/h/kC1L2rD7XZhKjyxCkR
@benjojo Thanks, I never would have thought to look at Samsung. I got some new-old-stock Intel 2.5” enterprise drives years ago from eBay, and they needed weird adapters to work with 3.5” SATA connectors, but they’ve been amazing.
moreentropy@chaos.so..
replied 29 Sep 2024 19:17 +0000
in reply to: https://benjojo.co.uk/u/benjojo/h/Y9fF3M74gN4Jhxzy5j
@benjojo have settled on WD red for now (for my 24/7 homelab) as they are supposed to be more enterprise-y with write cycles and I can buy replacements at local pc stores.
A wd blue i bought just died after ~2 years of 24/7 use.
benjojo
replied 29 Sep 2024 19:47 +0000
in reply to: https://chaos.social/users/moreentropy/statuses/113222448178775786
@moreentropy It's weird to hear people talking about the WD core brands (Red/Blue/Purple/Gold/etc) but with SSDs. I've always had trust issues with them re-using the lingo like that
wrmsr@peering.social
replied 30 Sep 2024 14:21 +0000
in reply to: https://benjojo.co.uk/u/benjojo/h/Y9fF3M74gN4Jhxzy5j
@benjojo I can relate to that, as I have multiple projects causing high write volumes to my SSDs coming from my databases. They are mostly Samsung consumer m.2 SSDs (ranging everything from 960 to 980) but I am at the point of having to replace one every 6-8months due to them all reaching their TBW limit. I've started now replacing everything with PM9A3 4TBs and, oh boy, was this ever a good decision. Looking at those power loss protection caps also has a good feeling (despite me running ZFS).
benjojo
replied 29 Sep 2024 16:31 +0000
in reply to: https://benjojo.co.uk/u/benjojo/h/Y9fF3M74gN4Jhxzy5j
On the bright side, at least I had a pile of nearly-but-not-entirely burnt out SSDs to give to the EMF Arcade people this year
IPngNetworks@ublog.t..
replied 29 Sep 2024 17:50 +0000
in reply to: https://benjojo.co.uk/u/benjojo/h/n5f5nhfQgc57Tf2fhR