benjojo.co.uk

benjojo posted 17 Jun 2026 15:37 +0000

How do these cloud SDKs get so huge (from my go mod cache)

$ du -h --max-depth=1 |& grep G
1.6G	./huaweicloud
1.5G	./oracle
3.8G	.

huaweicloud-sdk-go-v3@v0.1.187 is 239MB of _text_, text, This module is 50,000+ tiny (less than 8KB go files) that look like this: https://github.com/huaweicloud/huaweicloud-sdk-go-v3/blob/5b4358369ac2cc3cc8985b52bd2d7930d654a8b2/services/gaussdbforopengauss/v3/model/model_list_enhance_full_sql_statistics_request_body.go#L4

That's a staggering amount of (I assume/hope) automatically generated code. A true horror case of the compiler

cloc showing the single version of the module coming in at 1,919,603 lines of go code

holsta@mastodon.art replied 17 Jun 2026 15:42 +0000
in reply to: https://benjojo.co.uk/u/benjojo/h/3V3zStV92c4Y9M35Tp

@benjojo That seems impossible to secure.

benjojo replied 17 Jun 2026 15:47 +0000
in reply to: https://mastodon.art/users/holsta/statuses/116766209634677245

@holsta I mean the versions are pinned and it's a memory safe language, so I don't care as much as it pointlessly consumes CPU cycles to compile and time for me to upload

benjojo replied 17 Jun 2026 15:40 +0000
in reply to: https://benjojo.co.uk/u/benjojo/h/3V3zStV92c4Y9M35Tp

The AWS SDK is like this too, I don't use AWS directly but it's SDK finds it way into the import path and adds a very cool 2MB of _compiled code_ to the binary, It stands out in the size report for being the only thing that is mostly autogen'd code and not embedded assets

A "go size analyser" output, focused on the AWS package, where over 90% of the whole thing is a unhelpful "auto generated" area

kestral@masto.hacker.. replied 17 Jun 2026 15:42 +0000
in reply to: https://benjojo.co.uk/u/benjojo/h/S5VRPXVV1qN4ffQN8F

@benjojo from a distance this looked like a winamp skin and for a brief second all seemed right with the world. Then I zoomed in.

benjojo replied 17 Jun 2026 15:44 +0000
in reply to: https://masto.hackers.town/users/kestral/statuses/116766209061330161

@kestral "the goggles do nothing"

ignaloidas@not.acu.l.. replied 18 Jun 2026 06:13 +0000
in reply to: https://benjojo.co.uk/u/benjojo/h/S5VRPXVV1qN4ffQN8F

@benjojo@benjojo.co.uk if you go look at boto3 you can find the files that the stuff is generated from (because of Python, you can just do the generation dynamically from the source)

ignaloidas@not.acu.l.. replied 18 Jun 2026 06:15 +0000
in reply to: https://not.acu.lt/notes/anmo49imbb

@benjojo@benjojo.co.uk ah, it lives at botocore but still https://github.com/boto/botocore/tree/develop/botocore/data

dottedmag@social.tre.. replied 18 Jun 2026 06:08 +0000
in reply to: https://benjojo.co.uk/u/benjojo/h/3V3zStV92c4Y9M35Tp

@benjojo Well, by being generated. The API surface of these providers is huge.

I have worked in a project that had to talk to several public clouds. After a while I threw away the SDKs, took several structs describing request/responses our project actually used, and made a in-project library out of these.