benjojo.co.uk

Edent@mastodon.socia.. posted 10 Oct 2024 20:51 +0000

Trying to import a 983,537 row CSV into MySQL.

Very slow! To the post where I'm wondering how much of a pain it would be to convert it to a table and manually insert it.

benjojo replied 10 Oct 2024 20:57 +0000
in reply to: https://mastodon.social/users/Edent/statuses/113285104574956426

@Edent you doing inserts or LOADs?

Edent@mastodon.socia.. replied 10 Oct 2024 20:58 +0000
in reply to: https://benjojo.co.uk/u/benjojo/h/Yrvy94MGGJyrL4mQ6W

@benjojo literally just hitting "import" on phpMyAdmin.
So whatever the default is.
Guess I should have used LOADs?

benjojo replied 10 Oct 2024 21:02 +0000
in reply to: https://mastodon.social/users/Edent/statuses/113285131899560003

@Edent mmmm. I bet that's doing raw inserts. At bgptools when I need to import a lot of stuff at once I generate a CSV on the fly and provide it to the MySQL as a LOAD FILE, I tend to average 90 microseconds per row

Edent@mastodon.socia.. replied 10 Oct 2024 21:03 +0000
in reply to: https://benjojo.co.uk/u/benjojo/h/1L4FkTPpDH6qPWmh7M

@benjojo ah well! One to remember for next time.
Ta!

benjojo replied 10 Oct 2024 21:04 +0000
in reply to: https://mastodon.social/users/Edent/statuses/113285151188859035

@Edent I misread my own logs. I actually manage 5µs per row with LOAD FILE