Handle bulk inserts

[michaelgmiller]

Is it possible to setup a DAO to do bulk insert queries a la COPY FROM ?

[jonbodner]

Hrm, I'm assuming that you want to pass in a slice of slices or a slice of structs with the data. No, you can't walk a slice of slices or structs right now, but I have some thoughts on how to approach this issue. And I'm always happy to take submissions :-)

Let me think about this some more and see about adding in support for this functionality.

[michaelgmiller]

Yup, exactly, I'd like to be able to pass in a slice of structs! If it were Uber performing using binary COPY FROM, that'd be even better :). Don't have a ton of time, but if you have a clear idea about how to implement this, I can try to help.

[jonbodner]

Sorry for the delay in responding. My thought is to introduce the ability to specify a function that can transform the value of a placeholder. These could process the value either before it is converted to a prepared statement (which would allow dynamic SQL to be injected), or after it is converted (which would allow a value to be changed just before it is inserted).

My concern is that the pre-processing function is a bit of a footgun; if you inject the wrong SQL, you can set yourself up for injection, which Proteus is designed to avoid. I'm trying to figure out if there is a way to make it safe. If there isn't, I would fall back to supporting some built-in functions for building things like lists of structs properly, but they would be added on an as-requested basis.

[katherinewallace]

+1 this would be very useful

[jonbodner]

I will have some time to work on this in a couple of weeks. Is that ok? I’ll see what I can do then.

[katherinewallace]

I see this issue is still open. Is there any update on this?

[jonbodner]

Sorry, no change in status right now. I don't have time to make Proteus updates right now, but I'm hoping to get back to it in the near future. Do you have any thoughts on how you would like the feature to work? I'm still thinking of what a good implementation would be, especially given Go's lack of batch updates in the standard sql support.

[katherinewallace]

Let me take a look, and I'll let you know if I have ideas. I've been using Proteus a good amount, and would definitely find this very valuable.