Attributes got lost!

[odhiambo]

I am running listmonk v4.1.0.

I created a CSV of users (thousands of them!) with attributes. When I imported them as subscribers, the attributes kind of got lost! Here is a same from my CSV:

email,name,attributes
[email protected],Johannes Doe ,"{""phone"": ""+254700999999"", ""gender"": ""Male"", ""county"": ""Mandera"", ""sub_county"": ""Banissa"", ""cluster"": ""Cluster 6"", ""cohort"": ""Cohort  1"", ""label"": ""a""}"

When I view the subscriber, the attributes box is empty!

What might have gone wrong?

[cbertelli]

I tried uploading a csv with

email,name,attributes
[email protected],Johannes Doe , {"phone": "+254700999999", "gender": "Male", "county": "Mandera", "sub_county": "Banissa", "cluster": "Cluster 6", "cohort": "Cohort  1", "label": "a"}

and it worked. I think the hint in the interface is wrong because my plain upload successfully filled the required JSON.

[odhiambo]

When I try your way, the import fails with:

2024/12/20 13:07:05processing 'johndoe.csv'
2024/12/20 13:07:05error reading CSV 'parse error on line 2, column 35: bare " in non-quoted-field'

[cbertelli]

You are right! Sorry for this wrong suggestion. I got it from a database query and I didn't notice the double quotes during export.
The CSV file I used corresponds indeed to the tip.

email,name,attributes
[email protected],Pippo Cancellami,"{""phone"":""+254700999999"",""gender"":""Male"",""county"":""Mandera"",""sub_county"":""Banissa"",""cluster"":""Cluster 6"",""cohort"":""Cohort 1"",""label"":""a""}"

I quoted "Pippo Cancellami" and it found an unquoted string on column 23. It's very similar to yours.

[odhiambo]

So, what am I missing? Or not doing right?

[cbertelli]

I did almost the same thing as you. CSV is a very simple beast, but a wild and crazy one.
I took off quotes outside the JSON part, then I deleted all spaces before and after commas and colons.

[odhiambo]

I copied your format as-is and now it imported successfully. So these spaces inside the json are the issue it seems.

[odhiambo]

So, finally, do we call this a bug in listmonk or just a problem with JSON?
Here is a script I use to prepare a CSV file from Excel. It produces the same JSOn format that Listmonk does not like. I actually had to do a substation to replace the spaces: s/ ""/""/g - and after that Listmonk imports the contacts as expected.

Thank you for your time in helping me to figure out this.

PS: Do we report a bug with Listmonk JSON processor?

[cbertelli]

I've seen that Listmonk properly uses a JSONB column to handle the JSON and not a JSON-inside-a-text. I think there is no need to duplicate JSON processing besides (or against) what PostgreSQL already does, but this is indeed a slippery path. I think there are dedicated JSON processors (e. g. jq) that do something better than sed or perl, but I would do the same you did.