UTF-16 surrogate pairs #8
Conversation
Characters outside the Basic Multilingual Plane have until now been (mistakenly) parsed into characters with invalid code points.
Not sure why this hasn't been discovered before, it wouldn't have worked before. Also refactored the code slightly given large chunks of repeated code.
Also added an option to turn that off if in cases where it'll be simpler.
|
Hi. I'm not really maintaining this library anymore. Would you be interested in taking over maintenance? |
|
You know what, I looked at the known forks on github, and @jl2 seems to be furthest ahead (with more merges compared to the other forks too) https://github.com/jl2/ST-JSON Could I suggest handing maintainership over to jl2 instead? I'm more than happy to rebase my commits to that branch instead. |
|
If @jl2 is interested in maintaining this, I'm absolutely okay with that |
|
Hi, sorry I missed this earlier, but I'd be happy to maintain this. I think I have at least one backwards incompatible change in my fork (getjso* as a function instead of macro), but I can move that to a separate function to maintain compatibility. I'll take a look after work today to make sure I've got the latest code in my fork. |
|
Great! I've added you as a collaborator here, but if you want we can also make the readme point at your repository instead. |
|
Thank you! I think it's easiest to keep everything pointed here for now. And I've never been a collaborator on a GitHub project, so I'm curious to see how it works. |
|
Yay! Thanks to Marijn for creating this project, and jl2 for taking over its maintainance! |
Enclosed is my code to support JSON UTF-16 surrogate pairs.
Please review and suggest any appropriate improvements.
Credit goes to Jason Miller & Hans Hübner of yason, where I got the elegant and simple translation logic to convert UTF-16 to code points.
While I have also added writer as well (the original code would still work but may output non-Ascii characters - this is still possible with the output-literal-unicode option).
I am sure many JSON libraries (notably cl-json hankhero/cl-json#11) continue to suffer from this bug/missing feature, though this feature seems to be used mainly for representing emojis today!
PS: My tests are current embedded as comments within the source file, one day, I assume they can move into properly structured unit tests. It might be a good idea to share certain test samples across all JSON libraries too?