Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ion Reader expects wrong character at end of list of structs for text files #126

Open
biolore opened this issue Sep 1, 2020 · 1 comment

Comments

@biolore
Copy link

biolore commented Sep 1, 2020

When reading a list of structs from a text file/string, such as [{number:0},{number:1},{number:2}], the following exception is thrown at the end of the list:
System.FormatException: Illegal character: expected '}' character but encountered ']'

Below is a stub program which writes and then reads a list of simple structs in both a text and binary file formats, adapted from the cookbook's reading and writing ion data section.

using System;
using System.IO;
using Amazon.IonDotnet;
using Amazon.IonDotnet.Builders;

namespace IonScratchpad
{
    class Program
    {
        private const string TextFileName = "structList.txt.ion";
        private const string BinaryFileName = "structList.bin.ion";

        static void Main(string[] args)
        {
            // Write Binary
            WriteBinaryFile();

            // Read Binary
            ReadFile(BinaryFileName);

            // Write Text
            WriteTextFile();

            // Read Text
            ReadFile(TextFileName);
        }

        private static void ReadFile(string fileName)
        {
            try
            {
                Console.WriteLine($"Reading {fileName} with {nameof(IonReaderBuilder)}");
                using Stream fs = File.OpenRead(fileName);
                using IIonReader reader = IonReaderBuilder.Build(fs);
                ReadListOfStruct(reader);
                Console.WriteLine("Done reading binary file");
            }
            catch (Exception e)
            {
                Console.WriteLine($"Hit exception while reading {fileName}: {e.Message}");
            }
        }

        private static void ReadListOfStruct(IIonReader reader)
        {
            reader.MoveNext(); // position the reader at the first value, a struct
            reader.StepIn(); // step into the list
            while (reader.MoveNext() != IonType.None)
            {
                reader.StepIn(); // step into the struct
                reader.MoveNext(); // position the reader at the first value in the struct
                string fieldName = reader.CurrentFieldName; // retrieve the current value's field name
                int value = reader.IntValue(); // retrieve the current value's string value
                reader.StepOut(); // step out of the struct
                Console.WriteLine(fieldName + ": " + value);
            }

            reader.StepOut(); // step out of the list
        }

        private static void WriteBinaryFile()
        {
            try
            {
                Console.WriteLine($"Writing {BinaryFileName} with {nameof(IonBinaryWriterBuilder)}");
                using Stream fs = File.Create(BinaryFileName);
                using IIonWriter binaryWriter = IonBinaryWriterBuilder.Build(fs);
                WriteListOfStruct(binaryWriter);
                Console.WriteLine("Done writing binary file");
            }
            catch (Exception e)
            {
                Console.WriteLine($"Hit exception while writing {BinaryFileName}: {e.Message}");
            }
        }

        private static void WriteTextFile()
        {
            try
            {
                Console.WriteLine($"Writing {TextFileName} with {nameof(IonTextWriterBuilder)}");
                using TextWriter tw = new StreamWriter(TextFileName);
                using IIonWriter textWriter = IonTextWriterBuilder.Build(tw);
                WriteListOfStruct(textWriter);
                Console.WriteLine("Done writing text file");
            }
            catch (Exception e)
            {
                Console.WriteLine($"Hit exception while writing {TextFileName}: {e.Message}");
            }
        }

        private static void WriteListOfStruct(IIonWriter writer)
        {
            writer.StepIn(IonType.List);
            for (int i = 0; i < 3; i++)
            {
                writer.StepIn(IonType.Struct); // step into a struct
                writer.SetFieldName("number"); // set the field name for the next value to be written
                writer.WriteInt(i); // write the next value
                writer.StepOut(); // step out of the struct
            }

            writer.StepOut();

            writer.Finish();
        }
    }
}

Output of this stub is as follows:

Writing structList.bin.ion with IonBinaryWriterBuilder
Done writing binary file
Reading structList.bin.ion with IonReaderBuilder
number: 0
number: 1
number: 2
Done reading binary file
Writing structList.txt.ion with IonTextWriterBuilder
Done writing text file
Reading structList.txt.ion with IonReaderBuilder
number: 0
number: 1
number: 2
Hit exception while reading structList.txt.ion: Illegal character: expected '}' character but encountered ']'

And lastly the files generated by the stub are in this zip file: filesMadeByStub.zip

@CoderNate
Copy link
Contributor

This doesn't help explain why the text vs binary versions have different behavior but, as a workaround, after nt value = reader.IntValue(); you can call reader.MoveNext() (which returns IonType.None) before stepping out of the struct.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants