go-pipe

Description

go-pipe is a simple library for piping data from iterators to writers. Pipe uses the following interfaces, which allows reading and writing of objects to any number of inputs and outputs.

Iterator

The Next method will return io.EOF once it has read all the objects from the input.

type Iterator interface {
	Next() (interface{}, error)
}

Writer

Writer is a writer that accepts input as individual objects.

type Writer interface {
	WriteObject(object interface{}) error
	Flush() error
}

BatchWriter

BatchWriter is a writer that accepts input in batches as an array or slice of objects.

type Writer interface {
	WriteObjects(objects interface{}) error
	Flush() error
}

go-pipe includes concrete structs for writing to channels, slice, functions, and maps. It is also used in railgun project along with go-simple-serializer to process objects from files.

Usage

Go

Install the package with:

go get -d github.com/spatialcurrent/go-pipe/...

You can import go-pipe as a library with:

import (
  "github.com/spatialcurrent/go-pipe/pkg/pipe"
)

See pipe in the docs for information on how to use Go API.

Examples

See the many examples in the docs or the tests.

Testing

Run test using make test or (bash scripts/test.sh), which runs unit tests, go vet, go vet with shadow, errcheck, staticcheck, and misspell.

Contributing

Spatial Current, Inc. is currently accepting pull requests for this repository. We'd love to have your contributions! Please see Contributing.md for how to get started.

License

This work is distributed under the MIT License. See LICENSE file.