Skip to content

Commit

Permalink
Add LazyStreamChunkWriter
Browse files Browse the repository at this point in the history
  • Loading branch information
alexandre-daubois committed Apr 4, 2024
1 parent 2898efd commit b19fd3b
Show file tree
Hide file tree
Showing 5 changed files with 227 additions and 2 deletions.
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,6 @@
.phpunit.cache
/phpunit.xml

.php-cs-fixer.cache

composer.lock
24 changes: 23 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
[![Latest Version](http://poser.pugx.org/alexandre-daubois/lazy-stream/v/stable)](https://packagist.org/packages/alexandre-daubois/lazy-stream)
[![License](http://poser.pugx.org/alexandre-daubois/lazy-stream/license)](https://packagist.org/packages/alexandre-daubois/lazy-stream)

LazyStream is a library that provides a convenient way to write lazily to streams using generators. It allows you to write data incrementally to a stream, reducing memory usage and improving performance when dealing with large amounts of data.
LazyStream is a **pure PHP**, **zero-dependencies** library that provides a convenient way to write lazily to streams using generators. It allows you to write data incrementally to a stream, reducing memory usage and improving performance when dealing with large amounts of data.

## Features

Expand Down Expand Up @@ -108,6 +108,28 @@ $stream = new MultiLazyStreamWriter([
$stream->trigger();
```

## The `LazyStreamChunkWriter` class

The `LazyStreamChunkWriter` class is a specialized class that allows you to write data to a stream in chunks. The mechanism is pretty different from other writers. Instead of writing data from a generator, you can write data by calling the `send()` method. This allows to write data in a more controlled and "natural" way without having to worry about generators and iterators.

**The LazyStreamChunkWriter ALWAYS append data to the stream, thus the opening mode can not be controlled.** This is done to ensure a proper autoclosing behavior. If you need to write data in an empty stream, you should use the `LazyStreamWriter` class or ensure your stream is empty before sending data.

Here is an example of how to use the `LazyStreamChunkWriter` class:

```php
use LazyStream\LazyStreamChunkWriter;

$stream = new LazyStreamChunkWriter('https://user:[email protected]/my-file.json');

$data = /** fetch data from somewhere */;
$stream->send($data);

// normal flow of the application

$data = /** fetch data from somewhere else */;
$stream->send($data);
```

## Reading lazily a stream with `LazyStreamReader`

Files are already read lazily by default: when you call `fread()`, you only fetch the number of bytes you asked for, not more.
Expand Down
3 changes: 2 additions & 1 deletion composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@
"test": "./vendor/bin/phpunit"
},
"require-dev": {
"phpunit/phpunit": "^10.5.10"
"friendsofphp/php-cs-fixer": "^3.52",
"phpunit/phpunit": "^10.5.16"
}
}
86 changes: 86 additions & 0 deletions src/LazyStreamChunkWriter.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
<?php

/*
* (c) Alexandre Daubois <[email protected]>
*
* For the full copyright and license information, please view the LICENSE
* file that was distributed with this source code.
*/

namespace LazyStream;

use LazyStream\Exception\LazyStreamOpenException;
use LazyStream\Exception\LazyStreamWriterTriggerException;

/**
* Implementation of the LazyStreamWriterInterface that allows writing data
* to a stream by sending data when needed. The advantage of this class is that
* data can be sent to the stream in a more "natural" order, where
* {@see LazyStreamWriter} requires to provide a data provider iterator.
*/
class LazyStreamChunkWriter extends LazyStreamWriter
{
private \Generator $dataProvider;
protected string $openingMode = 'a';

/**
* @param string $uri A valid stream URI.
* @param bool $autoClose Whether the stream should be closed once the `trigger` method is done.
*/
public function __construct(
protected string $uri,
private bool $autoClose = false,
) {
// no parent constructor call because we don't want to provide
// an iterator as data provider

$this->dataProvider = (function (): \Generator {
while (true) {
$data = yield;

if (null === $this->handle) {
$this->openStream();
}

if (false === \fwrite($this->handle, $data)) {
throw new LazyStreamWriterTriggerException(sprintf('Unable to write to stream with URI "%s".', $this->metadata['uri']));
}

if ($this->autoClose) {
$this->closeStream();
}
}
})();
}

/**
* Sends data to the stream. If the stream is not open, it will be opened.
*/
public function send(mixed $data): void
{
try {
$this->dataProvider->send($data);
} catch (LazyStreamOpenException $lazyStreamOpenException) {
throw $lazyStreamOpenException;
} catch (\Throwable $throwable) {
throw new LazyStreamWriterTriggerException(previous: $throwable);
} finally {
if ($this->autoClose) {
$this->closeStream();
}
}
}

/**
* This method is not allowed to be called on this class. Use {@see self::send()} instead.
*/
public function trigger(): never
{
throw new \LogicException(sprintf('You must provide data to write to the stream by calling "%s::send()".', __CLASS__));
}

public function equals(LazyStreamWriter $other): bool
{
return $this->uri === $other->uri;
}
}
114 changes: 114 additions & 0 deletions tests/LazyStreamChunkWriterTest.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
<?php

/*
* (c) Alexandre Daubois <[email protected]>
*
* For the full copyright and license information, please view the LICENSE
* file that was distributed with this source code.
*/


use LazyStream\Exception\LazyStreamOpenException;
use LazyStream\Exception\LazyStreamWriterTriggerException;
use LazyStream\LazyStreamChunkWriter;
use LazyStream\LazyStreamWriter;
use PHPUnit\Framework\Attributes\CoversClass;
use PHPUnit\Framework\TestCase;

#[CoversClass(LazyStreamChunkWriter::class)]
class LazyStreamChunkWriterTest extends TestCase
{
public function testEqualsDifferentStreams(): void
{
$lazyStream = new LazyStreamChunkWriter('php://memory');
$other = new LazyStreamChunkWriter('php://input');

$this->assertFalse($lazyStream->equals($other));
}

public function testEqualsSameUri(): void
{
$lazyStream = new LazyStreamChunkWriter('php://memory');
$other = new LazyStreamChunkWriter('php://memory');

$this->assertTrue($lazyStream->equals($other));
}

public function testStreamIsLazilyOpened(): void
{
$lazyStream = new LazyStreamChunkWriter('php://memory');

$this->assertNull($lazyStream->getStreamHandle());
}

public function testTriggerStreamsThrows(): void
{
$lazyStream = new LazyStreamChunkWriter('php://memory');

$this->expectException(LogicException::class);
$this->expectExceptionMessage('You must provide data to write to the stream by calling "LazyStream\LazyStreamChunkWriter::send()".');
$lazyStream->trigger();
}

public function testInvalidStreamThrowsAtSend(): void
{
$lazyStream = new LazyStreamChunkWriter('php://invalid');

$this->expectException(LazyStreamOpenException::class);
$this->expectExceptionMessage('Unable to open "php://invalid" with mode "a".');
$lazyStream->send('test');
}

public function testGetType(): void
{
$lazyStream = new LazyStreamChunkWriter('php://memory');

$this->assertSame('MEMORY', $lazyStream->getMetadata()['stream_type']);
$this->assertNull($lazyStream->getStreamHandle());
}

public function testGetTypeOnTriggeredStreamWithAutoclose(): void
{
$lazyStream = new LazyStreamChunkWriter('php://memory', autoClose: true);

$this->assertSame('MEMORY', $lazyStream->getMetadata()['stream_type']);
$this->assertNull($lazyStream->getStreamHandle());
}

public function testGetTypeOnTriggeredStreamWithoutAutoclose(): void
{
$lazyStream = new LazyStreamChunkWriter('php://memory');

$lazyStream->send('test');
$this->assertNotNull($lazyStream->getStreamHandle());

$this->assertSame('MEMORY', $lazyStream->getMetadata()['stream_type']);
$this->assertNotNull($lazyStream->getStreamHandle());
}

public function testSend(): void
{
$lazyStream = new LazyStreamChunkWriter(__DIR__.\DIRECTORY_SEPARATOR.__METHOD__);

try {
$lazyStream->send('test');
$lazyStream->send('test2');
$this->assertSame('testtest2', file_get_contents(__DIR__.\DIRECTORY_SEPARATOR.__METHOD__));
} finally {
\unlink(__DIR__.\DIRECTORY_SEPARATOR.__METHOD__);
}
}

public function testSendWithAutoclose(): void
{
$lazyStream = new LazyStreamChunkWriter(__DIR__.\DIRECTORY_SEPARATOR.__METHOD__, true);

try {
$lazyStream->send('test');
$lazyStream->send('test2');
$this->assertSame('testtest2', file_get_contents(__DIR__.\DIRECTORY_SEPARATOR.__METHOD__));
} finally {
\unlink(__DIR__.\DIRECTORY_SEPARATOR.__METHOD__);
}
}
}

0 comments on commit b19fd3b

Please sign in to comment.