Skip to content

Commit

Permalink
Add JsonArtifact
Browse files Browse the repository at this point in the history
  • Loading branch information
vachillo committed Jul 3, 2024
1 parent 83a16ac commit c275510
Show file tree
Hide file tree
Showing 6 changed files with 93 additions and 28 deletions.
48 changes: 22 additions & 26 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,32 +5,6 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## Unreleased
### Added
- `Message` for storing messages in a `PromptStack`. Messages consist of a role, content, and usage.
- `DeltaMessage` for storing partial messages in a `PromptStack`. Multiple `DeltaMessage` can be combined to form a `Message`.
- `TextMessageContent` for storing textual content in a `Message`.
- `ImageMessageContent` for storing image content in a `Message`.
- Support for adding `TextArtifact`s, `ImageArtifact`s, and `ListArtifact`s to `PromptStack`.
- Support for image inputs to `OpenAiChatPromptDriver`, `AzureOpenAiChatPromptDriver`, `AmazonBedrockPromptDriver`, `AnthropicPromptDriver`, and `GooglePromptDriver`.
- Input/output token usage metrics to all Prompt Drivers.
- `FinishPromptEvent.input_token_count` and `FinishPromptEvent.output_token_count`.
- Support for storing Artifacts as inputs/outputs in Conversation Memory Runs.
- `Agent.input` for passing Artifacts as input.
- Support for `PromptTask`s to take `TextArtifact`s, `ImageArtifact`s, and `ListArtifact`s as input.

### Changed
- **BREAKING**: Moved/renamed `griptape.utils.PromptStack` to `griptape.common.PromptStack`.
- **BREAKING**: Renamed `PromptStack.inputs` to `PromptStack.messages`.
- **BREAKING**: Moved `PromptStack.USER_ROLE`, `PromptStack.ASSISTANT_ROLE`, and `PromptStack.SYSTEM_ROLE` to `Message`.
- **BREAKING**: Updated return type of `PromptDriver.try_run` from `TextArtifact` to `Message`.
- **BREAKING**: Updated return type of `PromptDriver.try_stream` from `Iterator[TextArtifact]` to `Iterator[DeltaMessage]`.
- **BREAKING**: Removed `BasePromptEvent.token_count` in favor of `FinishPromptEvent.input_token_count` and `FinishPromptEvent.output_token_count`.
- **BREAKING**: Removed `StartPromptEvent.prompt`. Use `StartPromptEvent.prompt_stack` instead.
- **BREAKING**: Removed `Agent.input_template` in favor of `Agent.input`.
- **BREAKING**: `BasePromptDriver.run` now returns a `Message` instead of a `TextArtifact`. For compatibility, `Message.value` contains the Message's Artifact value
- Default Prompt Driver model in `GoogleStructureConfig` to `gemini-1.5-pro`.



### Added
- `RagEngine` is an abstraction for implementing modular RAG pipelines.
Expand Down Expand Up @@ -67,6 +41,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Parameter `fail_fast` to `Structure`.
- `BooleanArtifact` for handling boolean values.
- `typos` to dev dependencies to catch typos in code/docs.
- `Message` for storing messages in a `PromptStack`. Messages consist of a role, content, and usage.
- `DeltaMessage` for storing partial messages in a `PromptStack`. Multiple `DeltaMessage` can be combined to form a `Message`.
- `TextMessageContent` for storing textual content in a `Message`.
- `ImageMessageContent` for storing image content in a `Message`.
- Support for adding `TextArtifact`s, `ImageArtifact`s, and `ListArtifact`s to `PromptStack`.
- Support for image inputs to `OpenAiChatPromptDriver`, `AzureOpenAiChatPromptDriver`, `AmazonBedrockPromptDriver`, `AnthropicPromptDriver`, and `GooglePromptDriver`.
- Input/output token usage metrics to all Prompt Drivers.
- `FinishPromptEvent.input_token_count` and `FinishPromptEvent.output_token_count`.
- Support for storing Artifacts as inputs/outputs in Conversation Memory Runs.
- `Agent.input` for passing Artifacts as input.
- Support for `PromptTask`s to take `TextArtifact`s, `ImageArtifact`s, and `ListArtifact`s as input.
- `JsonArtifact` for handling de/seralization of values.

### Changed
- **BREAKING**: `BaseVectorStoreDriver.upsert_text_artifact()` and `BaseVectorStoreDriver.upsert_text()` use artifact/string values to generate `vector_id` if it wasn't implicitly passed. This change ensures that we don't generate embeddings for the same content every time.
Expand All @@ -80,6 +66,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- **BREAKING**: removed `VectorStoreClient.top_n` and `VectorStoreClient.namespace` in favor of `VectorStoreClient.query_params`.
- `GriptapeCloudKnowledgeBaseClient` migrated to `/search` api.
- **BREAKING**: All `futures_executor` fields renamed to `futures_executor_fn` and now accept callables instead of futures; wrapped all future `submit` calls with the `with` block to address future executor shutdown issues.
- **BREAKING**: Moved/renamed `griptape.utils.PromptStack` to `griptape.common.PromptStack`.
- **BREAKING**: Renamed `PromptStack.inputs` to `PromptStack.messages`.
- **BREAKING**: Moved `PromptStack.USER_ROLE`, `PromptStack.ASSISTANT_ROLE`, and `PromptStack.SYSTEM_ROLE` to `Message`.
- **BREAKING**: Updated return type of `PromptDriver.try_run` from `TextArtifact` to `Message`.
- **BREAKING**: Updated return type of `PromptDriver.try_stream` from `Iterator[TextArtifact]` to `Iterator[DeltaMessage]`.
- **BREAKING**: Removed `BasePromptEvent.token_count` in favor of `FinishPromptEvent.input_token_count` and `FinishPromptEvent.output_token_count`.
- **BREAKING**: Removed `StartPromptEvent.prompt`. Use `StartPromptEvent.prompt_stack` instead.
- **BREAKING**: Removed `Agent.input_template` in favor of `Agent.input`.
- **BREAKING**: `BasePromptDriver.run` now returns a `Message` instead of a `TextArtifact`. For compatibility, `Message.value` contains the Message's Artifact value
- Default Prompt Driver model in `GoogleStructureConfig` to `gemini-1.5-pro`.

### Fixed
- `CoherePromptDriver` to properly handle empty history.
Expand Down
4 changes: 4 additions & 0 deletions docs/griptape-framework/data/artifacts.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,3 +47,7 @@ A [BooleanArtifact](../../reference/griptape/artifacts/boolean_artifact.md) is u

!!! info
Any object passed on init to `BooleanArtifact` will be coerced into a `bool` type. This might lead to unintended behavior: `BooleanArtifact("False").value is True`. Use [BooleanArtifact.parse_bool](../../reference/griptape/artifacts/boolean_artifact.md#griptape.artifacts.boolean_artifact.BooleanArtifact.parse_bool) to convert case-insensitive string literal values `"True"` and `"False"` into a `BooleanArtifact`: `BooleanArtifact.parse_bool("False").value is False`.

## JsonArtifact

A [JsonArtifact](../../reference/griptape/artifacts/json_artifact.md) is used for passing JSON-serliazable data around the framework. Any object passed to `value` will be converted using `json.dumps`, and any time `value` is retrieved, it will call `json.loads`.
2 changes: 2 additions & 0 deletions griptape/artifacts/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
from .error_artifact import ErrorArtifact
from .info_artifact import InfoArtifact
from .text_artifact import TextArtifact
from .json_artifact import JsonArtifact
from .blob_artifact import BlobArtifact
from .boolean_artifact import BooleanArtifact
from .csv_row_artifact import CsvRowArtifact
Expand All @@ -16,6 +17,7 @@
"ErrorArtifact",
"InfoArtifact",
"TextArtifact",
"JsonArtifact",
"BlobArtifact",
"BooleanArtifact",
"CsvRowArtifact",
Expand Down
29 changes: 29 additions & 0 deletions griptape/artifacts/json_artifact.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
from __future__ import annotations
from typing import TYPE_CHECKING, Any
import json
from attrs import define, field
from griptape.artifacts import TextArtifact

if TYPE_CHECKING:
from griptape.artifacts import BaseArtifact


@define
class JsonArtifact(TextArtifact):
value: Any = field(converter=json.dumps, metadata={"serializable": True}) # pyright: ignore reportRedeclaration

@property
def value(self) -> Any:
return json.loads(self.value)

Check warning on line 17 in griptape/artifacts/json_artifact.py

View check run for this annotation

Codecov / codecov/patch

griptape/artifacts/json_artifact.py#L17

Added line #L17 was not covered by tests

def to_text(self) -> str:
return json.dumps(self.value)

def __add__(self, other: BaseArtifact) -> JsonArtifact:
new = json.loads(other.to_text())
return JsonArtifact({**self.value, **new})

def __eq__(self, value: object) -> bool:
if isinstance(value, str):
return self.to_text() == value
return self.value == value
4 changes: 2 additions & 2 deletions griptape/artifacts/text_artifact.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,12 +28,12 @@ def __bool__(self) -> bool:

def generate_embedding(self, driver: BaseEmbeddingDriver) -> Optional[list[float]]:
self._embedding.clear()
self._embedding.extend(driver.embed_string(str(self.value)))
self._embedding.extend(driver.embed_string(self.to_text()))

return self.embedding

def token_count(self, tokenizer: BaseTokenizer) -> int:
return tokenizer.count_tokens(str(self.value))

def to_bytes(self) -> bytes:
return self.value.encode(encoding=self.encoding, errors=self.encoding_error_handler)
return self.to_text().encode(encoding=self.encoding, errors=self.encoding_error_handler)
34 changes: 34 additions & 0 deletions tests/unit/artifacts/test_json_artifact.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
import json
import pytest
from griptape.artifacts import JsonArtifact, TextArtifact


class TestJsonArtifact:
def test_value_type_conversion(self):
assert JsonArtifact({"foo": "bar"}).value == json.loads(json.dumps({"foo": "bar"}))
string = json.dumps({"foo": {}})
print(string)
string = json.loads(string)
print(string)

def test___add__(self):
assert JsonArtifact({"foo": "bar"}) + JsonArtifact({"value": "baz"}) == JsonArtifact(
{"foo": "bar", "value": "baz"}
)

new = JsonArtifact({"foo": "bar"}) + JsonArtifact({"value": "baz"})
assert new == {"foo": "bar", "value": "baz"}

assert JsonArtifact({"foo": "bar"}) + JsonArtifact({"foo": "baz"}) == JsonArtifact({"foo": "baz"})
with pytest.raises(json.JSONDecodeError):
JsonArtifact({"foo": "bar"}) + TextArtifact("invalid json")

def test___eq__(self):
assert JsonArtifact({"foo": "bar"}) == json.dumps({"foo": "bar"})
assert JsonArtifact({"foo": "bar"}) == {"foo": "bar"}

def test_to_text(self):
assert JsonArtifact({"foo": "bar"}).to_text() == json.dumps({"foo": "bar"})

def test_to_bytes_encoding(self):
assert JsonArtifact({"foo": "bar"}).to_bytes() == json.dumps({"foo": "bar"}).encode()

0 comments on commit c275510

Please sign in to comment.