Skip to content

Commit

Permalink
chore!: move blob protos to app under it's own package (#2659)
Browse files Browse the repository at this point in the history
Ref: celestiaorg/celestia-app#2570

This PR creates a `pkg/blob` where the protos for `Blob` and `BlobTx` is
created. The intention is that this becomes it's own go.mod to avoid
needing to depend on `celestia-core` and our `cosmos-sdk` fork. Other
common structures like namespace should do the same so we can achieve
some cannonical representation of the structs between node and core/app

The reason for creating the types here instead of in `celestia-core` is
so that only `celestia-app` becomes a multi go.mod repo as opposed to
either. Additionally, `Blob` and `BlobTx` were never part of the
upstream CometBFT.

I also made the decision to have a single representation of blobs rather
than a proto and go native. The reason being is one of simplification
and performance. With large blobs, copying the data across to another
type can lead to a noticeable impact. Having a single struct however
does have the dowside of potential coversion errors from uint32 to uint8
so data needs to be validated before it can be used (this is also the
same as before).

This PR also ports over the proofs. I have put this all under a
common/v1 proto package.
  • Loading branch information
cmwaters authored Oct 16, 2023
1 parent 0824022 commit 2480ff3
Show file tree
Hide file tree
Showing 16 changed files with 922 additions and 95 deletions.
96 changes: 96 additions & 0 deletions pkg/blob/blob.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
package blob

import (
"bytes"
"errors"
fmt "fmt"
math "math"
"sort"

"github.com/celestiaorg/celestia-app/pkg/namespace"
)

// ProtoBlobTxTypeID is included in each encoded BlobTx to help prevent
// decoding binaries that are not actually BlobTxs.
const ProtoBlobTxTypeID = "BLOB"

// NewBlob creates a new coretypes.Blob from the provided data after performing
// basic stateless checks over it.
func New(ns namespace.Namespace, blob []byte, shareVersion uint8) *Blob {
return &Blob{
NamespaceId: ns.ID,
Data: blob,
ShareVersion: uint32(shareVersion),
NamespaceVersion: uint32(ns.Version),
}
}

// Namespace returns the namespace of the blob
func (b Blob) Namespace() namespace.Namespace {
return namespace.Namespace{
Version: uint8(b.NamespaceVersion),
ID: b.NamespaceId,
}
}

// Validate runs a stateless validity check on the form of the struct.
func (b *Blob) Validate() error {
if b == nil {
return errors.New("nil blob")
}
if len(b.NamespaceId) != namespace.NamespaceIDSize {
return fmt.Errorf("namespace id must be %d bytes", namespace.NamespaceIDSize)
}
if b.ShareVersion > math.MaxUint8 {
return errors.New("share version can not be greater than MaxShareVersion")
}
if b.NamespaceVersion > namespace.NamespaceVersionMax {
return errors.New("namespace version can not be greater than MaxNamespaceVersion")
}
if len(b.Data) == 0 {
return errors.New("blob data can not be empty")
}
return nil
}

// UnmarshalBlobTx attempts to unmarshal a transaction into blob transaction. If an
// error is thrown, false is returned.
func UnmarshalBlobTx(tx []byte) (bTx BlobTx, isBlob bool) {
err := bTx.Unmarshal(tx)
if err != nil {
return BlobTx{}, false
}
// perform some quick basic checks to prevent false positives
if bTx.TypeId != ProtoBlobTxTypeID {
return bTx, false
}
if len(bTx.Blobs) == 0 {
return bTx, false
}
for _, b := range bTx.Blobs {
if len(b.NamespaceId) != namespace.NamespaceIDSize {
return bTx, false
}
}
return bTx, true
}

// MarshalBlobTx creates a BlobTx using a normal transaction and some number of
// blobs.
//
// NOTE: Any checks on the blobs or the transaction must be performed in the
// application
func MarshalBlobTx(tx []byte, blobs ...*Blob) ([]byte, error) {
bTx := BlobTx{
Tx: tx,
Blobs: blobs,
TypeId: ProtoBlobTxTypeID,
}
return bTx.Marshal()
}

func Sort(blobs []*Blob) {
sort.SliceStable(blobs, func(i, j int) bool {
return bytes.Compare(blobs[i].Namespace().Bytes(), blobs[j].Namespace().Bytes()) < 0
})
}
Loading

0 comments on commit 2480ff3

Please sign in to comment.