You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
as part of the plans for accelerator pods (to support either search- or query-like functionality, more details anon), we need to allow ~/.datamon/datamon.yaml to ref an additional bucket. this bucket will be used to facet searches, journal writes, etc. by pipeline- or otherwise project-specific in-cloud accelerators.
for the scope of this task, we're essentially talking about adding journaling (in the sense of journalled filesystems) or Write Ahead Logging (in the sense of Postgres) to commands that write to existing (meta and blob) stores.
The text was updated successfully, but these errors were encountered:
ransomw1c
added
the
spike
Design, investigation tasks or tasks that does not have clear deliverable
label
Sep 8, 2020
these items do not cover this issue because this issue is envisioning additional golang to not just upload to a context (in the datamon sense, post WAL implementation, not the generalized golang sense, a context mostly characterized by a set of [AWS/gg] buckets) that contains a WAL but also to migrate data between two contexts according to some specialized rules. in general, this would effectively allow end-user functionalities like incremental uploads. i've added the "spike" label to indicate that the exact functionalities for transfer between contexts – much less mapping these 'machine level instructions' (i.e. the scope of the spike) to 'productizeable features' (e.g. incremental uploads).
at time of writing, i opine features such as progress bars and others that can be realized without multiple blob buckets (or multiple contexts, whichever requirement rings truer – they are used interchangeably here) as described #443 are merely papering over the engineering work, something like very simple GC, essentially, described in this issue.
as part of the plans for accelerator pods (to support either search- or query-like functionality, more details anon), we need to allow
~/.datamon/datamon.yaml
to ref an additional bucket. this bucket will be used to facet searches, journal writes, etc. by pipeline- or otherwise project-specific in-cloud accelerators.for the scope of this task, we're essentially talking about adding journaling (in the sense of journalled filesystems) or Write Ahead Logging (in the sense of Postgres) to commands that write to existing (meta and blob) stores.
The text was updated successfully, but these errors were encountered: