Skip to content

Commit

Permalink
Updating docs to include transform plugins
Browse files Browse the repository at this point in the history
  • Loading branch information
@kellrott
kellrott committed Feb 20, 2024
1 parent c86673b commit c77cbeb
Show file tree
Hide file tree
Showing 41 changed files with 603 additions and 185 deletions.
17 changes: 13 additions & 4 deletions docs/docs/example/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -99,18 +99,18 @@
<li>
<ul class="sidebar-nav sidebar-nav-nested">
<li>
<a href="/sifter/docs/inputs/jsonload/"
<a href="/sifter/docs/inputs/plugin/"
class="sidebar-nav-item "
>jsonLoad</a></li>
>input plugin</a></li>
</ul>
</li>

<li>
<ul class="sidebar-nav sidebar-nav-nested">
<li>
<a href="/sifter/docs/inputs/plugin/"
<a href="/sifter/docs/inputs/jsonload/"
class="sidebar-nav-item "
>plugin</a></li>
>jsonLoad</a></li>
</ul>
</li>

Expand Down Expand Up @@ -346,6 +346,15 @@
</ul>
</li>

<li>
<ul class="sidebar-nav sidebar-nav-nested">
<li>
<a href="/sifter/docs/transforms/plugin/"
class="sidebar-nav-item "
>transform plugin</a></li>
</ul>
</li>

<li>
<ul class="sidebar-nav sidebar-nav-nested">
<li>
Expand Down
17 changes: 13 additions & 4 deletions docs/docs/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -99,18 +99,18 @@
<li>
<ul class="sidebar-nav sidebar-nav-nested">
<li>
<a href="/sifter/docs/inputs/jsonload/"
<a href="/sifter/docs/inputs/plugin/"
class="sidebar-nav-item "
>jsonLoad</a></li>
>input plugin</a></li>
</ul>
</li>

<li>
<ul class="sidebar-nav sidebar-nav-nested">
<li>
<a href="/sifter/docs/inputs/plugin/"
<a href="/sifter/docs/inputs/jsonload/"
class="sidebar-nav-item "
>plugin</a></li>
>jsonLoad</a></li>
</ul>
</li>

Expand Down Expand Up @@ -346,6 +346,15 @@
</ul>
</li>

<li>
<ul class="sidebar-nav sidebar-nav-nested">
<li>
<a href="/sifter/docs/transforms/plugin/"
class="sidebar-nav-item "
>transform plugin</a></li>
</ul>
</li>

<li>
<ul class="sidebar-nav sidebar-nav-nested">
<li>
Expand Down
28 changes: 14 additions & 14 deletions docs/docs/index.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,13 @@
<guid>https://bmeg.github.io/sifter/docs/transforms/hash/</guid>
<description>hash Parameters name Type Description field string Field to store hash value value string Templated string of value to be hashed method string Hashing method: sha1/sha256/md5 example - hash: value: &amp;#34;{{row.contents}}&amp;#34; field: contents-sha1 method: sha1 </description>
</item>
<item>
<title>input plugin</title>
<link>https://bmeg.github.io/sifter/docs/inputs/plugin/</link>
<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
<guid>https://bmeg.github.io/sifter/docs/inputs/plugin/</guid>
<description>plugin Run user program for customized data extraction.&#xA;Example inputs: oboData: plugin: commandLine: ../../util/obo_reader.py {{config.oboFile}} The plugin program is expected to output JSON messages, one per line, to STDOUT that will then be passed to the transform pipelines.&#xA;Example Plugin The obo_reader.py plugin, it reads a OBO file, such as the kind the describe the GeneOntology, and emits the records as single line JSON messages.&#xA;#!/usr/bin/env python import re import sys import json re_section = re.</description>
</item>
<item>
<title>Inputs</title>
<link>https://bmeg.github.io/sifter/docs/inputs/</link>
Expand Down Expand Up @@ -168,20 +175,6 @@
<guid>https://bmeg.github.io/sifter/docs/transforms/</guid>
<description>Transforms alter the data</description>
</item>
<item>
<title>plugin</title>
<link>https://bmeg.github.io/sifter/docs/inputs/plugin/</link>
<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
<guid>https://bmeg.github.io/sifter/docs/inputs/plugin/</guid>
<description>plugin Run user program for customized data extraction.&#xA;Example inputs: oboData: plugin: commandLine: ../../util/obo_reader.py {{config.oboFile}} The plugin program is expected to output JSON messages, one per line, to STDOUT that will then be passed to the transform pipelines.&#xA;Example Plugin The obo_reader.py plugin, it reads a OBO file, such as the kind the describe the GeneOntology, and emits the records as single line JSON messages.&#xA;#!/usr/bin/env python import re import sys import json re_section = re.</description>
</item>
<item>
<title>plugin</title>
<link>https://bmeg.github.io/sifter/docs/transforms/plugin/</link>
<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
<guid>https://bmeg.github.io/sifter/docs/transforms/plugin/</guid>
<description></description>
</item>
<item>
<title>project</title>
<link>https://bmeg.github.io/sifter/docs/transforms/project/</link>
Expand Down Expand Up @@ -238,6 +231,13 @@
<guid>https://bmeg.github.io/sifter/docs/transforms/tablewrite/</guid>
<description></description>
</item>
<item>
<title>transform plugin</title>
<link>https://bmeg.github.io/sifter/docs/transforms/plugin/</link>
<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
<guid>https://bmeg.github.io/sifter/docs/transforms/plugin/</guid>
<description>plugin Invoke external program for data processing&#xA;Parameters name Description commandLine Command line program to be called The command line can be written in any language. Sifter and the plugin communicate via NDJSON. Sifter streams the input to the program via STDIN and the plugin returns results via STDOUT. Any loggin or additional data must be sent to STDERR, or it will interupt the stream of messages. The command line code is executed using the base directory of the sifter file as the working directory.</description>
</item>
<item>
<title>uuid</title>
<link>https://bmeg.github.io/sifter/docs/transforms/uuid/</link>
Expand Down
17 changes: 13 additions & 4 deletions docs/docs/inputs/avroload/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -99,18 +99,18 @@
<li>
<ul class="sidebar-nav sidebar-nav-nested">
<li>
<a href="/sifter/docs/inputs/jsonload/"
<a href="/sifter/docs/inputs/plugin/"
class="sidebar-nav-item "
>jsonLoad</a></li>
>input plugin</a></li>
</ul>
</li>

<li>
<ul class="sidebar-nav sidebar-nav-nested">
<li>
<a href="/sifter/docs/inputs/plugin/"
<a href="/sifter/docs/inputs/jsonload/"
class="sidebar-nav-item "
>plugin</a></li>
>jsonLoad</a></li>
</ul>
</li>

Expand Down Expand Up @@ -346,6 +346,15 @@
</ul>
</li>

<li>
<ul class="sidebar-nav sidebar-nav-nested">
<li>
<a href="/sifter/docs/transforms/plugin/"
class="sidebar-nav-item "
>transform plugin</a></li>
</ul>
</li>

<li>
<ul class="sidebar-nav sidebar-nav-nested">
<li>
Expand Down
17 changes: 13 additions & 4 deletions docs/docs/inputs/embedded/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -99,18 +99,18 @@
<li>
<ul class="sidebar-nav sidebar-nav-nested">
<li>
<a href="/sifter/docs/inputs/jsonload/"
<a href="/sifter/docs/inputs/plugin/"
class="sidebar-nav-item "
>jsonLoad</a></li>
>input plugin</a></li>
</ul>
</li>

<li>
<ul class="sidebar-nav sidebar-nav-nested">
<li>
<a href="/sifter/docs/inputs/plugin/"
<a href="/sifter/docs/inputs/jsonload/"
class="sidebar-nav-item "
>plugin</a></li>
>jsonLoad</a></li>
</ul>
</li>

Expand Down Expand Up @@ -346,6 +346,15 @@
</ul>
</li>

<li>
<ul class="sidebar-nav sidebar-nav-nested">
<li>
<a href="/sifter/docs/transforms/plugin/"
class="sidebar-nav-item "
>transform plugin</a></li>
</ul>
</li>

<li>
<ul class="sidebar-nav sidebar-nav-nested">
<li>
Expand Down
17 changes: 13 additions & 4 deletions docs/docs/inputs/glob/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -99,18 +99,18 @@
<li>
<ul class="sidebar-nav sidebar-nav-nested">
<li>
<a href="/sifter/docs/inputs/jsonload/"
<a href="/sifter/docs/inputs/plugin/"
class="sidebar-nav-item "
>jsonLoad</a></li>
>input plugin</a></li>
</ul>
</li>

<li>
<ul class="sidebar-nav sidebar-nav-nested">
<li>
<a href="/sifter/docs/inputs/plugin/"
<a href="/sifter/docs/inputs/jsonload/"
class="sidebar-nav-item "
>plugin</a></li>
>jsonLoad</a></li>
</ul>
</li>

Expand Down Expand Up @@ -346,6 +346,15 @@
</ul>
</li>

<li>
<ul class="sidebar-nav sidebar-nav-nested">
<li>
<a href="/sifter/docs/transforms/plugin/"
class="sidebar-nav-item "
>transform plugin</a></li>
</ul>
</li>

<li>
<ul class="sidebar-nav sidebar-nav-nested">
<li>
Expand Down
17 changes: 13 additions & 4 deletions docs/docs/inputs/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -99,18 +99,18 @@
<li>
<ul class="sidebar-nav sidebar-nav-nested">
<li>
<a href="/sifter/docs/inputs/jsonload/"
<a href="/sifter/docs/inputs/plugin/"
class="sidebar-nav-item "
>jsonLoad</a></li>
>input plugin</a></li>
</ul>
</li>

<li>
<ul class="sidebar-nav sidebar-nav-nested">
<li>
<a href="/sifter/docs/inputs/plugin/"
<a href="/sifter/docs/inputs/jsonload/"
class="sidebar-nav-item "
>plugin</a></li>
>jsonLoad</a></li>
</ul>
</li>

Expand Down Expand Up @@ -346,6 +346,15 @@
</ul>
</li>

<li>
<ul class="sidebar-nav sidebar-nav-nested">
<li>
<a href="/sifter/docs/transforms/plugin/"
class="sidebar-nav-item "
>transform plugin</a></li>
</ul>
</li>

<li>
<ul class="sidebar-nav sidebar-nav-nested">
<li>
Expand Down
21 changes: 15 additions & 6 deletions docs/docs/inputs/jsonload/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -99,18 +99,18 @@
<li>
<ul class="sidebar-nav sidebar-nav-nested">
<li>
<a href="/sifter/docs/inputs/jsonload/"
class="sidebar-nav-item active"
>jsonLoad</a></li>
<a href="/sifter/docs/inputs/plugin/"
class="sidebar-nav-item "
>input plugin</a></li>
</ul>
</li>

<li>
<ul class="sidebar-nav sidebar-nav-nested">
<li>
<a href="/sifter/docs/inputs/plugin/"
class="sidebar-nav-item "
>plugin</a></li>
<a href="/sifter/docs/inputs/jsonload/"
class="sidebar-nav-item active"
>jsonLoad</a></li>
</ul>
</li>

Expand Down Expand Up @@ -346,6 +346,15 @@
</ul>
</li>

<li>
<ul class="sidebar-nav sidebar-nav-nested">
<li>
<a href="/sifter/docs/transforms/plugin/"
class="sidebar-nav-item "
>transform plugin</a></li>
</ul>
</li>

<li>
<ul class="sidebar-nav sidebar-nav-nested">
<li>
Expand Down
23 changes: 16 additions & 7 deletions docs/docs/inputs/plugin/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1">


<title>plugin &middot; Sifter</title>
<title>input plugin &middot; Sifter</title>



Expand Down Expand Up @@ -99,18 +99,18 @@
<li>
<ul class="sidebar-nav sidebar-nav-nested">
<li>
<a href="/sifter/docs/inputs/jsonload/"
class="sidebar-nav-item "
>jsonLoad</a></li>
<a href="/sifter/docs/inputs/plugin/"
class="sidebar-nav-item active"
>input plugin</a></li>
</ul>
</li>

<li>
<ul class="sidebar-nav sidebar-nav-nested">
<li>
<a href="/sifter/docs/inputs/plugin/"
class="sidebar-nav-item active"
>plugin</a></li>
<a href="/sifter/docs/inputs/jsonload/"
class="sidebar-nav-item "
>jsonLoad</a></li>
</ul>
</li>

Expand Down Expand Up @@ -346,6 +346,15 @@
</ul>
</li>

<li>
<ul class="sidebar-nav sidebar-nav-nested">
<li>
<a href="/sifter/docs/transforms/plugin/"
class="sidebar-nav-item "
>transform plugin</a></li>
</ul>
</li>

<li>
<ul class="sidebar-nav sidebar-nav-nested">
<li>
Expand Down
Loading

0 comments on commit c77cbeb

Please sign in to comment.