Merge pull request #1 from ccpgames/feature/first_build

Version 0.1.0 - Initial Build

.github/workflows/publish-to-pypi.yml

@@ -0,0 +1,29 @@
+name: Publish Python Package
+
+on:
+  release:
+    types: [ created ]
+
+jobs:
+  build-and-publish:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2
+      - name: Set up Python
+        uses: actions/setup-python@v2
+        with:
+          python-version: '3.8'
+      - name: Install Dependencies
+        run: |
+          python --version
+          python -m pip install --upgrade pip
+          pip install --upgrade setuptools wheel twine
+      - name: Build and Package
+        run: |
+          python setup.py sdist bdist_wheel
+      - name: Publish to PyPI
+        uses: pypa/[email protected]
+        with:
+          user: __token__
+          password: ${{ secrets.PYPI_API_TOKEN }}
+          verbose: true

.github/workflows/unit-test.yml

@@ -0,0 +1,28 @@
+name: Run Unit Tests
+
+on: [push, pull_request]
+
+jobs:
+  unit-tests:
+    runs-on: ubuntu-latest
+
+    strategy:
+      matrix:
+        python-version: ["3.8", "3.9", "3.10", "3.11", "3.12"]
+
+    steps:
+    - uses: actions/checkout@v2
+
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v2
+      with:
+        python-version: ${{ matrix.python-version }}
+
+    - name: Install Dependencies
+      run: |
+        python -m pip install --upgrade pip
+        pip install -r requirements.txt
+
+    - name: Run tests
+      run: |
+        python -m unittest discover -v -f ./tests

.gitignore

@@ -0,0 +1,11 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# Virtual environments
+venv*/
+.venv*/
+
+# IDEA Stuff
+.idea/

CHANGELOG.md

@@ -0,0 +1,15 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+
+## [0.1.0] - UNRELEASED
+
+Initial release
+
+### Added
+
+- Everything!

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 CCP Games
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -1,2 +1,84 @@
-# ccp-stencil
-An Alviss powered Template renderer where the context input can be files and entire directory structures can be rendered.
+# CCP Stencil
+
+An Alviss and Jinja2 powered template renderer where the context input can be 
+files and entire directory structures can be rendered.
+
+This is a generalized variant of the "CCP Borg Bootstrapper" project 
+bootstrapping and build and deployment tool which used entire "template" 
+projects that were rendered to bootstrap entire projects and to render CI/CD 
+manifests on demand. 
+
+The rest of this readme is (at the moment) just a "sketch" for how this package 
+should work and was written before any actual code or functionality.
+
+## Context
+
+- [x] No context (Weird use case...?!?)
+- [x] kwargs context (via code)
+- [x] Dict context (via code)
+- [x] Alviss file context (json/yaml + inheritance)
+- [ ] Args context (from commandline)
+
+## Template
+
+- [x] String template (via code)
+- [x] File template
+- [ ] Args template (from commandline)
+- [ ] Directory template
+
+## Renderer
+
+- [x] String renderer (via code)
+- [x] Stdout renderer (for commandline)
+- [x] File renderer
+- [ ] Directory renderer
+
+## Other features...?
+
+- ENV var rendering (can be done via ${__ENV__:FOO} in Alviss input)?
+- Meta-data header in files for Directory rendering that controls file names and/or if they should be rendered or not
+- Meta-data file for directories in Directory rendering that control the directory name?
+- Proper Jinja2 Environment Template Loader to enable Jinja's include/extend stuff?
+- Custom macros/scripts/filters?
+
+## Use Cases
+
+- From commandline (main use case, e.g. rendering CI/CD manifests)
+- From code
+
+### Command Line Use Case Examples
+
+Using these as a basis for functionality (this is written before any actual code)!
+
+#### Example 1
+
+```shell
+$ ccp-stencil -i context.yaml -t template.html -o result.html
+```
+
+- Alviss file input: `-i context.yaml`
+- Template file input: `-t template.html`
+- Render file output: `-o result.html`
+
+
+#### Example 2
+
+```shell
+$ ccp-stencil -a name=Bob -a age=7 -a color=Red -s "My name is {{name}} and I am {{age}} years old and my favorite color is {{color}}"
+My name is Bob and I am 7 years old and my favorite color is Red
+```
+
+- Args input: `-a name=Bob -a age=7 -a color=Red`
+- String template input: `-s "My name is {name} and I am {age} years old and my favorite color is {color}"`
+- Print (stdout) output: _No argument, this is default!_
+
+
+#### Example 3
+
+```shell
+$ ccp-stencil -T templates/ -O build/
+```
+
+- No context input: _No argument, this is default!_
+- Template directory input: `-T templates/`
+- Render directory output: `-O build/`

_cli_out_test.txt

@@ -0,0 +1 @@
+My name is Bob the Cat and I am 57 years old and my favorite color is Blue

_sandbox_input.yaml

@@ -0,0 +1,5 @@
+name: Bob
+age: 7
+colors:
+  favorite: Blue
+  weakness: Yellow

_sandbox_output.txt

@@ -0,0 +1 @@
+My name is Bob and I am 7 years old and my favorite color is NOT Blue

_sandbox_string.py

@@ -0,0 +1,20 @@
+from ccpstencil import stencils
+
+
+def main():
+    # ctx = stencils.DictContext()
+    # ctx = stencils.DictContext({'name': 'Bob', 'age': 7, 'color': 'Red'})
+    # ctx = stencils.KeyWordArgumentContext(name='Bob', age=7, color='Red')
+    ctx = stencils.AlvissContext('_sandbox_input.yaml')
+
+    # tpl = stencils.StringTemplate('My name is {{name}} and I am {{age}} years old and my favorite color is {{color}}')
+    tpl = stencils.FileTemplate('_sandbox_template.txt')
+
+    # rnd = stencils.StringRenderer(ctx, tpl)
+    # rnd = stencils.StdOutRenderer(ctx, tpl)
+    rnd = stencils.FileRenderer('./build/something/_sandbox_output.txt', ctx, tpl, overwrite=True)
+    print(rnd.render())
+
+
+if __name__ == '__main__':
+    main()

_sandbox_template.txt

@@ -0,0 +1 @@
+My name is {{name}} and I am {{age}} years old and my favorite color is NOT {{color}}

ccpstencil/__init__.py

@@ -0,0 +1,5 @@
+__version__ = '0.1.0'
+
+__author__ = 'Thordur Matthiasson <[email protected]>'
+__license__ = 'MIT License'
+__copyright__ = 'Copyright 2024 - CCP Games ehf'

ccpstencil/cli/ccp_stencil/_runner.py

@@ -0,0 +1,55 @@
+__all__ = [
+    'StencilRunner',
+]
+from ccptools.structs import *
+from ccpstencil.stencils import *
+import logging
+log = logging.getLogger(__file__)
+
+
+class StencilRunner:
+    def __init__(self):
+        self.verbose: bool = False
+
+        self.template: Optional[str] = None
+        self.string_template: Optional[str] = None
+
+        self.input: Optional[str] = None
+        self.additional_arg_list: List[str] = []
+
+        self.output: Optional[str] = None
+        self.no_overwrite: bool = False
+
+    def run(self):
+        rnd = self.get_renderer()
+        rnd.template = self.get_template()
+        rnd.context = self.get_context()
+        rnd.render()
+
+    def get_template(self) -> ITemplate:
+        if self.template:
+            return FileTemplate(self.template)
+        else:
+            return StringTemplate(self.string_template)
+
+    def _parse_arg(self, arg_str: str) -> Tuple[str, str]:
+        log.debug(f'_parse_arg({arg_str})')
+        return tuple(arg_str.split('=', maxsplit=1))  # noqa
+
+    def get_context(self) -> IContext:
+        if self.input:
+            ctx = AlvissContext(self.input)
+        else:
+            ctx = DictContext()
+
+        if self.additional_arg_list:
+            for arg in self.additional_arg_list:
+                ctx.nested_update(*self._parse_arg(arg))
+        return ctx
+
+    def get_renderer(self) -> IRenderer:
+        if self.output:
+            return FileRenderer(self.output,
+                                overwrite=not self.no_overwrite)
+        else:
+            return StdOutRenderer()

ccpstencil/cli/ccp_stencil/main.py

@@ -0,0 +1,59 @@
+import argparse
+
+from ccpstencil import __version__ as version
+from ccpstencil.structs import *
+from ._runner import StencilRunner
+
+import sys
+
+
+def main():
+    parser = argparse.ArgumentParser(description='Renders a template using Jinja2 and Alviss input context',
+                                     epilog=f'CCP-Stencil version {version}')
+
+    parser.add_argument('-v', '--verbose', action="store_true", help='Spits out DEBUG level logs')
+
+    parser.add_argument('-i', '--input', nargs='?', help='Alviss file with input context (YAML or JSON)')
+
+    parser.add_argument('-a', '--arg', action='append', help='Add additional Context arguments from the command line, e.g. -a foo=bar')
+
+    parser.add_argument('-o', '--output', help='File to write the results to (otherwise its just printed to stdout)',
+                              default='', nargs='?')
+    parser.add_argument('--no-overwrite', action="store_true", help='Makes sore existing output files are not overwritten')
+
+    input_group = parser.add_mutually_exclusive_group(required=True)
+    input_group.add_argument('-t', '--template', default='',
+                             help='Template file to render')
+    input_group.add_argument('-s', '--string-template', default='',
+                             help='Supply a string directly from the command line to use as a template instead of a file')
+
+    args = parser.parse_args()
+
+    if args.verbose:
+        import logging
+        logging.basicConfig(level=logging.DEBUG)
+
+        log = logging.getLogger(__name__)
+        log.info('Verbose logging enabled')
+        log.info(f'Args: {args=}')
+
+    runner = StencilRunner()
+    runner.verbose = args.verbose or False
+    runner.input = args.input or None
+    runner.output = args.output or None
+    runner.no_overwrite = args.no_overwrite or False
+    runner.template = args.template or None
+    runner.string_template = args.string_template or None
+    if args.arg:
+        for arg in args.arg:
+            runner.additional_arg_list.append(arg)
+
+    try:
+        runner.run()
+    except StencilError as e:
+        print(f'An error occurred: {e!r}', file=sys.stderr)
+        sys.exit(3)
+
+
+if __name__ == '__main__':
+    main()

ccpstencil/context/__init__.py

@@ -0,0 +1,4 @@
+from ccpstencil.structs.interfaces import IContext
+from ._dict import *
+from ._kwarg import *
+from ._alviss import *

ccpstencil/context/_alviss.py

@@ -0,0 +1,31 @@
+__all__ = [
+    'AlvissContext',
+]
+
+from ccpstencil.structs import *
+from alviss import quickloader
+from ccptools.tpu import iters
+import logging
+log = logging.getLogger(__file__)
+
+
+class AlvissContext(IContext):
+    def __init__(self, file_name: str, **kwargs):
+        self._file_name = file_name
+        self._data = quickloader.autoload(file_name)
+        self._update_map: Dict[Tuple[str], Any] = {}
+
+    def update(self, key: str, value: Any):
+        self._data[key] = value
+
+    def nested_update(self, key_tuple: Union[str, Tuple[str]], value: Any):
+        log.debug(f'nested_update({key_tuple=}, {value=})')
+        if isinstance(key_tuple, str):
+            key_tuple = key_tuple.split('.')
+        if isinstance(key_tuple, List):
+            key_tuple = tuple(key_tuple)
+        self._data.update(**iters.nest_dict(list(key_tuple), value))  # noqa
+
+    def as_dict(self) -> Dict:
+        log.debug(f'as_dict() YAML dump:\n{self._data.as_yaml(unmaksed=True)}')
+        return self._data.as_dict(unmaksed=True)