3.6.3

Bug Fixes:

• The eyaml-rotate-keys command-line tool failed to preserve block-style EYAML
  strings after key rotation.

3.6.2

Bug Fixes:

• The eyaml-rotate-keys command-line tool would generate a stack-dump when the
  key for an encrypted value contained dots. The underlying library for this
  tool now safely generates the internal YAMLPaths it uses.
• The default encoding when opening files is now set to utf-8 to support
  extended character sets on Windows.

3.6.1

3.6.1:
Enhancements:

• Enable verified support for ruamel.yaml up to version 0.17.10.

3.6.0

Bug Fixes:

• Some peculiar data constructions could cause the ConsolePrinter to stack-dump
  when writing debug messages.
• A NotImplementedError stack-dump was generated whenever a YAML Path was
  evaluated when written improperly for Collector Math operations. This
  specifically occurred when the RHS term was not demarcated with a parenthesis
  pair. Now, a YAML Path parsing error is generated, indicating where the
  missing character must appear.
• Use of an Array Element index in square-bracket notation ([N]) within a
  sub-path -- such as in a Search Expression or Collector -- caused incorrect
  YAML Path parsing. This usually manifested as a "not an integer index"
  error.
• Byte strings were causing stack-dumps during JSON serialization; they are now
  serialized as double-demarcated strings (a ' pair within a " pair) with a b
  prefix, like: {"byte_value": "b'BYTEVAL'"}.
• Bare Anchor name references were treated as Hash key names when & was not the
  very first non-separator character of a YAML Path or immediately following a
  [. So, /my_hash/&my_anchor was not working as expected.
• The Merger (and thus, the yaml-merge command-line tool) would only return the
  LHS document when neither LHS nor RHS documents were Hashes, no matter what
  the merge options were set to. This did not affect content which was
  children of Hashes.

Enhancements:

• YAML Path parsing errors are now a little more specific, indicating at which
  character index the issue occurs. API users who have been scanning error
  messages will need to update their code to accommodate the new text.
• Collector subtraction now handles Hash-of-Hashes and Array-of-Array results,
  which were not possible before.
• Array-of-Hash nodes can now be searched for the presence of a given key in
  its Hash elements using the . search operand, yielding matching elements (the
  entire Hash elements having the given key). The difference can be
  illustrated by contrasting these now-equivalent YAML Paths (where "books"
  is an Array-of-Hashes; imagine only some Hash elements have an "isbn" key):
  1. /books/*[.=isbn] or books.*[.=isbn]
  2. /books[.=isbn]/isbn or books[.=isbn].isbn
  3. /books/*[has_child(isbn)]/isbn or books.*[has_child(isbn)].isbn
  4. /books[has_child(isbn)]/isbn or books[has_child(isbn)].isbn
     All four of those queries yield exactly the same data. Note that example 2
     behaves like examples 3 and 4. Examples 2-4 yield the entire matching Hash,
     not just the "isbn" value. This enables access to other keys of the Hash
     without necessitating use of a [parent()] search keyword, which would be
     necessary for example 1 if you wanted to access any key other than "isbn"
     from the matches.
• YAML Merge Keys can now be accessed directly by Anchor name, yielding the
  entire original -- pre-merged -- reference Hash. This has very limited
  utility. Using this in isolation will only reveal the default values for any
  referenced keys, ignoring -- perhaps confusingly -- any local overrides. It
  can however be helpful when reverse-engineering very complex merge
  arrangements.
• The yaml-merge command-line tool (and the underlying Merger class) now offer
  an option -- --preserve-lhs-comments (-l) -- that will attempt to preserve
  LHS document comments. USE WITH CAUTION. At present, comment handling
  during a merge is unwieldy, so some comments or new-line characters may
  appear to become divorced from nodes they should obviously be attached to.
  As such, the default behavior of the merge engine will continue to be removal
  of all comments. At this time, RHS document comments will still be discarded
  during merge operations. This will be revisited when ruamel.yaml refactors
  how YAML comments are handled.
• The yaml-merge command-line tool now offers a new option, --multi-doc-mode
  (-M), which accepts one of the following modes:
  • CONDENSE_ALL: This is the default, which merges all multi-documents up
    into single documents during the merge.
  • MERGE_ACROSS: Condense no multi-documents; rather, only merge documents
    "across" from right to left such that the first document in the RHS multi-
    document merges only into the first document in the LHS multi-document, the
    second across similarly, and so on.
  • MATRIX_MERGE: Condense no multi-documents; rather, merge every RHS
    document in a multi-document RHS into every LHS document in a multi-
    document LHS.
• The [has_child(NAME)] Search Keyword now accepts an &NAME form of its first
  (only) parameter. This switches the function to match against Anchor/Alias
  names, including YAML Merge Keys.
• YAML Merge Keys can now be deleted by their Anchor/Alias name via the
  yaml-set command-line tool and the underlying Processor class.
• YAML Merge Keys can now be created, offering run-time merging of
  same-document Hash data. The yaml-set command-line tool offers a new option,
  --mergekey, which applies to --change targets the new YAML Merge Key, as long
  as each target is a Hash.
  WARNING: As a consequence of adding this capability to the yaml-set command-
  line tool, it is no longer possible to implicitly alias scalar nodes by
  passing only the --change and --anchor parameters. The operation must now be
  explicit by setting --aliasof or --mergekey along with --change and
  optionally with --anchor.
• The yaml-diff tool now supports multi-document sources. Only one document of
  any multi-document source on each side of the LHS-RHS comparison can be
  selected for the operation (diffs are performed only between two documents).
  Such selection is made via two new options, --left-document-index|-L and
  --right-document-index|-R. An error is emitted whenever a multi-document
  source is detected without an appropriate document index selection.
• YAML Unordered Sets -- https://yaml.org/type/set.html -- are now fully
  supported in YAML Paths, this project's API, and the reference command-line
  tools. Because an Unordered Set is effectively a Hash (map/dict) where the
  entries are key-value pairs all having null (None) values, their entries are
  accessible only by their exact key. While they look in YAML data like Arrays
  (sequences/lists) with a leading ? rather than a -, they are not; their
  entries cannot be accessed by a numerical index because they are defined in
  the YAML specification as deliberately unordered.

API Changes:

• The common.nodes utility class now has a generally-useful static method which
  accepts any String data and safely converts it to its native Python data-type
  equivalent with special handling for case-insensitive Booleans via
  ast.literal_eval: typed_value.
• The common.searches utility class now requires both terms to be of the same
  data-type for comparisons. When they types materially differ -- int and
  float are treated as similar enough -- a String comparision is performed.
  This is how it has always been excepting that types were lazily coalesced in
  older versions; they are now converted before the comparison is considered.
• The NodeCoords wrapper now supports more utility properties and methods:
  • .unwrapped_node is the same output as calling
    NodeCoords.unwrap_node_coords(data) except it can be called directly upon
    instances of NodeCoords rather than as a static method call. The static
    method is still available.
  • .deepest_node_coord returns whichever NodeCoord instance is most deeply
    wrapped by an instance of NodeCoords. Such wrapping comes from nesting
    Collectors. This method simplfies getting to the original data element(s).
  • .wraps_a(Type) indicates whether the deepest wrapped data element is of a
    given data-type.

3.5.0

Bug Fixes:

• Search expressions against Boolean values, [key=True] and [key=False], were
  impossible. Now, they are working and are not case-sensitive, so [key=True],
  [key=true], [key=TRUE], and such all work as expected.
• When null values were present, Deep Traversal (**) segments would always
  return every node with a null value even when they would not match filter
  conditions after the ** segment. When mustexist=False, this would also cause
  a YAMLPathException.
• Descendent searches were considering only the first child of the search
  ancestor. Now, ANY matching descendent node will correctly yield the
  ancestor.
• Some Python-generated complex data types were escaping JSONification,
  leading to unexpected stack-dumps when writing out JSON data for data types
  like date and datetime.

Enhancements:

• An entirely new segment type has been added to YAML Path and is now supported
  by the library and reference implementation command-line tools: Keyword
  Searches. Similar to programming language keywords, these reserved Keywords
  work much like functions, accepting parameters and performing algorythmic
  operations or returning data not otherwise accessible to other YAML Path
  segment types. These new capabilities -- explored on the project Wiki --
  include:
  • [has_child(NAME)]
  • [name()]
  • [max(NAME)]
  • [min(NAME)]
  • [parent([STEPS])]
• When stringified, YAML Paths with a solitary * wildcard segment were printed
  using their internal RegEx variant, [.=~/.*/]. They are now printed as they
  are entered, using a solitary *. As a consequence, any deliberate RegEx of
  [.=~/.*/] is also printed as its equivalent solitary *.
• The yaml-paths command now allows printing YAML Paths without protective
  escape symbols via a new --noescape option. While this makes the output more
  human-friendly, the unescaped paths will not be suitable for use as YAML Path
  input to other YAML Path processors where special symbols require escaping.
• [API] The NodeCoords class now tracks ancestry and the last YAML Path segment
  responsible for triggering its generation. The ancestry stack --
  List[AncestryEntry] -- was necessary to support the [parent()] Search
  Keyword. The responsible YAML Path segment tracking was necessary to enable
  Hash/map/dict key renaming via the [name()] Search Keyword. These optional
  attributes may be set when the NodeCoords is generated.
• [API] YAMLPath instances now have a pop() method. This mutates the YAMLPath
  by popping off its last segment, returning that segment.

3.4.1

Bug Fixes:

• yaml-set (and the underlying Processor class) were unable to change nodes
  having a null (None) value to anything else. This changes how null/None
  values are handled by the Processor during node retrieval; they are no longer
  discarded, so you will receive None as the data of any retrieved NodeCoords
  for appropriate null/None leaf nodes.

Enhancements:

• Python 3.9 is now supported (because common testing tools finally work with
  Python 3.9).
• The node deletion capability of the yaml-set command is now part of the
  library. See Processor::delete_nodes(...) and
  Processor::delete_gathered_nodes(...) for details.
• The node aliasing capability of the yaml-set command is now part of the
  library. See Processor::alias_nodes(...) and
  Processor::alias_gathered_nodes(...) for details.
• The node tagging capability of the yaml-set command is now part of the
  library. See Processor::tag_nodes(...) and
  Processor::tag_gathered_nodes(...) for details.
• The library now supports loading YAML from String rather than only from file.
  Simply pass a new literal=True keyword parameter to
  Parsers::get_yaml_data(...) or Parsers::get_yaml_multidoc_data(...) to
  indicate that source is literal serialized (String) YAML data rather than a
  file-spec. This mode is implied when reading from STDIN (source is "-").
• The emitter_write_folded_fix.py patch file for ruamel.yaml has been removed
  in favor of an author-supplied solution to the problem --
  https://sourceforge.net/p/ruamel-yaml/tickets/383/ -- for which the patch was
  originally written.

Known Issues:

• ruamel.yaml version 0.17.x is a major refactoring effort by the project's
  owner. As such, only select versions will be marked as compatible with
  yamlpath. Such marking occurs in this project's dependencies list via the
  setup.py file. This is necessary because I use yamlpath in production
  environments where stability is paramount; I need the freedom to update
  yamlpath at-will without incurring any unexpected failures due to
  incompatible ruamel.yaml changes. I will try to test some -- but not all --
  ruamel.yaml releases from time to time and update yamlpath dependency
  compatibilities accordingly.
• ruamel.yaml version 0.17.4 somewhat resolves a previously reported issue --
  https://sourceforge.net/p/ruamel-yaml/tickets/351/ -- wherein certain
  arrangements of comments or new-lines within YAML files near aliased hash
  keys would cause a total loss of data when the stream was written to file.
  Now, the data is no longer entirely lost. However, the preceding comment or
  new-line is deleted when the stream is written to file. This is deemed to be
  an acceptable compromise, for now, because the alternative is to either lose
  the entire document or lose all attempted changes to the affected document.
  Until the issue is properly fixed, an XFAIL test will continue to be in the
  yamlpath unit test suite.

3.4.0

Bug Fixes:

• For the yaml-diff command-line tool, custom identity keys for specific
  records in AoHs of the RHS document were not being considered for comparison.
  Any identity key set for the whole AoH was being applied to every record in
  the set.

Enhancements:

• The yaml-get command-line tool -- and the underlying Processor::get_nodes
  method -- now retrives nodes with null values. Non-JSON null results from
  yaml-get are printed using the ASCII NULL control-character (Hexadecimal 00)
  wherever encountered. While most terminals and shells won't visibly print
  this character, it is there and can be picked up by downstream parsers of
  STDOUT. When the output is rendered as JSON, the unquoted "null" identifier
  is used.
• The yaml-set command-line tool now enables assigning null values using a new
  --null|-N input option.
• The yaml-set command-line tool now supports assigning YAML Aliases to target
  --change|-g nodes, referencing any other node via --aliasof|-A whether it is
  already Anchored, or not. Should the Anchor not already exist, a unique name
  will be automatically assigned. This automatic name can controlled via
  --anchor|-H. Setting --change|-g and --aliasof|-A to the same node along
  with a new --anchor|-H explicitly renames an existing Anchor and its Aliases.
  The same is implicitly possible by specifying --change|-g and --anchor|-H
  without --aliasof|-A. Using these against non-YAML files merely duplicates
  the target value to the indicated --change|-g nodes.
• With a new --tag|-T auxilliary option, the yaml-set command-line tool now
  supports assigning custom YAML tags (data-type specifiers) to nodes created
  or updated during the operation.
• The yaml-merge and yaml-diff command-line tools now also handle YAML tags.
• The single-star wildcard segment translation was changed from [.!=""] to
  [.=~/.*/] which enables it to match any key or value (including empty-string
  and null).

API Changes:

• The yamlpath.func library is too big and some very useful general code has
  become attached to special-purpose classes. As such, it's time for
  refactoring before this becomes untennable. New, better-organized static
  classes were created under a new yamlpath.common namespace. Since this
  would be destructive to other Python code which depends on the old
  organization, this is a heads-up. Your existing code will continue to work
  without any changes, but only until 4.x (which is not yet planned). Until
  then, you must update your own code to use the new static classes for the
  following functions and methods so as to avoid fatal errors down the road:

  • yamlpath.func.append_list_element
    -> yamlpath.common.Nodes.append_list_element

  • yamlpath.func.build_next_node
    -> yamlpath.common.Nodes.build_next_node

  • yamlpath.func.clone_node
    -> yamlpath.common.Nodes.clone_node

  • yamlpath.func.create_searchterms_from_pathattributes
    -> yamlpath.common.Searches.create_searchterms_from_pathattributes

  • yamlpath.func.ensure_escaped
    -> yamlpath.YAMLPath.ensure_escaped

  • yamlpath.func.escape_path_section
    -> yamlpath.YAMLPath.escape_path_section

  • yamlpath.func.get_node_anchor
    -> yamlpath.common.Anchors.get_node_anchor

  • yamlpath.func.get_yaml_data
    -> yamlpath.common.Parsers.get_yaml_data

  • yamlpath.func.get_yaml_editor
    -> yamlpath.common.Parsers.get_yaml_editor

  • yamlpath.func.get_yaml_multidoc_data
    -> yamlpath.common.Parsers.get_yaml_multidoc_data

  • yamlpath.func.make_float_node
    -> yamlpath.common.Nodes.make_float_node

  • yamlpath.func.make_new_node
    -> yamlpath.common.Nodes.make_new_node

  • yamlpath.func.search_anchor
    -> yamlpath.common.Searches.search_anchor

  • yamlpath.func.search_matches
    -> yamlpath.common.Searches.search_matches

  • yamlpath.func.stringify_dates
    -> yamlpath.common.Parsers.stringify_dates
    NOTE that this method is deprecated and will be eliminated in favor of
    using its more comprehensive replacement,
    yamlpath.common.Parsers.jsonify_yaml_data

  • yamlpath.func.unwrap_node_coords
    -> yamlpath.wrappers.nodecoords.NodeCoords.unwrap_node_coords

  • yamlpath.func.wrap_type
    -> yamlpath.common.Nodes.wrap_type

  • yamlpath.merger.Merger.combine_merge_anchors
    -> yamlpath.common.Anchors.combine_merge_anchors

  • yamlpath.merger.Merger.delete_all_comments
    -> yamlpath.common.Anchors.delete_all_comments

  • yamlpath.merger.Merger.rename_anchor
    -> yamlpath.common.Anchors.rename_anchor

  • yamlpath.merger.Merger.replace_anchor
    -> yamlpath.common.Anchors.replace_anchor

  • yamlpath.merger.Merger.scan_for_anchors
    -> yamlpath.common.Anchors.scan_for_anchors

  • yamlpath.merger.Merger.set_flow_style
    -> yamlpath.common.Anchors.set_flow_style

  Until you update your code, a deprecation warning will be printed to STDERR
  every time the yamlpath.func file is imported and the first time one of the
  relocated Merger static methods is called. To be rid of the message, update
  your code to use the new sources of the deprecated functions/methods and
  remove every import of and from yamlpath.func.

3.3.0

Bug Fixes:

• It was impossible to install yamlpath 3.x without first installing
  ruamel.yaml via pip for Python 3.x. Not only has this been fixed but
  explicit tests have been created to ensure this never happens again.

Enhancements:

• A new command-line tool, yaml-diff, now compares exactly two
  YAML/JSON/Compatible documents, producing a GNU diff-like report of any
  differences in the data they present to parsers. Along with diff's "a"
  (added), "c" (changed), and "d" (deleted) report entries, affected YAML Paths
  are printed in lieu of line numbers. Further, a report entry of "s" (same)
  is available and can be enabled via command-line options. This tool also
  features optional special handling of Arrays and Arrays-of-Hashes, which can
  be configured as CLI options or via an INI file for distinct settings per
  YAML Path. See --help or the Wiki for more detail.

API Changes:

• NodeCoords now employ a new path attribute. This is an optional parameter
  which is assigned during construction to later report the translated origin
  YAML Path; this is where the node was found or created within the DOM. Note
  that Collector segments work against virtual DOMs, so the YAML Path of an
  outer Collector will be virtual, relative to its parent at construction; when
  nested, this will be a bare list index. Any NodeCoords in the virtual
  container which point to real nodes in the DOM will have their own concrete
  YAML Paths.
• YAMLPath instances now support nonmutating addition of individual segments
  via the + operator. Whereas the append() method mutates the YAMLPath being
  acted upon, + creates a new YAMLPath that is the original plus the new
  segment. In both cases, the orignal YAMLPath's seperator is retained during
  both operations. As with .append(), new segments added via + must also be
  properly escaped -- typically via path.escape_path_section -- before being
  added.

3.2.0

3.2.0:
Enhancements:

• Expanded YAML Path Search Expressions such that the OPERAND of a Search
  Expression may be a sub-YAML Path. This enables searching descendent nodes
  -- without moving the document pointer -- to yield ancestors with matching
  descendants. This has more utility when searching against Arrays-of-Hashes.

Bug Fixes:

• Date values in YAML could not be written to JSON streams; the JSON renderer
  would generate an incompatibility error. Now, dates are written as Strings
  to JSON. This affected: yaml-get, yaml-set (in stream mode), yaml-merge, and
  yaml-paths.

3.1.0

Enhancements:

• yaml-set can now delete nodes when using --delete rather than other input
  arguments.
• A new command-line tool has been created, yaml-validate, which validates
  YAML/JSON/compatible single- and multi-documents (files or STDIN).

Bug Fixes:

• The yaml-merge command-line tool wasn't allowing Scalar values supplied via
  STDIN -- with no structure, just bare Scalar values -- to be appended to
  exising Array data structures; rather, it was wholly overwriting the
  destination, deleting all pre-exisiting elements.
• The yaml-merge command-line tool wasn't accepting empty-strings as STDIN
  documents; it was reporting a document-read error, instead. This turns out
  to be useful when you want to use yaml-merge instead of yaml-set to
  deliberately write an empty-string value at some --mergeat location within
  the LHS document.
• The yaml-merge command would not accept any variation of "false" as a Scalar
  value input; it was instead reporting a document-read error. This turns out
  to be useful when using yaml-merge as if it were yaml-set to write a false
  Boolean value to the LHS document at some --mergeat location.

API Changes:

• The func.get_yaml_data and func.get_yaml_multidoc_data functions now
  return/yield tuples. The first field is the desired yaml_data (can be None
  for empty documents) and the second field is a Boolean which indicates True
  when the document loaded with no errors or False when an error occurred.
  This is necessary in order to accept a bare "false" Scalar value as a
  STDIN-supplied document.