PowerShell module for DSCv3

[michaeltlombardi]

Note

This discussion references the work and ideas of @Gijsreyn, especially those raised in #498 and #507.

For the purposes of this discussion, I'm using the term PowerShell DSC (PSDSC) to refer to versions 1 and 2 and the PSDesiredStateConfiguration module.

Bottom Line Up Front

There are good reasons to believe that developing and maintaining a PowerShell module for DSCv3 would be useful for several personas and tasks:

• People using DSC from within PowerShell would benefit from being able to invoke commands and manage the output using PowerShell commands and the pipeline.
• Maintainers of configurations for PSDSC would benefit from being able to convert their existing PSDSC configurations (PowerShell code) to DSC configuration documents (YAML or JSON).
• Authors and maintainers of DSC configuration documents would benefit from being able to represent, generate, manipulate, and export configuration documents with scripts or in the shell.
• Developers of DSC resources in PowerShell would benefit from enhanced authoring, including being able to participate more fully and clearly in the semantics of DSC while maintaining compatibility with PSDSC.

Current Context

At this time, the new Microsoft.PowerShell/DSC resource adapter enables users to leverage class-based PSDSC resources in DSC and supports a growing number of features that don't exist in PSDSC. The Microsoft.Windows.PowerShell/DSC resource adapter enables users to leverage PSDSC resources compatible with Windows PowerShell and PSDSC in DSC.

Every command for DSC has a JSON schema describing the objects that the commands expect and return as output, but working with those objects requires users to convert to and from JSON or YAML manually and manage them as dictionaries or PSCustomObjects.

Users with PSDSC configurations have to recreate those configurations manually in a new DSC configuration document by hand.

Anyone who wants to create or manipulate DSC configuration documents in PowerShell has to implement their own solution.

While the adapters recognize enhancements to PSDSC resources for DSC, authors get no author-time feedback for their resources. They can't use commands to inspect their resources to determine what capabilities their resources have or whether they're implemented to best practices.

Proposed Features

Replies to this discussion include proposals for the usage, design and scope of features for a module that enhances the user and developer experience for DSC in PowerShell.

[Gijsreyn]

Mikey, while going through the text you've just written, I was amazed that you could extract that much information from the PRs I opened. Thanks for the time and detail you placed in this discussion. I hope to see it getting some movement.

[michaeltlombardi]

Some of it was directly from your work, some of it was based on thoughts I've been having during the development cycle - the work you've been doing definitely galvanized me!

[michaeltlombardi]

Feature Group: Enhanced resource authoring and inspection

User Stories

As an open source developer, I want to be able to author my DSC resource in PowerShell, taking advantage of DSC's features with minimal effort.

Related features: DSC resource base class, DSC resource attributes, Get-DscResourceInfo command, *-ModuleDscInfo commands

As a PSDSC class-based resource author, I want to update my resource to participate more fully in the semantics of DSC without losing compatibility with PSDSC or requiring any additional dependencies in my module, so that I can fully support DSC users without breaking my existing users, or needing to bifurcate my development and maintenance.

Related features: DSC resource static metadata methods

As a maintainer of PowerShell class-based resources, I want to be able to semantically version my resources separately from my PowerShell module, so that my users can understand how the resources have changed between releases and, ideally, version pin their usage of my resources.

Related features: DSC resource attributes, DSC resource static metadata methods, Get-DscResourceInfo command, *-ModuleDscInfo commands

As a maintainer of PowerShell resources who wants to support DSC and PSDSC users, I want to be able to include information about my resources in my module manifest that DSC can use to surface information about my resources more readily.

Related features: *-ModuleDscInfo commands

As a user of DSC resources implemented in PowerShell, I want to be able to inspect those resources to see their capabilities, properties, usage, examples, and metadata, so that I can better understand whether and how I should use those resources.

Related features: DSC Resource attributes, DSC resource static metadata methods, Get-DscResourceInfo command, Get-DscHelp command

As an integrating developer, I want to be able to retrieve information about resources implemented in PowerShell that provide me with as much of the same information as a resource manifest, so that I can use that information in my integration and surface the metadata to my users.

Related features: DSC resource attributes, DSC resource static metadata methods, Get-DscResourceInfo command, Get-DscHelp command

As a resource author, I want to be able to implement DSC resources as PowerShell classes and rely on synthetic testing, so that I don't have to implement property equivalency checking by hand for each resource.

Related features: DSC resource base class, DSC resource static metadata methods, DSC resource attributes

As a resource author, I want to be able to implement assertion-only resources as PowerShell classes, so that I can provide a way to return information to my users for further use in their configuration.

Related features: DSC resource base class, DSC resource static metadata methods, DSC resource attributes

As a resource author, I want to be able to inspect my resources for best practices and potential issues, so that I can make sure my implementation is as useful, reliable, and performant as possible.

Related features: DSC resource base class, DSC resource static metadata methods, DSC resource attributes, Get-DscResourceInfo command, Assert-DscResource command

Feature: DSC base class and/or interface

We could provide a base class that resource authors can inherit from that implements synthetic testing, converting to and from YAML, returning a manifest for the resource, and otherwise makes implementing a DSC resource as a PowerShell class easier.

The downside to using a base class is that it requires taking a dependency on the DSC module itself, but the benefits are pretty powerful.

We could also provide an interface with default implementations to help users understand how to implement their own resources, or extend existing classes to work as a DSC resource without deriving from the base class. This option is not exclusive to providing a base class.

Feature: DSC resource attributes

We could provide attributes that authors can use to decorate their resource classes and their members. For example, consider the following pseudocode snippet:

# Define metadata about the resource
[DscResourceInfo(
    Description = "Manages the existence and contents of UTF8-encoded text files.",
    Version     = "1.2.0",
    SchemaUri   = "tailspintoys.com/schemas/dsc/resources/utf8file.schema.json",
    DocsUri     = "tailspintoys.com/docs/dsc/resources/utf8file",
    UseSyntheticTest
)]
class Utf8File {
  #region    Resource properties
    # Define the property, don't specify a JSON schema - auto-generate from definition.
    [DscResourceProperty(
        Key,
        Required,
        Description = @'
The path to the file the instance manages. The path can be absolute
or relative. Relative paths are resolved against the current working
directory when you invoke the resource.
'@
    )]
    [string] $Path
    
    # Indicates that this property uses the semantics of the _exist property
    [DscResourceStandardProperty()]
    [bool] $Exist
    
    # Only defines the description for the property, uses default options
    [DscResourceProperty("The contents of the file.")]
    [string] $Content
    
    # Read only property with custom JSON schema
    [DscResourceProperty(ReadOnly)]
    [DscJsonSchema(@'
{
    "title": "File length",
    "description": "The size of the file in bytes.",
    "type": "integer",
    "minimum": 0
}
'@)]
    [int] $Length
    
    # Read only properties with description
    [DscResourceProperty(
        ReadOnly,
        Description = "Defines when the file was created."
    )]
    [datetime] $CreationTimeUtc

    [DscResourceProperty(
        ReadOnly,
        Description = "Defines when the file was last updated."
    )]
    [datetime] $LastWriteTimeUtc
    
    # Write-only property
    [DscResourceProperty(
        WriteOnly,
        Description = "Indicates whether the resource should create parent folders if they don't exist."
    )]
    [bool] $CreateParentFolders = $true
  #endregion Resource properties
  #region    Resource methods
    [Utf8File] Get() {
        # Implementation
    }
    
    # Don't implement test, rely on synthetic test behavior
    
    [DscResourceSetMethod(ImplementsPreTest, HandlesExist)]
    [Utf8File] Set() {
        # Implementation
    }
    
    # Support the delete method
    [void] Delete() {
        # Implementation
    }
  #endregion Resource methods

The attributes provide information that authors can't currently surface for PSDSC resources. More consideration is required to determine the design and scope of such attributes. Like a base class or interface, using the attributes requires taking a dependency on the module.

Feature: DSC resource static metadata methods

As an alternative to using the resource attributes, we can define and document static methods that authors can implement on their resources to provide the same information. Consider the following pseudocode snippet:

class Utf8File {
  #region    Static metadata methods
    static [hashtable] GetResourceInfo() {
        return @{
            Description      = 'Manages the existence and contents of UTF8-encoded text files.'
            Version          = '1.2.0'
            SchemaUri        = 'tailspintoys.com/schemas/dsc/resources/utf8file.schema.json'
            DocsUri          = 'tailspintoys.com/docs/dsc/resources/utf8file'
            UseSyntheticTest = $true
        }
    }
    static [hashtable] GetResourcePropertyInfo() {
        return @{
            Path = @{
                Key         = $true
                Required    = $true,
                Description = @'
The path to the file the instance manages. The path can be absolute
or relative. Relative paths are resolved against the current working
directory when you invoke the resource.
'@
            }
            Exist = @{ StandardProperty = $true }
            Content = @{
                Description = 'The contents of the file.'
            }
            Length = @{
                ReadOnly   = $true
                JsonSchema = @'
{
    "title": "File length",
    "description": "The size of the file in bytes.",
    "type": "integer",
    "minimum": 0
}
'@
            }
            CreationTimeUtc = @{
                ReadOnly    = $true
                Description = 'Defines when the file was created.'
            }
            LastWriteTimeUtc = @{
                ReadOnly    = $true
                Description = 'Defines when the file was last updated.'
            }
            CreateParentFolders = @{
                WriteOnly   = $true
                Description = "Indicates whether the resource should create parent folders if they don't exist."
            }
        }
    }

    static [hashtable] GetResourceMethodInfo() {
        return @{
            # No entry for test because the resource uses synthetic test
            Set = @{
                ImplementsPreTest = $true
                HandlesExist      = $true
            }
        }
    }
  #endregion Static metadata methods
  #region    Resource properties
    [string]   $Path
    [bool]     $Exist
    [int]      $Length
    [datetime] $CreationTimeUtc
    [datetime] $LastWriteTimeUtc
    [bool]     $CreateParentFolders = $true

  #endregion Resource properties
  #region    Resource methods
    [Utf8File] Get() {
        # Implementation
    }
    
    # Don't implement test, rely on synthetic test behavior

    [Utf8File] Set() {
        # Implementation
    }
    
    # Support the delete method
    [void] Delete() {
        # Implementation
    }
  #endregion Resource methods
}

This option is less convenient for resource authors, but we could also support per-member static methods or similar other solutions. I propose using static methods rather than properties, because the static properties can't always be guaranteed to return their initial values.

Feature: Support v3 semantics with static methods, overloads, or new instance methods

Currently, any PSDSC resource with the [DscResource()]` attribute must implement the old interface for resources:

• The Get() method must return an instance of the resource class.
• The Test() method must return a boolean value.
• The Set() method must have a return type of [void].

For PSDSC-compatible resources, we have a few options for addressing these limitations:

• Define overloads for the instance methods that take a parameter, like DscVersion and apply the differing semantics based on that value. More consideration is required for what the parameter or parameters should be.
• Define static methods that perform the function, taking an instance as the only parameter. In this case, PSDSC would call the instance method and DSC would call the static method.
• Define new method names, like <Verb>Instance, for DSC. PSDSC would continue to call the existing instance methods.

This problem primarily effects being able to return better information from Test, the changed state from Set, and support for WhatIf mode. With both the static method and new methods, authors could use those methods as the primary implementation path and call them from the existing methods, munging their output (or ignoring it) as needed.

Feature: Get-DscResourceInfo command

The module could provide a command that finds and inspects resources implemented in PowerShell and returns that metadata to the pipeline. This would make it easier for users to inspect resources.

The data should include as much information as possible that is normally part of a resource manifest. It should also indicate whether the resource is compatible with PSDSC and whether it explicitly supports DSC's semantics.

Feature: *-ModuleDscInfo commands

The module could provide a Get-ModuleDscInfo command that inspects the private data for a module manifest to return information about the DSC resources it includes. It would also be useful to be able to automatically update that data based on the resources published within the module, so authors don't need to maintain it manually, and to validate the data.

There are alternatives to this approach, like generating a data file separate from the manifest itself, that may be worth considering. This could also simplify the adapter's role in discovering and caching resource information.

Feature: Get-DscHelp command

With a defined model for documenting resources, we could provide a command that surfaces the resource documentation for users, similar to the Get-Help command. This would make it much easier for users and configuration authors, who currently need to discover and read the documentation for every resource manually.

The command should support viewing the help in the terminal and online.

Feature: Assert-DscResource command

The module could provide a command that inspects a PowerShell class to determine whether it's valid for use in DSC and PSDSC, and whether it follows best practices. It would actually be invoking shared pester tests, using the command parameters to select the tests to run and passing the class definition.

We could also provide PSScriptAnalyzer rules for resources and surface that information, possibly with a second command.