NIST Devs: What can We Learn? What can we share?

[aj-stein-nist]

So @JustKuzya had a wonderful question today: I want to learn from my fellow NIST colleagues on a particular developer topic as it pertains to OSCAL. How do we share those topics and set up lunch-and-learn sharing sessions? We agreed that a public list is not a bad idea, with no commitment to us having a public meeting but list the topics publicly. NIST OSCAL staff and community members can vote, community can vote, but no plans exists to publish the content or videos of the sessions (if recorded).

At the very least, we thought it would help us share with one another, drive our learning with community feedback, and at least wrap that into our work in some (hopefully tangible) way.

For NIST staff and others: feel free to drop a topic yourself, or vote from the existing options.

[aj-stein-nist]

I am interested in learning more about XProc 1.0 for executing XSLT transformations and other tooling for orchestration. We do a little of this in our CI/CD pipelines, but I need to learn a lot more!

[aj-stein-nist]

Seems like we should start with this one in this sprint or the next sprint. More to follow.

[aj-stein-nist]

I am interested in learning more about the emerging XProc 3.0 standard, how it is different it is from XProc 1.0, good libraries to use with it, and most importantly how hard it is to "upgrade" XProc pipelines (in XPL files) from XProc version 1.0 to version 3.0.

[wendellpiez]

I have an application for usnistgov/xslt3-functions#5 that might help to elucidate.

It includes both XProc 1.0 and XProc 3.0 examples.

Let's find time for a walkthrough?

[aj-stein-nist]

Can we find time a lunch-and-learn kind of session for the team this week or next, and potentially record it?

[JustKuzya]

I would like to learn how to use github-actions:

• how to find plugins and usable modules for building from different codebases,
• test the code,
• test the deployments,
• do dynamic deployments,
• staging,
• and other github CI-CD tricks.

[nikitawootten-nist]

I'd also like to learn how the actions pipelines interact with each other!

[aj-stein-nist]

Ironically, y'all are in for a treat. I have relevant sprint work on this one, so maybe I find a way to work it in.

#1434

[aj-stein-nist]

I cannot commit to it quite yet, but I will to give a fun, small session on how to use GitHub Actions to your detriment or betterment potentially next week. :-)

[aj-stein-nist]

OK here we go from today's first session:

https://github.com/usnistgov/OSCAL/wiki/Developer-Knowledge-Base#20230421---basics-of-github-actions

[xee5ch]

NIST OSCAL staff and community members can vote, community can vote, but no plans exists to publish the content or videos of the sessions (if recorded).

Hello from the oscal.club crew. We would love if there was a way to help make community exchange of ideas, notes, code, video recordings, whatever in between, a reality to share with the community!

[aj-stein-nist]

I'd like to learn more about git, specifically how to manage making mass changes like:

• moving history between repositories.
• rewriting a batch of commits with something wrong in them (git-filter is out of style I think now?)

@nikitawootten-nist taught me about git subtree this week. Is that their only one way to do such things? Why pick one over the other?

[aj-stein-nist]

I think there would be value for the team members (and maybe the community) for: an imperative language programmer's guide to XSLT 3.0. I stumble around this stuff often and I can volunteer to teach my error-prone learning experience, and work with others as they see fit.

[aj-stein-nist]

I also have heard a number of team members express interest or a lack of familiarity around Oxygen XML IDE tooling, me included. I would love if we could share the knowledge around that, and I am willing to get use started if need be.

[wendellpiez]

As a declarative language, XSLT depends on an implementation (an XSLT processor) that is coded typically using an imperative language. So Saxon is written in Java, etc. Other XSLT engines exist on other platforms. The further you go back into XSLT the more there are to choose from - probably six or eight to choose from for XSLT 1.0, including implementations in Java, C, and Python at least.

Indeed oXygen itself depends on the 'component architecture' of commodity Java tooling supporting higher-level languages not only XSLT but also XQuery, the schema technologies, etc. etc. so these topics are connected.

I'd be happy to set aside time to talk about this with interested team.

[wendellpiez]

This thread has already listed more than one topic of interest:

1. XProc - 1.0, 3.0, tools, pipeline, applications
2. CI/CD integrations
3. XSLT - what it is and how to learn it
4. oXygen

Obviously these have interdependencies but it helps to split them apart.

[GaryGapinski]

XSpec should probably be considered. There are 443 .xspec documents in the develop branch.
There are 401 Schematron (.sch) documents.

In my opinion, folks who have not been previously been exposed to XSLT should begin with XSLT 3.0 and XPath 3.1 (particularly https://www.w3.org/TR/xpath-datamodel-31/) and ignore the prior versions.

[wendellpiez]

I agree with starting with XSLT/Xpath 3.1. Unless one is locked to an XSLT 2.0 processor, it has no interest. XSLT 1.0 is something of a specialized skill good to have in certain narrow circumstances. Like campfire cooking. You don't often need it and you can live fine without it. But there are times when a better meal than you expected on a campfire is a good thing.

XSpec and Schematron are also awesome topics, and they go with XSLT. (But I am not seeing that many Schematron or even XSpec files in the repo - what am I missing?)

[GaryGapinski]

…I am not seeing that many Schematron or even XSpec files in the repo - what am I missing?

gapinski@nuc7i7bnh:~$ cd ~/Projects/GitHub/usnistgov/OSCAL/
gapinski@nuc7i7bnh:~/Projects/GitHub/usnistgov/OSCAL$ git status
On branch develop
Your branch is up to date with 'origin/develop'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   build/metaschema (new commits)
	modified:   docs/themes/uswds (new commits)

no changes added to commit (use "git add" and/or "git commit -a")
gapinski@nuc7i7bnh:~/Projects/GitHub/usnistgov/OSCAL$ find . -name '*.xspec' | nl | tail
   434	./src/utils/util/resolver-pipeline/testing/3_merged/merge-custom.xspec
   435	./src/utils/util/resolver-pipeline/testing/3_merged/merge.xspec
   436	./src/utils/util/resolver-pipeline/testing/4_modified/modify.xspec
   437	./src/utils/util/resolver-pipeline/testing/5_finished/finish.xspec
   438	./src/utils/util/resolver-pipeline/testing/2_metadata/version-util.xspec
   439	./src/utils/util/resolver-pipeline/testing/2_metadata/random-util.xspec
   440	./src/utils/util/resolver-pipeline/testing/2_metadata/metadata.xspec
   441	./src/utils/util/resolver-pipeline/testing/2_metadata/uuid-method-choice.xspec
   442	./src/specifications/profile-resolution/profile-resolution-examples/example-set.xspec
   443	./src/specifications/profile-resolution/profile-resolution-examples/full-set.xspec
gapinski@nuc7i7bnh:~/Projects/GitHub/usnistgov/OSCAL$ find . -name '*.sch' | nl | tail
   392	./build/metaschema/support/xspec/tutorial/schematron/demo-03.sch
   393	./build/metaschema/support/xspec/tutorial/schematron/quickfix.sch
   394	./build/metaschema/support/xspec/tutorial/schematron/demo-01.sch
   395	./build/metaschema/support/xspec/src/schemas/xspec.sch
   396	./build/metaschema/toolchains/xslt-M4/validate/metaschema-simple-check.sch
   397	./build/metaschema/toolchains/xslt-M4/validate/metaschema-composition-check.sch
   398	./build/metaschema/toolchains/xslt-M4/util/html-docs-check.sch
   399	./src/utils/schematron/oscal-profile.sch
   400	./src/specifications/profile-resolution/spec-checkup.sch
   401	./src/specifications/profile-resolution/example-checkup.sch
gapinski@nuc7i7bnh:~/Projects/GitHub/usnistgov/OSCAL$ 

[wendellpiez]

Interesting. I see Schematrons there being reported from build/metaschema/support/xspec/tutorial ... surely not ours. So I wonder what the numbers look like with submodules split out. That submodule is likely to have plenty of XSpec in it too.

[GaryGapinski]

Much less subordinate to just src:

gapinski@nuc7i7bnh:~/Projects/GitHub/usnistgov/OSCAL$ find src -name '*.xspec' | nl | tail
     7	src/utils/util/resolver-pipeline/testing/3_merged/merge-custom.xspec
     8	src/utils/util/resolver-pipeline/testing/3_merged/merge.xspec
     9	src/utils/util/resolver-pipeline/testing/4_modified/modify.xspec
    10	src/utils/util/resolver-pipeline/testing/5_finished/finish.xspec
    11	src/utils/util/resolver-pipeline/testing/2_metadata/version-util.xspec
    12	src/utils/util/resolver-pipeline/testing/2_metadata/random-util.xspec
    13	src/utils/util/resolver-pipeline/testing/2_metadata/metadata.xspec
    14	src/utils/util/resolver-pipeline/testing/2_metadata/uuid-method-choice.xspec
    15	src/specifications/profile-resolution/profile-resolution-examples/example-set.xspec
    16	src/specifications/profile-resolution/profile-resolution-examples/full-set.xspec
gapinski@nuc7i7bnh:~/Projects/GitHub/usnistgov/OSCAL$ find src -name '*.sch' | nl | tail
     1	src/utils/schematron/oscal-profile.sch
     2	src/specifications/profile-resolution/spec-checkup.sch
     3	src/specifications/profile-resolution/example-checkup.sch
gapinski@nuc7i7bnh:~/Projects/GitHub/usnistgov/OSCAL$ 

though I thinkoscal-cli incorporates some Schematron from an unknown source.

[wendellpiez]

@GaryGapinski crossing threads ... check out usnistgov/xslt3-functions#5 would be interested in your thoughts etc.

[GaryGapinski]

That repository is not visible to me.

[wendellpiez]

Hm, shouldn't that repo be public? @aj-stein-nist ?

[aj-stein-nist]

It has been private until it is marked public, are we ready now?

[wendellpiez]

Yes -- I just made it public, thanks!

[aj-stein-nist]

@nikitawootten-nist offered to do a talk on git commit signing, for OSCAL and beyond, colleagues can feel free to vote it up and prepare for an upcoming lunch and learn session.