From a0c0917ce96bd76f31ba6bd0d3b816d70c77c61f Mon Sep 17 00:00:00 2001 From: Scott M Stark Date: Wed, 7 Sep 2022 10:38:50 -0500 Subject: [PATCH 1/7] Update what changes are allowed in a TCK release. https://github.com/jakartaee/specification-committee/issues/63 Signed-off-by: Scott M Stark --- content/committees/specification/tckprocess/index.md | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/content/committees/specification/tckprocess/index.md b/content/committees/specification/tckprocess/index.md index 8115714099..1d46bb8afe 100644 --- a/content/committees/specification/tckprocess/index.md +++ b/content/committees/specification/tckprocess/index.md @@ -436,13 +436,7 @@ is not a mechanism to attempt to handle implementation specific issues. ### How Tests May be Added to a TCK {#_how_tests_may_be_added_to_a_tck} -The only time tests may be added to a TCK are in a major or minor -release. A service release which updates the exclude list MUST not have -test additions or updates (see exception below for addressing newer Java SE versions). - -### How tests may be Updated to address newer Java SE versions {#_how_tests_may_be_updated_for_new_java_se_versions} - -A service release may update tests to work with newer Java SE versions. +Tests may be added, updated, removed or excluded as the specification team sees fit. ## Process for releasing a point revision of a TCK From 9da20ad18360f7f0a0b0a35b4c3b1342faccff15 Mon Sep 17 00:00:00 2001 From: Scott M Stark Date: Wed, 19 Oct 2022 08:52:05 -0500 Subject: [PATCH 2/7] Update to wording based on last spec committee meeting discussions Signed-off-by: Scott M Stark --- content/committees/specification/tckprocess/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/committees/specification/tckprocess/index.md b/content/committees/specification/tckprocess/index.md index 1d46bb8afe..a288216d90 100644 --- a/content/committees/specification/tckprocess/index.md +++ b/content/committees/specification/tckprocess/index.md @@ -436,7 +436,7 @@ is not a mechanism to attempt to handle implementation specific issues. ### How Tests May be Added to a TCK {#_how_tests_may_be_added_to_a_tck} -Tests may be added, updated, removed or excluded as the specification team sees fit. +Tests may be added, updated, removed or excluded as the specification team sees fit in major or minor updates. In service releases, specification teams should only make changes that are documented in TCK challenge issues, and service release notes should document which challenges have been addressed. ## Process for releasing a point revision of a TCK From a66d894d748e5ca3927d0cff69222980129a2ccd Mon Sep 17 00:00:00 2001 From: Scott M Stark Date: Wed, 19 Oct 2022 08:54:55 -0500 Subject: [PATCH 3/7] Update section title Signed-off-by: Scott M Stark --- content/committees/specification/tckprocess/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/committees/specification/tckprocess/index.md b/content/committees/specification/tckprocess/index.md index a288216d90..5a5e9f7045 100644 --- a/content/committees/specification/tckprocess/index.md +++ b/content/committees/specification/tckprocess/index.md @@ -434,7 +434,7 @@ Handling](https://www.eclipse.org/projects/dev_process/#6_5_Grievance_Handling) procedure SHOULD be followed to escalate the resolution. Note that this is not a mechanism to attempt to handle implementation specific issues. -### How Tests May be Added to a TCK {#_how_tests_may_be_added_to_a_tck} +### How Tests May be Added or Changed in a TCK {#_how_tests_may_be_added_to_a_tck} Tests may be added, updated, removed or excluded as the specification team sees fit in major or minor updates. In service releases, specification teams should only make changes that are documented in TCK challenge issues, and service release notes should document which challenges have been addressed. From 1fea5c22a25e4da1e9affc20e49e3b3df47e8603 Mon Sep 17 00:00:00 2001 From: Andrew Pielage Date: Fri, 21 Oct 2022 15:13:21 +0100 Subject: [PATCH 4/7] Merge branch 'add-language-allowing-test-modification-after-challenge' of https://github.com/scottkurz/jakarta.ee Signed-off-by: Andrew Pielage --- .../specification/tckprocess/index.md | 30 +++++++++++++++++-- 1 file changed, 28 insertions(+), 2 deletions(-) diff --git a/content/committees/specification/tckprocess/index.md b/content/committees/specification/tckprocess/index.md index 5a5e9f7045..5257325d66 100644 --- a/content/committees/specification/tckprocess/index.md +++ b/content/committees/specification/tckprocess/index.md @@ -237,7 +237,29 @@ update and release of an official distribution of the TCK including the new exclude list. The associated `challenge` issue MUST be closed with an `accepted` label to indicate it has been resolved. -The specification project may approve (user) workarounds for an `accepted` TCK challenge (as alternative to excluding TCK tests). +The specification project may approve (user) workarounds for an `accepted` TCK challenge (as an alternative to excluding TCK tests). + +As another alternative to excluding a challenged test, it may be possible +to adjust the test validation logic to "expand" the validation check. +E.g. if a test expects a value "A1" for a specific variable "x", but a challenge +is raised arguing that the specification language actually allows for either +values "A1" and "A2" (but no other values) to be valid values for "x", then +it could be a valid course of action to modify the challenged test to allow +either "A1" OR "A2" for "x". + +Since this line of thinking might be applied to cases that aren't quite as +straightfoward as this example, care should be taken when using this approach. +A particular danger is that an implementation that has already demonstrated compliance +before the challenge was raised might actually not pass the new, modified test. + +To limit the confusion and additional work such a scenario would cause, if +there is already at least one certified compatible implementation before the challenge, +the new, modified TCK should be run against at least one such implementation (and ideally all of them) +before the changes are published, released, and finalized. + +If such a change were released, and it was later found to cause a previously-certified implementation +to fail the new, modified test, then excluding the test would likely be the only option, and this would +require yet another, additional service release. #### Rejected Challenges and Remedy {#_rejected_challenges_and_remedy} @@ -436,7 +458,11 @@ is not a mechanism to attempt to handle implementation specific issues. ### How Tests May be Added or Changed in a TCK {#_how_tests_may_be_added_to_a_tck} -Tests may be added, updated, removed or excluded as the specification team sees fit in major or minor updates. In service releases, specification teams should only make changes that are documented in TCK challenge issues, and service release notes should document which challenges have been addressed. +Tests may be added, updated, removed or excluded as the specification team sees fit in major or minor updates. + +In service releases, specification teams should only make changes that are documented in TCK challenge issues, and +service release notes should document which challenges have been addressed. Test updates within a service release +should follow the guidance described under Accepted Challenges. ## Process for releasing a point revision of a TCK From a11871f109a720697b2da01a04a84242432f48b1 Mon Sep 17 00:00:00 2001 From: Andrew Pielage Date: Fri, 21 Oct 2022 16:15:24 +0100 Subject: [PATCH 5/7] Adjust header link to match header Signed-off-by: Andrew Pielage --- content/committees/specification/tckprocess/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/committees/specification/tckprocess/index.md b/content/committees/specification/tckprocess/index.md index 5257325d66..bb28c65181 100644 --- a/content/committees/specification/tckprocess/index.md +++ b/content/committees/specification/tckprocess/index.md @@ -456,7 +456,7 @@ Handling](https://www.eclipse.org/projects/dev_process/#6_5_Grievance_Handling) procedure SHOULD be followed to escalate the resolution. Note that this is not a mechanism to attempt to handle implementation specific issues. -### How Tests May be Added or Changed in a TCK {#_how_tests_may_be_added_to_a_tck} +### How Tests May be Added or Changed in a TCK {#_how_tests_may_be_added_or_changed_in_a_tck} Tests may be added, updated, removed or excluded as the specification team sees fit in major or minor updates. From ff4ef523d26c28f66dd64289c5526eb8c3180122 Mon Sep 17 00:00:00 2001 From: Andrew Pielage Date: Mon, 31 Oct 2022 14:28:30 +0000 Subject: [PATCH 6/7] Fix typo Signed-off-by: Andrew Pielage --- content/committees/specification/tckprocess/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/committees/specification/tckprocess/index.md b/content/committees/specification/tckprocess/index.md index bb28c65181..36ab56aff0 100644 --- a/content/committees/specification/tckprocess/index.md +++ b/content/committees/specification/tckprocess/index.md @@ -248,7 +248,7 @@ it could be a valid course of action to modify the challenged test to allow either "A1" OR "A2" for "x". Since this line of thinking might be applied to cases that aren't quite as -straightfoward as this example, care should be taken when using this approach. +straightforward as this example, care should be taken when using this approach. A particular danger is that an implementation that has already demonstrated compliance before the challenge was raised might actually not pass the new, modified test. From 567f7363cc8ec02cc4852b7cd4a7eca4f78e7e9a Mon Sep 17 00:00:00 2001 From: Scott M Stark Date: Wed, 2 Nov 2022 18:04:06 -0600 Subject: [PATCH 7/7] Update version to 1.3 Signed-off-by: Scott M Stark --- content/committees/specification/tckprocess/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/committees/specification/tckprocess/index.md b/content/committees/specification/tckprocess/index.md index 5a5e9f7045..cc9b4641dd 100644 --- a/content/committees/specification/tckprocess/index.md +++ b/content/committees/specification/tckprocess/index.md @@ -1,5 +1,5 @@ --- -title: "Jakarta EE TCK Process 1.2" +title: "Jakarta EE TCK Process 1.3" date: "2022-02-07T00:00:00+00:00" aliases: - committees/specification/TCKProcess/