Programming exercises: Improve preliminary AI feedback

[undernagruzez]

You can test this PR only on ts1

Checklist

General

• I tested all changes and their related features with all corresponding user types on a test server.
• This is a small issue that I tested locally and was confirmed by another developer on a test server.
• Language: I followed the guidelines for inclusive, diversity-sensitive, and appreciative language.
• I chose a title conforming to the naming conventions for pull requests.

Server

• Important: I implemented the changes with a very good performance and prevented too many (unnecessary) and too complex database calls.
• I strictly followed the principle of data economy for all database calls.
• I strictly followed the server coding and design guidelines.
• I added multiple integration tests (Spring) related to the features (with a high test coverage).
• I added pre-authorization annotations according to the guidelines and checked the course groups for all new REST Calls (security).
• I documented the Java code using JavaDoc style.

Client

• Important: I implemented the changes with a very good performance, prevented too many (unnecessary) REST calls and made sure the UI is responsive, even with large data (e.g. using paging).
• I strictly followed the principle of data economy for all client-server REST calls.
• I strictly followed the client coding and design guidelines.
• Following the theming guidelines, I specified colors only in the theming variable files and checked that the changes look consistent in both the light and the dark theme.
• I added multiple integration tests (Jest) related to the features (with a high test coverage), while following the test guidelines.
• I added authorities to all new routes and checked the course groups for displaying navigation elements (links, buttons).
• I documented the TypeScript code using JSDoc style.
• I added multiple screenshots/screencasts of my UI changes.
• I translated all newly inserted strings into English and German.

Motivation and Context

We previously used individual due dates and repository locks to control when requests were sent to Athena, allowing better management of the feedback process. This has now been simplified by removing individual due dates and locks, which are no longer needed. Additionally, animations have been aligned with the default Artemis indicator for consistency. Generated feedback results are now rated by default, with a tooltip explaining that the grade is an AI assessment, subject to change. I also added sorting to the generated feedback to improve clarity.

Description

Removed the individual due date and lock mechanism for feedback requests to Athena.
Updated animations to be consistent with Artemis’ default loading indicators.
Added automatic rating of AI-generated results with a tooltip explaining that it's a preliminary grade.
Implemented sorting for generated feedback to ensure a clearer presentation of results.

Steps for Testing

Prerequisites:
1 Instructor
1 Students
1 Programming Exercise

Log in to Artemis.
Navigate to Course Administration.
Create a programming exercise, set the due date, enable manual assessment, enable grading suggestions, and select an Athena module for feedback as the instructor.
As a student, solve the programming exercise and request feedback.
Ensure the animations function as expected and that the feedback status is clearly displayed.
Verify that the feedback results can be viewed, including from the code editor.
As the instructor, set the due date to the past and assess the submission.
Ensure that the AI-generated result is no longer visible in the assessment view.
Submit the assessment, and as a student, verify that the final grade is visible.

Exam Mode Testing

Prerequisites:

1 Instructor
2 Students
1 Exam with a Programming Exercise
Log in to Artemis.
As a student, participate in the exam, solve the programming exercise, and submit.
As the instructor, assess the submission.
Ensure that the student can access the assessment without any change in functionality compared to the standard mode.

Testserver States

Note

These badges show the state of the test servers.
Green = Currently available, Red = Currently locked
Click on the badges to get to the test servers.

Review Progress

Performance Review

• I (as a reviewer) confirm that the client changes (in particular related to REST calls and UI responsiveness) are implemented with a very good performance even for very large courses with more than 2000 students.
• I (as a reviewer) confirm that the server changes (in particular related to database calls) are implemented with a very good performance even for very large courses with more than 2000 students.

Code Review

• Code Review 1
• Code Review 2

Manual Tests

• Test 1
• Test 2

Exam Mode Test

• Test 1
• Test 2

Performance Tests

• Test 1
• Test 2

Test Coverage

Info: ✅ ❌ in Confirmation (assert/expect) have to be adjusted manually, also delete trivial files!

Client

Class/File	Line Coverage	Confirmation (assert/expect)
code-editor-actions.component.ts	81.3%	✅ ❌
programming-exercise.utils.ts	94.82%	✅ ❌
participation.utils.ts	76.19%	✅ ❌
result.service.ts	91.52%	✅ ❌
result.utils.ts	90.65%	✅ ❌
updating-result.component.ts	97.05%	✅ ❌
course-exercise-details.component.ts	89.2%	✅ ❌
exercise-details-student-actions.component.ts	86.03%	✅ ❌
request-feedback-button.component.ts	89.65%	✅ ❌

Server

Class/File	Line Coverage	Confirmation (assert/expect)
Exercise.java	92%	✅ ❌
ParticipationResource.java	87%	✅ ❌
ProgrammingExerciseCodeReviewFeedbackService.java	74%	✅ ❌

Screenshots

Results timeline:

Feedback window:

Summary by CodeRabbit

Summary by CodeRabbit

Release Notes

• New Features

  • Introduced a new requestFeedback method to streamline feedback requests, improving user interaction.
  • Added a RequestFeedbackButtonComponent to enhance the user experience in the code editor.

• Bug Fixes

  • Improved error handling and messaging for feedback requests, ensuring users receive clear notifications based on their actions and submission states.

• Improvements

  • Refined logic for filtering results based on assessment types, particularly for AI assessments.
  • Enhanced participation management functionality by improving error handling and refining logic for participation-related operations.
  • Streamlined initialization processes in the user interface for better usability and clarity.
  • Updated localization for clarity, including changes to key names and messages.
  • Improved clarity and functionality of feedback request tests, ensuring they accurately reflect intended behavior under different conditions.

[github-actions]

⚠️ Unable to deploy to test servers ⚠️

The docker build needs to run through before deploying.

[github-actions]

There hasn't been any activity on this pull request recently. Therefore, this pull request has been automatically marked as stale and will be closed if no further activity occurs within seven days. Thank you for your contributions.

[coderabbitai]

Actionable comments posted: 2

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: ASSERTIVE

📥 Commits

Files that changed from the base of the PR and between 129ae1d and eff2cc5.

📒 Files selected for processing (3)

• src/main/webapp/app/exercises/programming/shared/code-editor/code-editor.module.ts (2 hunks)
• src/test/javascript/spec/integration/code-editor/code-editor-container.integration.spec.ts (2 hunks)
• src/test/javascript/spec/service/result.service.spec.ts (1 hunks)

🧰 Additional context used

📓 Path-based instructions (3)

src/main/webapp/app/exercises/programming/shared/code-editor/code-editor.module.ts (1)

Pattern src/main/webapp/**/*.ts: angular_style:https://angular.io/guide/styleguide;methods_in_html:false;lazy_loading:true;code_reuse:true;tests:meaningful;types:PascalCase;enums:PascalCase;funcs:camelCase;props:camelCase;no_priv_prefix:true;strings:single_quotes;localize:true;btns:functionality;links:navigation;icons_text:newline;labels:associate;code_style:arrow_funcs,curly_braces,open_braces_same_line,indent_4;memory_leak_prevention:true;routes:naming_schema;chart_framework:ngx-charts;responsive_layout:true

src/test/javascript/spec/integration/code-editor/code-editor-container.integration.spec.ts (1)

Pattern src/test/javascript/spec/**/*.ts: jest: true; mock: NgMocks; bad_practices: avoid_full_module_import; perf_improvements: mock_irrelevant_deps; service_testing: mock_http_for_logic; no_schema: avoid_NO_ERRORS_SCHEMA; expectation_specificity: true; solutions: {boolean: toBeTrue/False, reference: toBe, existence: toBeNull/NotNull, undefined: toBeUndefined, class_obj: toContainEntries/toEqual, spy_calls: {not_called: not.toHaveBeenCalled, once: toHaveBeenCalledOnce, with_value: toHaveBeenCalledWith|toHaveBeenCalledExactlyOnceWith}}

src/test/javascript/spec/service/result.service.spec.ts (1)

Pattern src/test/javascript/spec/**/*.ts: jest: true; mock: NgMocks; bad_practices: avoid_full_module_import; perf_improvements: mock_irrelevant_deps; service_testing: mock_http_for_logic; no_schema: avoid_NO_ERRORS_SCHEMA; expectation_specificity: true; solutions: {boolean: toBeTrue/False, reference: toBe, existence: toBeNull/NotNull, undefined: toBeUndefined, class_obj: toContainEntries/toEqual, spy_calls: {not_called: not.toHaveBeenCalled, once: toHaveBeenCalledOnce, with_value: toHaveBeenCalledWith|toHaveBeenCalledExactlyOnceWith}}

🔇 Additional comments (4)

src/main/webapp/app/exercises/programming/shared/code-editor/code-editor.module.ts (2)

26-26: LGTM: Import statement is correct.

The import statement for RequestFeedbackButtonComponent follows the coding guidelines and is correctly placed at the top of the file with other imports.

---

39-39: Verify if RequestFeedbackButtonComponent is a standalone component.

The placement of RequestFeedbackButtonComponent in the imports array is correct only if it's a standalone component. If it's a regular component, it should be in the declarations array instead.

Let's verify if it's a standalone component:

If the script confirms that RequestFeedbackButtonComponent is a standalone component, no changes are needed. Otherwise, please move it to the declarations array.

✅ Verification successful

Since the previous script did not yield results, we'll refine our search to accurately determine if RequestFeedbackButtonComponent is a standalone component.

---

RequestFeedbackButtonComponent is confirmed as a standalone component. Its placement in the imports array is correct.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check if RequestFeedbackButtonComponent is a standalone component

# Search for @Component decorator with standalone property
rg -e '@Component\s*\(\s*\{[^}]*standalone\s*:\s*true' 'src/main/webapp/app/overview/exercise-details/request-feedback-button/request-feedback-button.component.ts'

# If the above command returns a result, the component is standalone and should remain in the imports array.
# If it doesn't return a result, the component should be moved to the declarations array.

Length of output: 163

---

Script:

#!/bin/bash
# Description: Verify if RequestFeedbackButtonComponent is a standalone component by searching for 'standalone: true' within its @Component decorator.

# Search for @Component decorator
component_exists=$(rg -l '@Component' 'src/main/webapp/app/overview/exercise-details/request-feedback-button/request-feedback-button.component.ts')

if [ -n "$component_exists" ]; then
  # Search for 'standalone: true' within the @Component decorator
  standalone=$(rg -A5 '@Component\s*\(\s*\{' 'src/main/webapp/app/overview/exercise-details/request-feedback-button/request-feedback-button.component.ts' | rg 'standalone\s*:\s*true')
  
  if [ -n "$standalone" ]; then
    echo "Standalone"
  else
    echo "Not Standalone"
  fi
else
  echo "@Component decorator not found in the specified component."
fi

Length of output: 645

src/test/javascript/spec/integration/code-editor/code-editor-container.integration.spec.ts (2)

77-77: LGTM: RequestFeedbackButtonComponent import added.

The import of RequestFeedbackButtonComponent has been correctly added to the test file. This is a necessary step to include the component in the test environment.

---

127-127: LGTM: RequestFeedbackButtonComponent added to TestBed declarations.

The RequestFeedbackButtonComponent has been properly added to the declarations array in the TestBed configuration. This ensures that the component is available for use in the tests.

To ensure that the new component is being utilized in the tests, we can run the following script:

✅ Verification successful

Verification Successful: RequestFeedbackButtonComponent is properly utilized in the tests.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check if RequestFeedbackButtonComponent is used in any test cases

# Search for usage of RequestFeedbackButtonComponent in test cases
rg --type typescript 'RequestFeedbackButtonComponent' src/test/javascript/spec/integration/code-editor/code-editor-container.integration.spec.ts

Length of output: 179

---

Script:

#!/bin/bash
# Description: Check if RequestFeedbackButtonComponent is used in any test cases

# Search for usage of RequestFeedbackButtonComponent in the specific test file without specifying the type
rg 'RequestFeedbackButtonComponent' src/test/javascript/spec/integration/code-editor/code-editor-container.integration.spec.ts

Length of output: 327

[julian-christl]

reapprove code

[FelixTJDietrich]

Thanks for removing the failed feedback. Tested on TS1 and functionality looks good and works mostly as expected.

There are some improvements we can do as a follow-up:

I submitted and immediately pressed the request AI feedback button and get this:

Improvements:

• Disable the feedback button on the online editor and the exercise details if it is not an available option
• Improve error message to wait for the build to complete, or make AI requests not depend on it.

The AI Feedback does not show a score:

Improvement:

• Show a score and maybe make it differentiable from an automatic testing result

How often are you allowed to press request feedback? Do failed requests really end up with the tutor?

Possible improvements:

• Allow only one successful AI request per submission commit
• Make automatic feedback requests available for AssessmentType automatic and not route requests to the tutor