Merge branch 'master' of https://github.com/nus-cs3281/2024

students/Arif-Khalid/knowledge.md

@@ -91,3 +91,41 @@ Code quality is always important but is especially so when there are so many peo
 I learned about code quality through analysing the responses of seniors to my own pull requests as well as other's pull requests, supplementing my knowledge by reading articles on code quality both generally and specific to web development
 * Inspection of pull requests gave me understanding of what is good quality code and what is considered bad along with the reasoning behind those decisions
 * Articles online provided me with more general guidelines pertaining to code quality in large projects, helping fill in the gaps that I didnt encounter in PR reviews
+
+### Testing
+Testing is another important part of any project as it reduces the occurrence of major errors and bugs throughout development. With little prior experience in testing, I sought to learn more about it and apply it in WATcher.
+* Jasmine
+  * A testing framework for javascript
+  * Clean and intuitive syntax
+  * Suite of functionality developed over many years
+  * I learned Jasmine through looking through test cases in CATcher and WATcher, along with reading its official documentation
+    * ``describe(string, function)`` houses related specs labeled by string and defined as function
+    * ``it(string, function)`` defines a spec labeled by string and defined as function
+    * ``expect(any).toBeFalse`` defines an expectation of ``any``. There are a large number of matchers for any possible comparison
+    * ``beforeEach(function)`` defines a function to be called before each of the specs in this describe block
+    * ``createSpyObj(string, string[])`` creates a spy object that acts as a stub for classes that are depended on by what is being tested. Spies can track calls to it and all arguments
+* Test case design
+  * Boundary Value Analysis and equivalence partitioning
+    * Boundary value analysis is a technique where tests are designed to include representatives of boundary values in a range
+    * Equivalence partitioning is a technique where input data is partitioned into units of equivalent data for which tests can be written
+    * These techniques allow for a smaller number of tests to be written, for essentially the same amount of coverage
+      * This is because inputs which would fail/pass for the same reason, such as being an input of an invalid type, are grouped as a single or only a few test cases.
+      * The alternative would be to create tests for each input type in this example, straining developer resources for not much benefit
+  * Testing for behaviour
+    * A common mistake is to test for implementation rather than behaviour
+    * This would result in failed test cases when implementation changes even though the resulting behaviour, what the user would experience, remains the same
+    * Test cases should test for what the result is versus what the implementation is
+    * An example would be testing whether a variable changes in component A correctly vs testing what other components receive from component A after the change
+    * A developer might edit the implementation of component A so the variable no longer changes, however the accurate behaviour of emission to other components remains the same and the test cases should not fail
+  * Testing coverage
+    * Test coverage is how much of the code has actually been ran through during testing
+      * Function/method coverage : based on functions executed e.g., testing executed 90 out of 100 functions
+      * Statement coverage : based on the number of lines of code executed e.g., testing executed 23k out of 25k LOC
+      * Decision/branch coverage : based on the decision points exercised e.g., an if statement evaluated to both true and false with separate test cases during testing is considered 'covered'
+      * Condition coverage : based on the boolean sub-expressions, each evaluated to both true and false with different test cases
+    * A good future implementation would be to implement code coverage as a github action report when making pull requests to main
+    * At the very least, all public functions of a class should be uniquely tested in order to verify behaviour seen by other components
+I learned about testing web applications through Nereus, reading Jasmine documentation, articles and youtube videos about testing and the [CS2113 website](https://nus-cs2113-ay2324s2.github.io/website/index.html)
+* Nereus imparted knowledge of testing which helped me understand the core fundamentals, allowing me to more quickly pick up the technique as I learnt, especially the test case implementation
+* The Jasmine documentation gave me confidence in creating my own test cases for unique behaviour such as changing routes in testing
+* Youtube videos, articles and the CS2113 website helped me to understand and implement test case design techniques to create comprehensive and well designed test cases

students/Arif-Khalid/progress.md

@@ -1,3 +1,17 @@
+# Key Contributions
+* Actively participated in submitting and resolving issues found, implementing features and team discussions
+* Focus on bug fixes that hurt the usability of CATcher and WATcher
+* Reviewed PRs constructively with focus on scalability and code style of the existing code base
+* Update CI/CD workflow, specifically improving automation by using an on push deployment and a release drafter template and bot which formatted and displayed release changes  
+  [Automatic deployment #272](https://github.com/CATcher-org/WATcher/pull/272), [Release changelog automation #285](https://github.com/CATcher-org/WATcher/pull/285)
+* Revamp and updates to filter system of WATcher
+  * Adopted a centralised service and observer pattern behaviour  
+    [Refactor certain filters into its own service #259](https://github.com/CATcher-org/WATcher/pull/259), [Refactor sorting #261](https://github.com/CATcher-org/WATcher/pull/261), [Refactor milestone filters #264](https://github.com/CATcher-org/WATcher/pull/264)
+  * Update filters to be stored into and restored from url  
+    [Add filters to url #314](https://github.com/CATcher-org/WATcher/pull/314)
+  * Allow filters to be kept across repo changes  
+    [Keep filters when switching repo #281](https://github.com/CATcher-org/WATcher/pull/281)
+
 | Week | Achievements |
 | ---- | ------------ |
 | 2 | Merged PR: [Hide 0 issue columns #223](https://github.com/CATcher-org/WATcher/pull/223) |
@@ -19,10 +33,36 @@
 | 7 | Merged PR: [Automatic deployment #272](https://github.com/CATcher-org/WATcher/pull/272) |
 | 7 | Submitted Issue: [Release changelog automation #273](https://github.com/CATcher-org/WATcher/issues/273) |
 | 7 | Submitted Issue: [Save milestones by name #283](https://github.com/CATcher-org/WATcher/issues/283) |
-| 7 | Pending PR: [Keep filters when switching repo #281](https://github.com/CATcher-org/WATcher/pull/281) |
-| 7 | Pending PR: [Release changelog automation #285](https://github.com/CATcher-org/WATcher/pull/285) |
 | 8 | Submitted Issue: [Incorrect numbering in user-workflow #35](https://github.com/CATcher-org/catcher-org.github.io/issues/35) |
+| 8 | Submitted Issue: [Add tests for filters service #295](https://github.com/CATcher-org/WATcher/issues/295) |
 | 8 | Reviewed PR: [Update Design page #14](https://github.com/CATcher-org/WATcher-docs/pull/14) |
-| 8 | Merged PR: [Refactor milestones to save by name #289](https://github.com/CATcher-org/WATcher/pull/289) |
 | 8 | Reviewed PR: [Enable pre push hook for npm run test #288](https://github.com/CATcher-org/WATcher/pull/288) |
 | 8 | Reviewed PR: [Remove sorting by assignees in issue sorter #286](https://github.com/CATcher-org/WATcher/pull/286) |
+| 8 | Merged PR: [Refactor milestones to save by name #289](https://github.com/CATcher-org/WATcher/pull/289) |
+| 8 | Pending PR: [Update user workflow #12](https://github.com/CATcher-org/WATcher-docs/pull/12) |
+| 9 | Submitted Issue: [Label filter bar test case error #305](https://github.com/CATcher-org/WATcher/issues/305) |
+| 9 | Reviewed PR: [Hide redundant column pagination #309](https://github.com/CATcher-org/WATcher/pull/309) |
+| 9 | Reviewed PR: [Status filter checkboxes #310](https://github.com/CATcher-org/WATcher/pull/310) |
+| 9 | Merged PR: [Keep filters when switching repo #281](https://github.com/CATcher-org/WATcher/pull/281) |
+| 9 | Merged PR: [Release changelog automation #285](https://github.com/CATcher-org/WATcher/pull/285) |
+| 10 | Reviewed PR: [Integrate Grouping Service #313](https://github.com/CATcher-org/WATcher/pull/313) |
+| 10 | Merged PR: [Keep milestones when switching repo #311](https://github.com/CATcher-org/WATcher/pull/311) |
+| 10 | Merged PR: [Add filters to url #314](https://github.com/CATcher-org/WATcher/pull/314) |
+| 11 | Submitted Issue: [Default preset view is custom when it should be currently active #333](https://github.com/CATcher-org/WATcher/issues/333) |
+| 11 | Submitted Issue: [Remove quotations from filters in url #340](https://github.com/CATcher-org/WATcher/issues/340) |
+| 11 | Submitted Issue: [Showing of preset views before selecting a repo](https://github.com/CATcher-org/WATcher/issues/352) |
+| 11 | Merged PR: [Fix default preset view #334](https://github.com/CATcher-org/WATcher/pull/334) |
+| 11 | Merged PR: [Remove quotation marks from url #345](https://github.com/CATcher-org/WATcher/pull/345) |
+| 11 | Reviewed PR: [Include groupby params in url #319](https://github.com/CATcher-org/WATcher/pull/319) |
+| 11 | Reviewed PR: [Add preset views #320](https://github.com/CATcher-org/WATcher/pull/320) |
+| 11 | Reviewed PR: [Update repo on back and forward navigation #322](https://github.com/CATcher-org/WATcher/pull/322) |
+| 11 | Reviewed PR: [Reset GroupingContextService only if "keep filter" is selected #324](https://github.com/CATcher-org/WATcher/pull/324) |
+| 11 | Reviewed PR: [Create release 1.2.0 #327](https://github.com/CATcher-org/WATcher/pull/327), Reviewed PR: [Resolve conflicts for 1.2.0 #329](https://github.com/CATcher-org/WATcher/pull/329), Reviewed PR: [Deploy V1.2.0 #331](https://github.com/CATcher-org/WATcher/pull/331) |
+| 11 | Reviewed PR: [Create release V1.2.1 #349](https://github.com/CATcher-org/WATcher/pull/349), Reviewed PR: [Deploy V1.2.1 #350](https://github.com/CATcher-org/WATcher/pull/350) |
+| 11 | Reviewed PR: [Optimise Github API calls #360](https://github.com/CATcher-org/WATcher/pull/360) |
+| 12 | Submitted Issue: [Reset of sort filter after some time #372](https://github.com/CATcher-org/WATcher/issues/372) |
+| 12 | Merged PR: [Show preset view only when repo is set #355](https://github.com/CATcher-org/WATcher/pull/355) |
+| 12 | Merged PR: [Fix top and bottom shadow of columns #357](https://github.com/CATcher-org/WATcher/pull/357) |
+| 12 | Merged PR: [Create release V1.2.2 #364](https://github.com/CATcher-org/WATcher/pull/364), Merged PR: [Deploy V1.2.2 #365](https://github.com/CATcher-org/WATcher/pull/365) |
+| 12 | Merged PR: [Fix reset of filters on label fetch #374](https://github.com/CATcher-org/WATcher/pull/374) |
+| 13 | Merged PR: [Add test cases for filters service #312](https://github.com/CATcher-org/WATcher/pull/312) |

students/EuniceSim142/knowledge.md

@@ -1,11 +1,54 @@
 ### CS3282
-(outline:)
-- SQL Injection
-    - Hibernate (HQL) --> vulnerabilities that may result in SQLi
-    - Stored procedures / Parameterized queries
-- Testing: E2E, integration tests (for new sql actions)
-
-- ARF
+#### SQL Injection (with ORM: Hibernate)
+After v9 migration from NoSQL to SQL database, we have to protect the new database from SQL injection attacks. 
+
+While working on this project, we had to evaluate if our backend which uses Hibernate ORM. I have previous experience with running SQLi attacks from a computer security introductory module and so had some knowledge of how to do simple testing for it, but this gave me a great chance to understand how injection attacks on modern servers are tested for and prevented.
+
+However, since our backend uses Hibernate ORM, I had to also research on how malicious attackers will attack a Hibernate backend. Hibernate increases our attack surface, so not only do we have to test for SQLi attacks written vulnerable SQL queries, we also have to test for SQLi attacks on HQL (hibernate query language) as well as attacks on hibernate itself. 
+
+Based on the research conducted, we have decided the following:
+1. **SQLi**: Hibernate does us a favour as its Criteria API uses parameterized queries with prepared statements, which isolates the SQL code (which is written by developers) from data (provided by users) without restricting users. We will still write unit test cases (at database layer) to check that passing in a parameter with a SQL injection string fails, but we fortunately do not have to add any additional functionality.
+2. **Attacks on Hibernate**: This is when an attacker specifically targets the hibernate architecture / some vulnerability in Hibernate. This is likely a low priority risk -- with how large-scale Hibernate is used, any vulnerability here would urgently be patched by Hibernate developers (we can find these from CVE database). As long as we keep up-to-date on hibernate versions (ie. upgrade if there is a an important update / bug fix for potential attack on hibernate), which our package manager can help with, we should be ok.
+3. **HQLi**: Taking advantage of vulnerabilities in the Hibernate query language to execute an injection attack. Conclusion: These attacks requires a VERY advanced level of knowledge about HQLi, which is not the most popular language. Additionally, some attacks shown seem to rely on the developer using unsafe methods (eg. createQuery but with string concatenation of the query). 
+- [Video: good understanding of ORM Injection attacks (Very sophisticated attacks on HQLi)](https://www.youtube.com/watch?v=DKEwWy043WI)
+- [Source (Summary of HQLi attacks)](https://www.sonarsource.com/blog/exploiting-hibernate-injections/)
+
+From this project and the research we had to do before discussing ways to move forward, I learned a lot about SQL injection attacks as well as some additional information about our hibernate database:
+- Most modern applications nowadays have AT LEAST be some minimal level of protection for their database / backend servers. The basic SQLi attacks we learn in university will likely not work in real life in isolation. Often, advanced hackers use SQLi attacks in combination with many other techniques to gain access to a server. This was a great experience to understand how SQLi is foundational knowledge for these attacks and how to build upon it to create actual attacks (eg, to modify to use on ORM servers, tricks to bypass certain validation technqiues).
+- Attacks on ORM -- Hibernate Query Language Injection attacks. 
+- Parameterized queries and prepared statements (used in Hibernate's Criteria API)
+- Hibernate: configured hibernate to print the SQL queries the hibernate will generate and use (simulates how a real attacker may clone our repo and make modifications to local configs to observe the queries the orm will create and use)
+
+#### Testing: Migrating E2E Tests
+- E2E Tests: I migrated a few E2E tests for v9 migration: mainly updated the tests to load SQL entities if they have been migrated at that point in v9 development, and for other entities that have not yet migrated, continue using the NoSQL entities. 
+- SQL Injection Tests: Added tests at the database layer to test fields which accept long strings with little-to-no validations. Ensure that hibernate does not treat the data provided as SQL code.
+- Integration tests: I've previously done this during data migration (modifying or writing new integration test cases), but also added new tests for ARF (Account Request Form).
+
+#### Feature Design
+In CS3281, I worked mainly on database migration and had very minimal work on a new feature. Even during the orientation where we technically did work on a feature, it was not merged in and there was very little discussions on the design -- the main objective was to gain an understanding of how the application was structured. 
+
+I gained an excellent idea of how Teammates does Design planning as well as effort estimation and planning for timelines when working on ARF (Account request form):
+1. Initial test design: database / API endpoint design, frontend / UI design, test case design, etc.
+2. Communication with stakeholders (ie. admin): frequent communication with main user(s) of ARF and integrating their requests / feedback into our design (eg. what should be allowed depending on entity's state, UI preferences, etc)
+3. Peer-evaluation: not only for PR reviews, but also evaluating each others' design plans and any additional details we may have missed (eg. following REST principles, fields or entites we may have missed out on, edge cases)
+4. Weekly (Scrum?) Meetings: To keep each other up-to-date on our own progress and discuss issues encountered or possible improvements to make (which we sometimes realise when we start development).
+5. Communicating when timelines cannot be met
+6. Testing in staging / pre-release
+7. Release: being present when we start release and respond if fixes are needed and urgent. 
+
+#### Solr Searching
+ARF was my first chance working with our search engine, solr and it was an unexpectedly large hurdle. 
+
+1. Configurations & Documentation: This was my main pain point when working on this. The changed we implemented was not significant -- simple adding 2 new fields and updating the id field. However, the documentation for solr was very lacking (only about how to setup the docker container) and when there was issues with inserting items for indexing or even getting the instance up and running, there was very little debugging statements to track the source of the issue -- hence, quite a bit of effort and time was taken to trace the issues (not only for local development, but also for running tests as each environment had its own seperate config file)
+However from this, I learned a lot about solr configs (), using solr's GUI () to debug / track the state of the solr instance and using solr in general for insert + indexing, retrieving entries, sorting, etc. 
+
+2. Using solr: 
+    - Updating solr's schema (solr.sh) to add 2 new fields
+    - Updating solr's fields: initially, solr prepends all id of entries with 'java.util.UUID:' as the field was a UUID object when passed to solr for indexing.
+    - Matching with any word in the given string and evaluate the performance cost for this (as the new comments field allows for huge number of words)
+
+Resource:
+- [Apache Solr Guide](https://solr.apache.org/guide/solr/latest/getting-started/introduction.html)
 
 ---