Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* Cscttv 2132 (#24) * Add openshift config for api devel. * Modify Docker build path. * Add openshift config for identityserver devel. * Test permission change in container app folder. * Modify run time permissions * Change SDK version in Identityserver build. * Change build branch to devel. Change Rahti container image path. * Remove researchfi prefix from Openshift config. * Add README for api application. * Upgrade Nuget packages. * CSCTTV-2132_2 (#25) * Modify Openshift template and Dockerfile for Identityserver. * Modify image pull path. * Modify image pull path. * Add README in Identityserver project. * Update README.md * Update README.md * Modify Git source branch and container path in Identityserver template. * Remove Identityserver tempkey.jwk and add it in gitignore. * Change app folder permission in Identityserver container. * Do not initialize ElasticsearchService without required configuration values. * CSCTTV-2123 Elasticsearch sync (#26) * First draft of how data can be synced in Elasticsearch. * Elasticsearch connection parameters from configuration. * Add data types for Elasticsearch. * Add more data in Elasticsearch person index. * Delete tempkey.jwk * Add Elasticsearch environment variables in api README. Add identityserver's tempkey.jwk in .gitignore. * Simplify Elasticsearch service and change it to singleton instead of scoped service. * Use connection resiliency with SQL Server provider. * Always set field Modified to current timestamp when creating a new entry in database. * Add more logging in ElasticsearchService. * Improve handling of NEST error messages when adding or deleting entry in Elasticsearch. * Remove unused controller. * Use raw SQL query for getting DimUserProfile based on ORCID Id in DimPid. Clean up startup.cs dependencies. * Use raw SQL query when retrieving ORCID registered data source from DimRegisteredDataSource. * Upgrade Nuget packages in api project. * Code clean up, remove repetitive code. * Use raw sql when updating FactFieldValue table. Change created and modified timestamp to DateTime.UtcNow instead of DateTime.Now. Add namespace api.models.ProfileEditor. * Add unit tests for TtvSqlService. In TtvSqlService add own function for getting FactFieldValues FK column name based on field identifier. * Simplify queries which get data for profile editor and elasticsearch index. * When getting data for profile editor, get only those DimFieldDisplaySettings, which have at least one related FactFieldValue. * Update Elasticsearch in a background task. (#27) * When profile is created, include ORCID name automatically in profile. * In ORCID controller remove unnecessary retrieval of DimKnownPerson entity. * Remove multiple publicationIds from profile in the same api query. * Remove cached profiledata when publications are added or removed. * Use field identifier 550 with ORCID publications. When userprofile is deleted, remove cached profiledata and remove Elasticsearch entry in background task. * Enable split SQL query behaviour globally. Fix bug in ORCID publication handling. * Call SaveChangesAsync only once when ORCID data is imported. * Call SaveChangesAsync only once when adding demo data into user profile. * Modify Identityserver Javascript client configuration handling. * Add cooperation selection api (demo) (#28) * Add draft version of cooperation selection api. * Modify cooperation choice text content. * Add Swagger documentation. (#29) * Add Swagger documentation. * Simplify cooperationdemo api. Swagger documentation improvements. * Comment out Swagger, because the configuration does not work in Docker build. * Enable Swagger in development environment * Modify xml documentation generation. Modify environment handling in startup. * Add LanguageService. (#30) * Language service and name translation improvements. * Add DimUserChoice in Ttv model. Add DimUserChoice handling in CooperationChoicesController. (#31) * Compress api responses. * Profile API for adding and removing funding decisions. (#32) * Add controller and models for DimFundingDecision handling. * Add funding decision handling in user profile. * Add funding decision in demo data. * Add DimFundingDecision demo data. * Modify demo data. Modify FundingDecisionController functionality. * Simplify addFungingDecision API response. * CSCTTV-1703 research dataset handling in profile API (#33) * Add controller and models for DimResearchDataset handling. * Add relation between DimResearchDataset and FactFieldValues. Modify FundingDecisionController to use updated Ttv data model. * Add part of research dataset demo data. * Modify research dataset demo data handling. Modify demo data deletion. * Modify research dataset structure in profile data API response. * Comment out profile data query items. * Include ORCID ID in demodata source id field. * Modify research dataset demo data. * Fix bug in research dataset demodata creation. Modify user profile deletion. * Add funding decision and research dataset column names in TtvSqlService. * Optimize user profile and related data deletion. * Add department name in affiliation. Demo implementation, not production ready. (#34) * Improve handling of null values in name translation. * Restore previous functionality in user profile deletion. * Change funding decision handling to use projectId instead of funderProjectNumber. (#35) * Fix typo in funding decision property name. * Add OpenShift templates for demo version. * Change start and end date property names in funding decision item. * Add property datasetCreated in research dataset item. * Change demo version openshift build config git branch. * Comment out querying DimIdentifierlessData because of data model mismatch. * Add user profile id in cooperation item query. * Return API response immediately when there are no items to add or remove. * Add debug controller for getting debug data and statistics. * Sql model update 2021 12 (#37) * Remove log print of user profile existence check. * Add api response compression. * Ttv model update according to latest SQL database structure. Major change to handling of registered data source, work in progress. * Version which builds without errors. Elasticsearch synchronization and debug controller have some parts of code commented out. * Modify user profile creation, adding demo data and adding ORCID data. * Modify registered data source handling in profile data load. * Fix bug, which prevents deleting DimFieldDisplaySettings related to user profile. * Move profile editor data retrieval code into UserProfileService. * Modify profile data handling in Elasticsearch synchronization. * Modify SourceId field handling. Fix ORCID import bugs in DimName and DimResearcherDescription. * Handle missing name element when parsing ORCID record. * Upgrade Nuget packages. Modify DemoDataService. * Filter query to get only ORCID related entities. Remove manual adjustment of EntityState. * Get list of available FieldIdentifiers from UserProfileService. Add unit test to validate FieldIdentifier list content. * Add type code in publication and ORCID publication. (#38) * Fix bug where fromCache property was not set in ApiResponseProfileDataGet. * Take DOI from DimPublication.Doi instead of DimPublication.DoiHandle * Set ORCID member api scopes. Store and update user's ORCID access token on ORCID login. (#39) * Store ORCID tokens from Keycloak external IDP token endpoint. Add TokenService. (#40) * Add missing dependency injection in TokenService * Add Keycloak account linking. Add named http clients. Modify configuration parameters. (#41) * Require specific scope and orcid claim in controller authorization policy. (#42) * Add DuplicateHandlerService (#43) * Add DupplicateHandlerService and unit tests. Add temporary models and controller for testing purposes. * Modify DuplicateHandlerService unit tests * Logout and delete Keycloak user when user profile is deleted. (#44) * EF model update. Registered data source handling changes. (#45) * EF model scaffolded from test database on 5.5.2022. * Modify ORCID and Tiedejatutkimus.fi data source handling. * Add missing model files. * Store ORCID and TTV registered data source and organization values into a singleton service on startup. (#46) * Fixes suggested by Visual Studio code analyzer (#47) * Remove IdentityServer (#48) * Affiliation organization and department name handling in ORCID import (#49) * Add OrcidImportService. Move logic from controller to service. Modify Orcid json parser unit tests. * Add OrganizationHandlerService. Modify organization name handling in ORCID import and profile data deletion. * Modify affiliation related organization search in ORCID import. Modify affiliation deletion. * Modify affiliation department name handling. * Modify ORCID disambiguation source mapping, add unit test. * Move logic from controllers to services. (#50) * Improvements in background Elasticsearch index update (#51) * Queue background tasks according to Microsoft documentation. * Check if Elasticsearch sync feature is enabled before adding items to task queue. * Update NuGet packages (#52) * Add ” (test)” in temporary organization names * Update TTV model. Add DueDateDueTime into DimCallProgramme * Update TTV model change. PK in BrGrantedPermission * Remove demo data related code from user profile deletion. * Do not delete DimResearchDataset on profile deletion * Migrate from ASP.NET Core 5.0 to 6.0 (#53) * Add sharing API for profile editor (#54) * Add SharingController and SharingService * Profile sharing related database queries on application startup. * DimReferenceData handling when setting user profile default permissions. * Add SharingController and draft data * Add API endpoints for getting available sharing permissions and purposes. Language service bugfix. * Rename sharing purpose related model. * Add endpoints for listing, adding and removing of share permissions. * Change OKM organization id (#55) * Add Openshift template for Artifactory build. Remove unused templates. (#56) * Startup.cs code clean up. * Use interfaces in dependecy injection. (#57) * Add Serilog for logging. (#58) * Add GitHub codeql (#61) * Add GitHub action CodeQL. * Modify CodeQL GitHub action * Check only csharp in GitHub action CodeQL. * Modify log format. * Modify log message content. (#62) * CSCTTV-2724 Improvements in profile creation (#63) * Improved TTV database search when creating user profile. * Add research activity search on profile creation. * CSCTTV-2720 Improve testability (#64) * Add debug methods to create and delete a user profile to any ORCID ID in dim_pid. * Get ORCID record from public API when a specific flag is set in access token. * Fix bug in parsing of ORCID record education element * Import from ORCID only one entry of the same publication. (#65) * Enable database command logging * Use TagWith in queries to make debug log more readable. * Optimized profile data query using Dapper (#66) * Add Dapper. Get profiledata using raw sql. * Profile data person section query converted to raw sql. * Partially completed version of new profile data query using Dapper. * Add dim_education handling in new profile data query. * Add dim_publication and dim_orcid_publication handling in new profile data query. * Moved profile data SQL string into TtvSqlService. Use new profile data query in Elasticsearch index update. * Add missing ITtvSqlService.cs, which was dropped by malfunctioning IDE. * Removed old code from UserProfileService * Profile data deletion using Dapper (#67) * Initial code for deleting user profile using Dapper. Modify unit tests. Upgrade packages. * First working version of profile data deletion using Dapper. * Fix bug in deleting ORCID put codes from dim_pid. Add unit tests SQL statement creation. * Removed commented code. * Profile data response using new data structure. (#68) * New data structure in profile data response * Profile data personal section converted to new data structure. * Profile data activity section converted to new data structure. * Remove commented code and temporary modification in controller. * Modify DuplicateHandlerService unit tests. * Fix bug in Swagger documentation. Set controller route names explicitly. * Optimized profile deletion (#69) * Delete multiple IDs from dim_pid and dim_orcid_publication in the same SQL statement. * Delete multiple rows in one SQL statement using WHERE id IN condition * Update TTvSqlService unit tests. * Update field identifiers, unit tests and dependencies. * Use Automapper to create Elasticsearch data object from profile editor data object. (#70) * Limit profiledata in Elastic index to published items only. (#71) * Add property activitiesAndRewards into profiledata. Add property researchDatasets into Elasticsearch profiledata. (#72) * Update SQL query for dim_research_activity. Update profiledata contents. (#73) * Funding decision handling in SQL query and API response (#74) * Research dataset handling in SQL query and API response (#75) * Fix missing itemMeta in profile editor data. * Add API endpoint accountdelete for removing Keycloak user. (#76) * Cscttv 2799 hide national identification number (#77) * Add own folder structure for Keycloak related files. * Modify Java project naming. Add readme. * Modify Keycloak Dockerfile * Add custom Keycloak extension into build_dependencies folder. * Add OpenShift template for Keycloak devel. Modify Keycloak Dockerfile. (#78) * Rename Keycloak OpenShift template entries * Add Keycloak version number 19 in OpenShift template to avoid collision with old installation. * Fix bug in Keycloak custom provider URL. * Comment out automatic certificate generation in Keycloak container. * Add list of unique data sources in profile data and Elasticsearch index. (#79) * Add SectorId in organization data. (#80) * Add property primaryValue in Elasticsearch data. (#81) * Remove obsolete API code. Organize models. Update dependencies. (#82) * Remove DemoDataService * Reduce EntityFramework database log level. * Remove duplicate field PrimaryValue from elasticsearch model. (#83) * Remove duplicate PrimaryValue fields from Elasticsearch models. (#84) * Rename controller and route for profiledata. (#85) * Add OpenShift templates for QA and production. (#86) * Fix bug where duplicate dates are inserted during ORCID import. (#88) * Fix bug in cache key naming. (#89) * Set production CORS allowed origins. (#90) * Change ORCID and TTV data source organization names. Add UtilityService unit tests. (#91) * Comment out sharing feature related database query. (#92) * Add QA domains in allowed origins for production. (#93) * Add affiliation organization sector data. (#94) * Check that Elasticsearch index exists. Add index mapping. (#95) * Check that Elasticsearch index exists. Add Elasticsearch mapping for sector in afiiliation. * Fix bug in ElasticsearchService constructor. * Modify DimKnownPerson handling in profile creation phase. (#96) * Add role code and name in research activity. (#97) * Allow all data sources when adding TTV data based on known person dim_names (#98) * Revert allowing all data sources (#99) * Add profile settings controller and possibility to hide profile from portal. (#100) * Improved TTV data collection (#101) * New version of TTV data collection * Fix incorrect DimFieldDisplaySettings reference in TTV data collection. * Add research activity type handling in profile editor data. (#102) * Name and other name handling when collecting TTV data. (#103) * Modify Elasticsearch person index mapping (#104) * Change how funding decisions are searched during profile creation. (#105) * Web link handling in TTV data collection (#106) * Fix bug where PATCH fails on research activity. (#107) * Filter dim_names based on dim_registered_data_source.name (#108) * Trigger Elasticsearch person index update from controllers (#110) * Handle multi level organization relationship in affiliation organization name (#112) * Prevent duplicates and -1 rows in fact_contribution SQL query. (#113) * Add fields in profile editor and elasticsearch publication. (#114) * Add field authorsText to publication * Add fields JournalName, ConferenceName and ParentPublication name into publication. * ORCID import improved checking and simplified query. (#115) * Ignore possible duplicates when adding items to profile. (#116) * Make sure publication year is null instead of 0. (#117) * Entity framework model update. Scaffolded from database. (#118) * Add ORCID webhook handling (#119) * Add WebhookController. Add backgroung task for updating ORCID data. * ORCID webhook registration * Improve OrcidApiService unit tests. * Add ORCID webhook registration and unregistration API calls. * Modify configuration parameter names for ORCID webhook. Improve OrcidApiService unit tests. * Upgrade NuGet packages. (#120) * Add AdminController. Add admin commands for ORCID webhooks. Add comments. (#121) * Add admin command to update user profile in Elasticsearch (#122) * TTV model update 28.12.2022 (#123) * Add GitHub action: build and unit test (#124) * Ttv model update 30.12.2022 (#125) * Ttv model update 3.1.2023 (#126) * Cscttv 2075 opensearch logging configuration (#127) * Add custom NuGet source for local packages. Add custom OpenSearch sink for Serilog. * Structured log messages. * Change log messages to use structured format. * Structured log messages in services. * Add log message in AccountLinkController * Handle NuGet custom files in Docker build * Modify Dockerfile * Modify Dockerfile * Modify Dockerfile * Modify Dockerfile * Change log model folder name to avoid gitignore * Modify Serilog config in appsettings.json * TTV model update 10.1.2023 (#128) * TTV model update 10.1.2023 * Add missing models * Store log user information in a variable in admin controller. (#129) * Define outputTemplate for Serilog console sink. (#130) * Use monthly indexes in OpenSearch logging (#131) * CSCTTV-3117 deleted orcid data handling (#133) * Handle deletion of ORCID name, web link, researcher description and email address. * Handle deletion of ORCID keywords, external ids, educations and publications * Handle affiliation and related identifierless data deletion * Update Elasticsearch after ORCID webhook (#134) * TTV model update 16.2.2023 (#135) * Remove obsolete models (#136) * Improve ORCID webhook logging (#137) * Fix controller annotation which was breaking Swagger UI (#138) * Add LocalIdentifier in DimAffiliation. (#141) * Profile only publication and research activity (#142) * TTV model update according to CSCTTV-2855 and CSCTTV-2858 * Add missing foreing key in helper method. TTV model update 16.3.2023. * Use IsNullOrWhiteSpace to check if affiliation department name is valid in ORCID import. This prevents inserting null value into dim_identifierless_data. (#143) * Add more unit tests for UserProfileService. (#144) * Cscttv 3206 send logs to logstash using serilog http sink (#145) * Use Serilog HttpSink * Remove Serilog.Sinks.OpenSearch. Add Serilog.Sinks.Http. * Modify Serilog configuration * Modify Serilog configuration * Modify Serilog configuration. * Modify Serilog configuration. * Use custom Http client with Serilog Http sink to enable basic authentication. Add Serilog debug logging. * Cscttv 3204 ttv data update (#148) * Add background task for updating TTV data in user profile. Modify TTV data addition to be idempotent. Modify unit tests. * Convert UserProfileService.AddTtvDataToUserProfile to an idenpotent method. * Modify funding decisions SQL query to filter out existing IDs. Add more unit tests. * Use dimUserProfile.Id as an admin webhook parameter * Improve logging of TTV data addition method (#149) * Cscttv 3242 orcid activity import (#150) * Parse distinctions from ORCID record * First draft of ORCID research activity import, update and deletion * Use internal item meta types instead of field identifier values. Add profile only research activity data in profile editor api. * Add organization and department handling in ORCID research activity import. * Move repetitive code from to OrganizationHandlerService. Add mor unit tests. * ORCID invited positions import * In profile editor data for research activity: add organization name, department and sector * Add unit tests for ORCID import: memberships and services * ORCID qualification import and unit test * ORCID peer review import and unit test * Cscttv 3242 international collaboration nullable (#151) * Allow null in research activity property InternationalCollaboration, since ORCID values never have it * Add OrganizationName, DepartmentName and sector properties in Elasticsearch research activity model. * Cscttv 2856 2857 dim profile only dataset dim profile only funding decision (#152) * Add DimProfileOnlyDataset and DimProfileOnlyFundingDecision * Cscttv 3242 international collaboration nullable (#151) * Allow null in research activity property InternationalCollaboration, since ORCID values never have it * Add OrganizationName, DepartmentName and sector properties in Elasticsearch research activity model. * Affiliation type handling according to new DB schema. (#153) * Include DimNames in query so that background TTV data update can process related items. (#154) * Add TemporaryUniqueId in ProfileEditorItemMeta (#155) * Add TemporaryUniqueId in ProfileEditorItemMeta * Include DimNames in query so that background TTV data update can process related items. (#154) * Fix ProfileEditorItemMeta initialization in unit tests * Improve log messages in TTV data addition. (#156) * Improve logging in Admin controller (#157)
- Loading branch information
