Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Index configuration tool] Add support for insecure HTTPS endpoint #218

Merged
merged 3 commits into from
Jun 28, 2023

Conversation

kartg
Copy link
Member

@kartg kartg commented Jun 27, 2023

Description

This commit adds support for parsing the "insecure" flag from the Data Prepper YAML. This translates to a "verify" flag that is passed to the Python requests library. This commit also includes unit tests for this logic.

  • Category: Enhancement

Issues Resolved

N/A

Testing

$ python -m coverage run -m unittest
.........................
----------------------------------------------------------------------
Ran 25 tests in 0.506s

OK

$ python -m coverage report --omit "*/tests/*"
Name                  Stmts   Miss  Cover
-----------------------------------------
index_operations.py      29      2    93%
main.py                  88      0   100%
utils.py                 13      0   100%
-----------------------------------------
TOTAL                   130      2    98%

Check List

  • New functionality includes testing
    • All tests pass, including unit test, integration test and doctest
  • New functionality has been documented
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@codecov
Copy link

codecov bot commented Jun 27, 2023

Codecov Report

Merging #218 (c141676) into main (2807196) will increase coverage by 34.44%.
The diff coverage is n/a.

❗ Current head c141676 differs from pull request most recent head cc3f56e. Consider uploading reports for the commit cc3f56e to get more accurate results

@@              Coverage Diff              @@
##               main     #218       +/-   ##
=============================================
+ Coverage     57.07%   91.51%   +34.44%     
=============================================
  Files            54       25       -29     
  Lines          2057     1155      -902     
  Branches        174      122       -52     
=============================================
- Hits           1174     1057      -117     
+ Misses          793       83      -710     
+ Partials         90       15       -75     
Flag Coverage Δ
unittests ?

Flags with carried forward coverage won't be shown. Click here to find out more.

see 79 files with indirect coverage changes

This commit adds support for parsing the "insecure" flag from the Data Prepper YAML. This translates to a "verify" flag that is passed to the Python requests library. This commit also includes unit tests for this logic.

Signed-off-by: Kartik Ganesh <[email protected]>
Separated the error messages for source and sink validation. Also fixed the unit test for this.

Signed-off-by: Kartik Ganesh <[email protected]>
Fixed unnecessary bool() cast

Signed-off-by: Kartik Ganesh <[email protected]>
@kartg kartg force-pushed the ict-insecure-flag-support branch from c141676 to cc3f56e Compare June 28, 2023 16:01
@kartg kartg merged commit 547a5ad into opensearch-project:main Jun 28, 2023
4 checks passed
@kartg kartg deleted the ict-insecure-flag-support branch June 28, 2023 20:03
gregschohn added a commit to gregschohn/opensearch-migrations that referenced this pull request Jun 28, 2023
* main: (57 commits)
  [Index configuration tool] Add support for insecure HTTPS endpoint (opensearch-project#218)
  Refactor inner classes from ExpiringTrafficStreamMap into separate classes. One other refactorings to simplify data encapsulation between the main map itself and each ExpiringKeyQueue.
  Move ExpiringTrafficStreamMap file to a new package to accommodate refactoring in the next commit.
  Make getOrCreateNodeMap() less pessimistic and add a TODO for work that needs to be done to remove (or at least describe) an obscure and rare race condition.
  [index configuration tool] Change to parsing Data Prepper pipeline YAML as input (opensearch-project#215)
  changing argument and variable names + updating description
  run on any .py changes within the top cluster_migration_core directory
  run lint on all .py changes
  Removing accidentally included kafka.properties file
  Migrations-1150 - Enhance Setting Kafka Properties for Kafka Puller
  Disabling python related workflows for non-python directories
  Using wildcards to add coverage report files instead of adding individual files
  removing finalized
  renaming workflow + no longer double executing tests
  Introduce Copilot Deployments (opensearch-project#201)
  Wildcard isn't supported - add each cov file manually
  run gradle build instead of assemble
  Update codecov version
  Add github workflow
  Bump aws-cdk-lib in /deployment/cdk/opensearch-service-migration (opensearch-project#206)
  ...

Signed-off-by: Greg Schohn <[email protected]>

# Conflicts:
#	TrafficCapture/trafficReplayer/src/main/java/org/opensearch/migrations/replay/Accumulation.java
#	TrafficCapture/trafficReplayer/src/main/java/org/opensearch/migrations/replay/ExpiringTrafficStreamMap.java
#	TrafficCapture/trafficReplayer/src/main/java/org/opensearch/migrations/replay/CapturedTrafficToHttpTransactionAccumulator
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants