Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release 4.0.16 update #5962

Merged
merged 43 commits into from
Jan 23, 2024
Merged

Release 4.0.16 update #5962

merged 43 commits into from
Jan 23, 2024

Conversation

lghiur
Copy link
Member

@lghiur lghiur commented Jan 22, 2024

No description provided.

buger and others added 30 commits June 15, 2023 10:35
… director for all required vars (#5088)

[TT-2301] Create scope local variables for director for all required vars (#5088)

<!-- Provide a general summary of your changes in the Title above -->

## Description

Fix logger according to scope;
Add scope local variables for director to use, avoiding unexpected
behaviour/races.

<!-- Describe your changes in detail -->

## Related Issue

<!-- This project only accepts pull requests related to open issues. -->
<!-- If suggesting a new feature or change, please discuss it in an
issue first. -->
<!-- If fixing a bug, there should be an issue describing it with steps
to reproduce. -->
<!-- OSS: Please link to the issue here. Tyk: please create/link the
JIRA ticket. -->

Fixes #2980 #2952 

https://tyktech.atlassian.net/browse/TT-2301

## Motivation and Context

<!-- Why is this change required? What problem does it solve? -->

## How This Has Been Tested

<!-- Please describe in detail how you tested your changes -->
<!-- Include details of your testing environment, and the tests -->
<!-- you ran to see how your change affects other areas of the code,
etc. -->
<!-- This information is helpful for reviewers and QA. -->

## Screenshots (if appropriate)

## Types of changes

<!-- What types of changes does your code introduce? Put an `x` in all
the boxes that apply: -->

- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Refactoring or add test (improvements in base code or adds test
coverage to functionality)

## Checklist

<!-- Go over all the following points, and put an `x` in all the boxes
that apply -->
<!-- If there are no documentation updates required, mark the item as
checked. -->
<!-- Raise up any additional concerns not covered by the checklist. -->

- [ ] I ensured that the documentation is up to date
- [ ] I explained why this PR updates go.mod in detail with reasoning
why it's required
- [ ] I would like a code coverage CI quality gate exception and have
explained why

Co-authored-by: Tit Petric <[email protected]>
bump gw version

<!-- Provide a general summary of your changes in the Title above -->

## Description

<!-- Describe your changes in detail -->

## Related Issue
https://tyktech.atlassian.net/browse/TT-9187
<!-- This project only accepts pull requests related to open issues. -->
<!-- If suggesting a new feature or change, please discuss it in an
issue first. -->
<!-- If fixing a bug, there should be an issue describing it with steps
to reproduce. -->
<!-- OSS: Please link to the issue here. Tyk: please create/link the
JIRA ticket. -->

## Motivation and Context

<!-- Why is this change required? What problem does it solve? -->

## How This Has Been Tested

<!-- Please describe in detail how you tested your changes -->
<!-- Include details of your testing environment, and the tests -->
<!-- you ran to see how your change affects other areas of the code,
etc. -->
<!-- This information is helpful for reviewers and QA. -->

## Screenshots (if appropriate)

## Types of changes

<!-- What types of changes does your code introduce? Put an `x` in all
the boxes that apply: -->

- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Refactoring or add test (improvements in base code or adds test
coverage to functionality)

## Checklist

<!-- Go over all the following points, and put an `x` in all the boxes
that apply -->
<!-- If there are no documentation updates required, mark the item as
checked. -->
<!-- Raise up any additional concerns not covered by the checklist. -->

- [ ] I ensured that the documentation is up to date
- [ ] I explained why this PR updates go.mod in detail with reasoning
why it's required
- [ ] I would like a code coverage CI quality gate exception and have
explained why
…5168)

<!-- Provide a general summary of your changes in the Title above -->

## Description

<!-- Describe your changes in detail -->
This PR reverts commit
0e27df8

## Related Issue

<!-- This project only accepts pull requests related to open issues. -->
<!-- If suggesting a new feature or change, please discuss it in an
issue first. -->
<!-- If fixing a bug, there should be an issue describing it with steps
to reproduce. -->
<!-- OSS: Please link to the issue here. Tyk: please create/link the
JIRA ticket. -->
https://tyktech.atlassian.net/browse/TT-9199

## Motivation and Context

<!-- Why is this change required? What problem does it solve? -->

## How This Has Been Tested

<!-- Please describe in detail how you tested your changes -->
<!-- Include details of your testing environment, and the tests -->
<!-- you ran to see how your change affects other areas of the code,
etc. -->
<!-- This information is helpful for reviewers and QA. -->

## Screenshots (if appropriate)

## Types of changes

<!-- What types of changes does your code introduce? Put an `x` in all
the boxes that apply: -->

- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Refactoring or add test (improvements in base code or adds test
coverage to functionality)

## Checklist

<!-- Go over all the following points, and put an `x` in all the boxes
that apply -->
<!-- If there are no documentation updates required, mark the item as
checked. -->
<!-- Raise up any additional concerns not covered by the checklist. -->

- [ ] I ensured that the documentation is up to date
- [ ] I explained why this PR updates go.mod in detail with reasoning
why it's required
- [ ] I would like a code coverage CI quality gate exception and have
explained why

---------

Co-authored-by: Tit Petric <[email protected]>
Co-authored-by: Ilija Bojanovic <[email protected]>
Backport TT-9177 to 4.3.4 (#5153)

<!-- Provide a general summary of your changes in the Title above -->

## Description

<!-- Describe your changes in detail -->

## Related Issue

<!-- This project only accepts pull requests related to open issues. -->
<!-- If suggesting a new feature or change, please discuss it in an
issue first. -->
<!-- If fixing a bug, there should be an issue describing it with steps
to reproduce. -->
<!-- OSS: Please link to the issue here. Tyk: please create/link the
JIRA ticket. -->

## Motivation and Context

<!-- Why is this change required? What problem does it solve? -->

## How This Has Been Tested

<!-- Please describe in detail how you tested your changes -->
<!-- Include details of your testing environment, and the tests -->
<!-- you ran to see how your change affects other areas of the code,
etc. -->
<!-- This information is helpful for reviewers and QA. -->

## Screenshots (if appropriate)

## Types of changes

<!-- What types of changes does your code introduce? Put an `x` in all
the boxes that apply: -->

- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Refactoring or add test (improvements in base code or adds test
coverage to functionality)

## Checklist

<!-- Go over all the following points, and put an `x` in all the boxes
that apply -->
<!-- If there are no documentation updates required, mark the item as
checked. -->
<!-- Raise up any additional concerns not covered by the checklist. -->

- [ ] I ensured that the documentation is up to date
- [ ] I explained why this PR updates go.mod in detail with reasoning
why it's required
- [ ] I would like a code coverage CI quality gate exception and have
explained why

---------

Co-authored-by: Zaid Albirawi <[email protected]>

---------

Co-authored-by: Sredny M <[email protected]>
Co-authored-by: Matias <[email protected]>
Synchronizes github actions and goreleaser configs from 4-lts into
4.0.14

---------

Co-authored-by: Tit Petric <[email protected]>
<!-- Provide a general summary of your changes in the Title above -->

## Description

Made the changes that were applied in master, release 5, and
release-4-lts to keep consistency

## Related Issue

https://tyktech.atlassian.net/browse/TT-9177

## Motivation and Context

<!-- Why is this change required? What problem does it solve? -->

## How This Has Been Tested

Same as ticket description

## Screenshots (if appropriate)

## Types of changes

<!-- What types of changes does your code introduce? Put an `x` in all
the boxes that apply: -->

- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Refactoring or add test (improvements in base code or adds test
coverage to functionality)

## Checklist

<!-- Go over all the following points, and put an `x` in all the boxes
that apply -->
<!-- If there are no documentation updates required, mark the item as
checked. -->
<!-- Raise up any additional concerns not covered by the checklist. -->

- [ ] I ensured that the documentation is up to date
- [ ] I explained why this PR updates go.mod in detail with reasoning
why it's required
- [ ] I would like a code coverage CI quality gate exception and have
explained why
…ected signature from logs in HMAC (#5648)

Merging to release-4-lts: [10308] Hide expected signature from logs in HMAC (#5648)

[10308] Hide expected signature from logs in HMAC (#5648)

This PR removes the expected HMAC signature from the logs which are
printed when there is a mismatch.
manual change:

- verify github actions
- copy over goreleaser
- copy over plugin-compiler
- copy over ci/Dockerfile.std/slim

remaining changes should be automation changes (TD-1998 to track).

---------

Co-authored-by: Tit Petric <[email protected]>
Update a failing test.

Co-authored-by: Tit Petric <[email protected]>
…ss or nil pointer dereference (#5589) (#5594)

[TT-7127] Fix goplugin invalid memory address or nil pointer dereference
(#5589)

https://tyktech.atlassian.net/browse/TT-7127

Closes #4198 #4197

Signed-off-by: Chenyang Yan <[email protected]>
Co-authored-by: Chenyang Yan <[email protected]>

[TT-7127]:
https://tyktech.atlassian.net/browse/TT-7127?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

---------

Co-authored-by: Tit Petric <[email protected]>
Co-authored-by: Tit Petric <[email protected]>
…on stdlib in 3.11 (deprecated since python 3.0) (#5664)

[TT-10329] fix getargspec removal from python stdlib in 3.11 (deprecated since python 3.0) (#5664)

https://tyktech.atlassian.net/browse/TT-10329

---------

Co-authored-by: Tit Petric <[email protected]>
… new idle connections to be generated. (#5526)

Merging to release-4.3.8: [TT-9964] Prevent new idle connections to be generated. (#5526)

[TT-9964] Prevent new idle connections to be generated. (#5526)
[TT-8547] Fix websocket error proxying  (#4918)

## Description

Upgrade websocket only when server responded with 101. 
Fixes the case when websocket have own authentification, so we can proxy
this error.

## Motivation and Context

<!-- Why is this change required? What problem does it solve? -->

## How This Has Been Tested

<!-- Please describe in detail how you tested your changes -->
<!-- Include details of your testing environment, and the tests -->
<!-- you ran to see how your change affects other areas of the code,
etc. -->
<!-- This information is helpful for reviewers and QA. -->

## Screenshots (if appropriate)

## Types of changes

<!-- What types of changes does your code introduce? Put an `x` in all
the boxes that apply: -->

- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Refactoring or add test (improvements in base code or adds test
coverage to functionality)

## Checklist

<!-- Go over all the following points, and put an `x` in all the boxes
that apply -->
<!-- If there are no documentation updates required, mark the item as
checked. -->
<!-- Raise up any additional concerns not covered by the checklist. -->

- [ ] I ensured that the documentation is up to date
- [ ] I explained why this PR updates go.mod in detail with reasoning
why it's required
- [ ] I would like a code coverage CI quality gate exception and have
explained why
Mark legacy graphql test as flaky to pass CI

https://tyktech.atlassian.net/browse/TT-10432

Co-authored-by: Tit Petric <[email protected]>
…efore handling any additional logic (#5345) (#5736)

[TT-9327] Decoding the URL request first, before handling any additional
logic (#5345)

<!-- Provide a general summary of your changes in the Title above -->
this path works: /payment-intents
but this path doesn't: /payment%2Dintents

Encoded URLs aren't being rewritten when URL rewrite is applied.

One edge case scenario that could break backwards compatibility (as
described by @buger ), is that users can rely on escaped characters, and
try to match them from the the url rewrite rules.

In order to accomodate that, we are running url rewrite middleware
twice:
- once on the raw path
- if transformations are failing and the url contains encoded
characters, then we run it second time, with decoded URL

<!-- Describe your changes in detail -->


## Related Issue

<!-- This project only accepts pull requests related to open issues. -->
<!-- If suggesting a new feature or change, please discuss it in an
issue first. -->
<!-- If fixing a bug, there should be an issue describing it with steps
to reproduce. -->
<!-- OSS: Please link to the issue here. Tyk: please create/link the
JIRA ticket. -->

## Motivation and Context

<!-- Why is this change required? What problem does it solve? -->

## How This Has Been Tested

Unit test and manually

## Screenshots (if appropriate)

## Types of changes

<!-- What types of changes does your code introduce? Put an `x` in all
the boxes that apply: -->

- [√ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Refactoring or add test (improvements in base code or adds test
coverage to functionality)

## Checklist

<!-- Go over all the following points, and put an `x` in all the boxes
that apply -->
<!-- If there are no documentation updates required, mark the item as
checked. -->
<!-- Raise up any additional concerns not covered by the checklist. -->

- [ ] I ensured that the documentation is up to date
- [ ] I explained why this PR updates go.mod in detail with reasoning
why it's required
- [ ] I would like a code coverage CI quality gate exception and have
explained why

[TT-9327]:
https://tyktech.atlassian.net/browse/TT-9327?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

---------

Co-authored-by: Laurentiu Ghiur <[email protected]>
Co-authored-by: Tit Petric <[email protected]>
[TT-9761] Add reload interval (#5418)

<!-- Provide a general summary of your changes in the Title above -->

Add ability to manage the reload interval to take the load off the
gateway

## Description

<!-- Describe your changes in detail -->

## Related Issue

<!-- This project only accepts pull requests related to open issues. -->
<!-- If suggesting a new feature or change, please discuss it in an
issue first. -->
<!-- If fixing a bug, there should be an issue describing it with steps
to reproduce. -->
<!-- OSS: Please link to the issue here. Tyk: please create/link the
JIRA ticket. -->

## Motivation and Context

<!-- Why is this change required? What problem does it solve? -->

## How This Has Been Tested

<!-- Please describe in detail how you tested your changes -->
<!-- Include details of your testing environment, and the tests -->
<!-- you ran to see how your change affects other areas of the code,
etc. -->
<!-- This information is helpful for reviewers and QA. -->

## Screenshots (if appropriate)

## Types of changes

<!-- What types of changes does your code introduce? Put an `x` in all
the boxes that apply: -->

- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Refactoring or add test (improvements in base code or adds test
coverage to functionality)

## Checklist

<!-- Go over all the following points, and put an `x` in all the boxes
that apply -->
<!-- If there are no documentation updates required, mark the item as
checked. -->
<!-- Raise up any additional concerns not covered by the checklist. -->

- [ ] I ensured that the documentation is up to date
- [ ] I explained why this PR updates go.mod in detail with reasoning
why it's required
- [ ] I would like a code coverage CI quality gate exception and have
explained why

---------

Co-authored-by: Tit Petric <[email protected]>
Co-authored-by: Tit Petric <[email protected]>

[TT-9761]:
https://tyktech.atlassian.net/browse/TT-9761?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

---------

Co-authored-by: Zaid Albirawi <[email protected]>
Co-authored-by: Tit Petric <[email protected]>
…ose idle connections (#5231) (#5741)

[TT-9284] Ensure that old transport will close idle connections (#5231)

<details open>
  <br />
  <table>
    <tr>
      <th>Summary</th>
<td>[CS] The gateway leaks file handles via sockets when max_conn_time
is above 0</td>
    </tr>
    <tr>
      <th>Type</th>
      <td>
<img alt="Bug"

src="https://tyktech.atlassian.net/rest/api/2/universal_avatar/view/type/issuetype/avatar/10303?size=medium"
/>
        Bug
      </td>
    </tr>
    <tr>
      <th>Status</th>
      <td>In Code Review</td>
    </tr>
    <tr>
      <th>Points</th>
      <td>N/A</td>
    </tr>
  </table>
</details>
<!--
  do not remove this marker as it will break jira-lint's functionality.
  added_by_jira_lint
-->

---

TBD

## Description

<!-- Describe your changes in detail -->

## Related Issue

<!-- This project only accepts pull requests related to open issues. -->
<!-- If suggesting a new feature or change, please discuss it in an
issue first. -->
<!-- If fixing a bug, there should be an issue describing it with steps
to reproduce. -->
<!-- OSS: Please link to the issue here. Tyk: please create/link the
JIRA ticket. -->

## Motivation and Context

<!-- Why is this change required? What problem does it solve? -->

## How This Has Been Tested

<!-- Please describe in detail how you tested your changes -->
<!-- Include details of your testing environment, and the tests -->
<!-- you ran to see how your change affects other areas of the code,
etc. -->
<!-- This information is helpful for reviewers and QA. -->

## Screenshots (if appropriate)

## Types of changes

<!-- What types of changes does your code introduce? Put an `x` in all
the boxes that apply: -->

- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Refactoring or add test (improvements in base code or adds test
coverage to functionality)

## Checklist

<!-- Go over all the following points, and put an `x` in all the boxes
that apply -->
<!-- If there are no documentation updates required, mark the item as
checked. -->
<!-- Raise up any additional concerns not covered by the checklist. -->

- [ ] I ensured that the documentation is up to date
- [ ] I explained why this PR updates go.mod in detail with reasoning
why it's required
- [ ] I would like a code coverage CI quality gate exception and have
explained why

---------

Co-authored-by: Tit Petric <[email protected]>

[TT-9284]:
https://tyktech.atlassian.net/browse/TT-9284?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

---------

Co-authored-by: Tit Petric <[email protected]>
…outeHandler (#5482) (#5733)

[TT-9924]remove muxer, proxy from explicitRouteHandler (#5482)

Remove muxer from explicitRouteHandler

[TT-9924]:
https://tyktech.atlassian.net/browse/TT-9924?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

---------

Co-authored-by: Jeffy Mathew <[email protected]>
Co-authored-by: Tit Petric <[email protected]>
TT-8942 Change default RPC pool size (#5030)

<!-- Provide a general summary of your changes in the Title above -->

## Description
Default RPC pool size is set to 20 and because of the relatively heavy
CPU and Memory footprint of the RPC connections in the MDCB model this
can cause issues where either the gateways or MDCB fail on load cause by
scaling events or component failure at MDCB side.

<!-- Describe your changes in detail -->

## Related Issue

<!-- This project only accepts pull requests related to open issues. -->
<!-- If suggesting a new feature or change, please discuss it in an
issue first. -->
<!-- If fixing a bug, there should be an issue describing it with steps
to reproduce. -->
<!-- OSS: Please link to the issue here. Tyk: please create/link the
JIRA ticket. -->
https://tyktech.atlassian.net/browse/TT-8942
## Motivation and Context

<!-- Why is this change required? What problem does it solve? -->
https://tyktech.atlassian.net/browse/TT-8942
## How This Has Been Tested

<!-- Please describe in detail how you tested your changes -->
<!-- Include details of your testing environment, and the tests -->
<!-- you ran to see how your change affects other areas of the code,
etc. -->
<!-- This information is helpful for reviewers and QA. -->

## Screenshots (if appropriate)

## Types of changes

<!-- What types of changes does your code introduce? Put an `x` in all
the boxes that apply: -->

- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Refactoring or add test (improvements in base code or adds test
coverage to functionality)

## Checklist

<!-- Go over all the following points, and put an `x` in all the boxes
that apply -->
<!-- If there are no documentation updates required, mark the item as
checked. -->
<!-- Raise up any additional concerns not covered by the checklist. -->

- [ ] I ensured that the documentation is up to date
- [ ] I explained why this PR updates go.mod in detail with reasoning
why it's required
- [ ] I would like a code coverage CI quality gate exception and have
explained why
…) (#5746)

TT-8934 Fix chunked response analytics (#5495)

<!-- Provide a general summary of your changes in the Title above -->

## Description

<!-- Describe your changes in detail -->

When transfer-encoding is chunked on the upstream response, analytics
records raw responses also contain the chunked characters (example

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Transfer-Encoding#examples).
This is not expected.

The proposed solution is to delete the transfer-encoding header before
reading and writing the response body to the rawResponse field. Without
this header, Go will process the header as it should.


## Related Issue

<!-- This project only accepts pull requests related to open issues. -->
<!-- If suggesting a new feature or change, please discuss it in an
issue first. -->
<!-- If fixing a bug, there should be an issue describing it with steps
to reproduce. -->
<!-- OSS: Please link to the issue here. Tyk: please create/link the
JIRA ticket. -->
https://tyktech.atlassian.net/browse/TT-8934
## Motivation and Context

<!-- Why is this change required? What problem does it solve? -->
https://tyktech.atlassian.net/browse/TT-8934
## How This Has Been Tested

<!-- Please describe in detail how you tested your changes -->
<!-- Include details of your testing environment, and the tests -->
<!-- you ran to see how your change affects other areas of the code,
etc. -->
<!-- This information is helpful for reviewers and QA. -->

Added tests

## Screenshots (if appropriate)

## Types of changes

<!-- What types of changes does your code introduce? Put an `x` in all
the boxes that apply: -->

- [X] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Refactoring or add test (improvements in base code or adds test
coverage to functionality)

## Checklist

<!-- Go over all the following points, and put an `x` in all the boxes
that apply -->
<!-- If there are no documentation updates required, mark the item as
checked. -->
<!-- Raise up any additional concerns not covered by the checklist. -->

- [ ] I ensured that the documentation is up to date
- [ ] I explained why this PR updates go.mod in detail with reasoning
why it's required
- [ ] I would like a code coverage CI quality gate exception and have
explained why

---------

Co-authored-by: Tomas Buchaillot <[email protected]>
Co-authored-by: Tit Petric <[email protected]>
… one of them non-existing (#5124)

[TT-8558] Do not stop applying polices when one of them non-existing (#5124)

In multiple policies applied to a key case, if one of the policies is
not found, the other policies should continue to be applied so that
their APIs continues to work.
…poses key in the log (#5080)

[TT-926] fix security vulnerability that exposes key in the log  (#5080)

<!-- Provide a general summary of your changes in the Title above -->

## Description


The problem involved the HashKey function in
[storage.go](https://github.com/TykTechnologies/tyk/blob/76a00611f33e150b1b6145264909baeefa1cbd4f/storage/storage.go#L170).
It returns the key without hashing if "hash_keys" is set to false in the
tyk.conf file. Now a conditional has been added that obfuscates the key
if "hash_keys" and "enable_key_logging" are set to false.

## Related Issue

[View issue here](https://tyktech.atlassian.net/browse/TT-926)

## Motivation and Context

This issue is a low level security vulnerability.

## How This Has Been Tested

<!-- Please describe in detail how you tested your changes -->
<!-- Include details of your testing environment, and the tests -->
<!-- you ran to see how your change affects other areas of the code,
etc. -->
<!-- This information is helpful for reviewers and QA. -->

## Screenshots (if appropriate)

## Types of changes

<!-- What types of changes does your code introduce? Put an `x` in all
the boxes that apply: -->

- [x] Bug fix (non-breaking change which fixes an issue)

## Checklist

<!-- Go over all the following points, and put an `x` in all the boxes
that apply -->
<!-- If there are no documentation updates required, mark the item as
checked. -->
<!-- Raise up any additional concerns not covered by the checklist. -->

- [ ] I ensured that the documentation is up to date
- [ ] I explained why this PR updates go.mod in detail with reasoning
why it's required
- [ ] I would like a code coverage CI quality gate exception and have
explained why
@lghiur lghiur changed the base branch from master to release-4-lts January 22, 2024 12:10
Copy link

sweep-ai bot commented Jan 22, 2024

Sweeping

Resolving merge conflicts: track the progress here.

I'm currently resolving the merge conflicts in this PR. I will stack a new PR once I'm done.

[!CAUTION]

An error has occurred: Cmd('git') failed due to: exit code(1)
cmdline: git commit -m Start of Merge Conflict Resolution
stdout: 'On branch sweep/release-4.0.16-update-merge-conflict_0
nothing to commit, working tree clean' (tracking ID: 09f6f57e6b)

Copy link

sweep-ai bot commented Jan 22, 2024

Sweeping

Fixing PR: track the progress here.

I'm currently fixing this PR to address the following:

[Sweep GHA Fix] The GitHub Actions run failed with the following error logs:

The command:
Run cd ci/terraform
yielded the following error:
##[error]Process completed with exit code 1.
##[group]Run colour=bad
�[36;1mcolour=bad�[0m
�[36;1mpretext=":boom: Could not add new build $refs/heads/release-4.0.16-update from TykTechnologies/tyk to CD. Please review this run and correct it if needed. See https://github.com/TykTechnologies/tyk-ci/wiki/IntegrationEnvironment for what this is about."�[0m
�[36;1mcurl https://raw.githubusercontent.com/rockymadden/slack-cli/master/src/slack -o /tmp/slack && chmod +x /tmp/slack�[0m
�[36;1m/tmp/slack chat send \�[0m
�[36;1m--actions '{"type": "button", "style": "primary", "text": "See log", "url": "https://github.com/TykTechnologies/tyk/actions/runs/7610898647"}' \�[0m
�[36;1m--author 'Bender' \�[0m
�[36;1m--author-icon 'https://hcoop.net/~alephnull/bender/bender-arms.jpg' \�[0m
�[36;1m--author-link 'https://github.com/TykTechnologies/tyk-ci' \�[0m
�[36;1m--channel '#service-integration' \�[0m
�[36;1m--color $colour \�[0m
�[36;1m--fields '{"title": "Repo", "value": "TykTechnologies/tyk", "short": false}' \�[0m
�[36;1m--footer 'github-actions' \�[0m
�[36;1m--footer-icon 'https://assets-cdn.github.com/images/modules/logos_page/Octocat.png' \�[0m
�[36;1m--image 'https://assets-cdn.github.com/images/modules/logos_page/Octocat.png' \�[0m
�[36;1m--pretext "$pretext" \�[0m
�[36;1m--text 'Commit message: Remove del-env, config docs (#5931)�[0m
�[36;1m�[0m
�[36;1mAs per the https://tyktech.atlassian.net/browse/SYSE-292 policy, delete�[0m
�[36;1mold workflows for 4-lts;�[0m
�[36;1m�[0m
�[36;1mRelated to: https://github.com/TykTechnologies/tyk-docs/pull/3831�[0m
�[36;1m�[0m
�[36;1mCo-authored-by: Tit Petric <[email protected]>' \�[0m
�[36;1m--title 'Failed to add new build for CD' \�[0m
�[36;1m--title-link 'https://github.com/TykTechnologies/tyk/actions/runs/7610898647'�[0m
shell: /usr/bin/bash -e {0}
env:
  SLACK_CLI_TOKEN: ***
  GOPRIVATE: github.com/TykTechnologies
##[endgroup]
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 26518  100 26518    0     0   203k      0 --:--:-- --:--:-- --:--:--  205k
{
  "ok": false,
  "error": "no_text"
}
##[error]Process completed with exit code 1.

Here are the logs:
�[0m�[1mInitializing the backend...�[0m

�[0m�[1mInitializing provider plugins...�[0m
- terraform.io/builtin/terraform is built in to Terraform

�[0m�[1m�[32mTerraform has been successfully initialized!�[0m�[32m�[0m
�[0m�[32m
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.�[0m
�[31m╷�[0m�[0m
�[31m│�[0m �[0m�[1m�[31mError: �[0m�[0m�[1mUnsupported attribute�[0m
�[31m│�[0m �[0m
�[31m│�[0m �[0m�[0m  on outputs.tf line 19, in output "tyk":
�[31m│�[0m �[0m  19:   value = data.terraform_remote_state.integration.outputs�[4m.tyk�[0m�[0m
�[31m│�[0m �[0m    �[90m├────────────────�[0m
�[31m│�[0m �[0m�[0m    �[90m│�[0m �[1mdata.terraform_remote_state.integration.outputs�[0m is object with 2 attributes
�[31m│�[0m �[0m�[0m
�[31m│�[0m �[0mThis object does not have an attribute named "tyk".
�[31m╵�[0m�[0m
�[31m╷�[0m�[0m
�[31m│�[0m �[0m�[1m�[31mError: �[0m�[0m�[1mUnsupported attribute�[0m
�[31m│�[0m �[0m
�[31m│�[0m �[0m�[0m  on outputs.tf line 24, in output "region":
�[31m│�[0m �[0m  24:   value = data.terraform_remote_state.integration.outputs�[4m.region�[0m�[0m
�[31m│�[0m �[0m    �[90m├────────────────�[0m
�[31m│�[0m �[0m�[0m    �[90m│�[0m �[1mdata.terraform_remote_state.integration.outputs�[0m is object with 2 attributes
�[31m│�[0m �[0m�[0m
�[31m│�[0m �[0mThis object does not have an attribute named "region".
�[31m╵�[0m�[0m

[!CAUTION]

An error has occurred: Message is too long, max tokens is -212582 (tracking ID: ff891db782)

@buger
Copy link
Member

buger commented Jan 22, 2024

API tests result: failure 🚫
Branch used: refs/heads/release-4.0.16-update
Commit: 4614a59 Merging to release-4-lts: TT-10329 fix getargspec removal from python stdlib in 3.11 (deprecated since python 3.0) (#5664)

TT-10329 fix getargspec removal from python stdlib in 3.11 (deprecated since python 3.0) (#5664)

https://tyktech.atlassian.net/browse/TT-10329


Co-authored-by: Tit Petric [email protected]
Triggered by: push (@lghiur)
Execution page

Copy link

sweep-ai bot commented Jan 23, 2024

Sweeping

Resolving merge conflicts: track the progress here.

I'm currently resolving the merge conflicts in this PR. I will stack a new PR once I'm done.

Created Pull Request: #5964

@buger
Copy link
Member

buger commented Jan 23, 2024

API tests result: success
Branch used: refs/pull/5962/merge
Commit: d0c9a08
Triggered by: pull_request (@alephnull)
Execution page

@buger
Copy link
Member

buger commented Jan 23, 2024

API tests result: failure 🚫
Branch used: refs/heads/release-4.0.16-update
Commit: d0c9a08 OIDC based AWS login

Removed bitrot
Triggered by: push (@alephnull)
Execution page

@buger
Copy link
Member

buger commented Jan 23, 2024

API tests result: failure 🚫
Branch used: refs/heads/release-4.0.16-update
Commit: 7d64317 test upgrades every time
Triggered by: push (@alephnull)
Execution page

Copy link
Contributor

@alephnull alephnull left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There was a problem pushing to CI which was non-critical but I've fixed anyway so that ECR has updated images. I can only confirm the releng code and the other code is best reviewed by someone who knows what is going on.

@buger
Copy link
Member

buger commented Jan 23, 2024

API tests result: failure 🚫
Branch used: refs/pull/5962/merge
Commit: ed353c2
Triggered by: pull_request (@alephnull)
Execution page

@buger
Copy link
Member

buger commented Jan 23, 2024

API tests result: failure 🚫
Branch used: refs/heads/release-4.0.16-update
Commit: ed353c2 Merging to release-4-lts: Merging to release-4.3.8: TT-9964 Prevent new idle connections to be generated. (#5526)

Merging to release-4.3.8: TT-9964 Prevent new idle connections to be generated. (#5526)

TT-9964 Prevent new idle connections to be generated. (#5526)
Triggered by: push (@alephnull)
Execution page

@buger
Copy link
Member

buger commented Jan 23, 2024

API tests result: success
Branch used: refs/pull/5962/merge
Commit: 63faf3c
Triggered by: pull_request (@alephnull)
Execution page

@buger
Copy link
Member

buger commented Jan 23, 2024

API tests result: failure 🚫
Branch used: refs/pull/5962/merge
Commit: ddb27bf
Triggered by: pull_request (@alephnull)
Execution page

@buger
Copy link
Member

buger commented Jan 23, 2024

API tests result: failure 🚫
Branch used: refs/heads/release-4.0.16-update
Commit: 63faf3c vbump
Triggered by: push (@alephnull)
Execution page

@buger
Copy link
Member

buger commented Jan 23, 2024

API tests result: failure 🚫
Branch used: refs/heads/release-4.0.16-update
Commit: ddb27bf vbump
Triggered by: push (@alephnull)
Execution page

@buger
Copy link
Member

buger commented Jan 23, 2024

API tests result: success
Branch used: refs/pull/5962/merge
Commit: ddb27bf
Triggered by: pull_request (@alephnull)
Execution page

@buger
Copy link
Member

buger commented Jan 23, 2024

API tests result: failure 🚫
Branch used: refs/heads/release-4.0.16-update
Commit: ddb27bf vbump
Triggered by: push (@alephnull)
Execution page

1 similar comment
@buger
Copy link
Member

buger commented Jan 23, 2024

API tests result: failure 🚫
Branch used: refs/heads/release-4.0.16-update
Commit: ddb27bf vbump
Triggered by: push (@alephnull)
Execution page

@buger
Copy link
Member

buger commented Jan 23, 2024

API tests result: success
Branch used: refs/pull/5962/merge
Commit: f2488a4
Triggered by: pull_request (@ilijabojanovic)
Execution page

@buger
Copy link
Member

buger commented Jan 23, 2024

API tests result: failure 🚫
Branch used: refs/heads/release-4.0.16-update
Commit: f2488a4 Update api-tests.yml
Triggered by: push (@ilijabojanovic)
Execution page

@lghiur lghiur merged commit 5a9989e into release-4-lts Jan 23, 2024
32 of 35 checks passed
@buger
Copy link
Member

buger commented Jan 23, 2024

API tests result: success
Branch used: refs/heads/release-4-lts
Commit: 5a9989e Release 4.0.16 update (#5962)

Co-authored-by: Leonid Bugaev [email protected]
Co-authored-by: Tit Petric [email protected]
Co-authored-by: maciej [email protected]
Co-authored-by: Tit Petric [email protected]
Co-authored-by: Ilija Bojanovic [email protected]
Co-authored-by: Sredny M [email protected]
Co-authored-by: Matias [email protected]
Co-authored-by: Tyk-ITS Account [email protected]
Co-authored-by: Zaid Albirawi [email protected]
Co-authored-by: Jeffy Mathew [email protected]
Co-authored-by: Tomas Buchaillot [email protected]
Co-authored-by: Alok G Singh [email protected]
Co-authored-by: Alok G Singh [email protected]
Triggered by: push (@lghiur)
Execution page

@buger
Copy link
Member

buger commented Jan 23, 2024

API tests result: failure 🚫
Branch used: refs/heads/release-4.0.16
Commit: 5a9989e Release 4.0.16 update (#5962)

Co-authored-by: Leonid Bugaev [email protected]
Co-authored-by: Tit Petric [email protected]
Co-authored-by: maciej [email protected]
Co-authored-by: Tit Petric [email protected]
Co-authored-by: Ilija Bojanovic [email protected]
Co-authored-by: Sredny M [email protected]
Co-authored-by: Matias [email protected]
Co-authored-by: Tyk-ITS Account [email protected]
Co-authored-by: Zaid Albirawi [email protected]
Co-authored-by: Jeffy Mathew [email protected]
Co-authored-by: Tomas Buchaillot [email protected]
Co-authored-by: Alok G Singh [email protected]
Co-authored-by: Alok G Singh [email protected]
Triggered by: push (@lghiur)
Execution page

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants