[TT-15724] - support for aws static credentials

[LLe27]

Description

• Added the support to define AWS static credentials access_key_id and secret_access_key in the pump configurations.

Related Issue

TT-15724

Motivation and Context

How This Has Been Tested

Screenshots (if appropriate)

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Checklist

• Make sure you are requesting to pull a topic/feature/bugfix branch (right side). If pulling from your own
  fork, don't request your master!
• Make sure you are making a pull request against the master branch (left side). Also, you should start
  your branch off our latest master.
• My change requires a change to the documentation.
  • If you've changed APIs, describe what needs to be updated in the documentation.
• I have updated the documentation accordingly.
• Modules and vendor dependencies have been updated; run go mod tidy && go mod vendor
• I have added tests to cover my changes.
• All new and existing tests passed.
• Check your code additions will not fail linting checks:
  • go fmt -s
  • go vet

[probelabs]

🔍 Code Analysis Results

Change Impact Analysis

This pull request introduces support for static AWS credentials (access_key_id and secret_access_key) for the Kinesis pump. This allows users to configure AWS authentication directly within the pump's settings, offering an alternative to the default AWS credential chain (e.g., IAM roles, environment variables).

• What this PR accomplishes:

  • It enhances the Kinesis pump's authentication mechanism by allowing users to provide static AWS credentials directly in the configuration. This is particularly useful for environments where using IAM roles or shared credential files is not feasible.

• Key technical changes introduced:

  • The KinesisConf struct in pumps/kinesis.go has been extended to include AccessKeyID, SecretAccessKey, and SessionToken.
  • The Init method in pumps/kinesis.go is updated with conditional logic. If both AccessKeyID and SecretAccessKey are provided, it initializes the AWS SDK with a StaticCredentialsProvider. Otherwise, it falls back to the default credential resolution chain.
  • New unit tests have been added in pumps/kinesis_test.go to validate the configuration parsing and the credential selection logic.
  • The README.md file is updated to document the new configuration options and provide usage examples.

• Affected system components:

  • pumps/kinesis.go: The core logic of the Kinesis pump is modified to handle the new authentication method.
  • pumps/kinesis_test.go: A new test file is added to ensure the new functionality is working as expected and to prevent future regressions.
  • README.md: The project's documentation is updated to inform users about the new feature.

Architecture Visualization

The following diagram illustrates the updated authentication flow within the KinesisPump.Init function. It shows how the pump decides whether to use the newly introduced static credentials or fall back to the default AWS credential chain.

graph TD
    A[Start KinesisPump Init] --> B{Are AccessKeyID AND SecretAccessKey provided in config?};
    B -- Yes --> C[Create StaticCredentialsProvider];
    C --> E[Load AWS Config with Static Credentials];
    B -- No --> D["Use Default AWS Credential Chain <br/>(IAM Role, Env Vars, etc.)"];
    D --> F[Load AWS Config with Default Credentials];
    E --> G[Initialize Kinesis Client];
    F --> G;
    G --> H[Kinesis Pump Ready];

Loading

---

Powered by Visor from Probelabs

Last updated: 2025-10-01T16:10:06.942Z | Triggered by: synchronize | Commit: baf041b

[probelabs]

🔍 Code Analysis Results

Security Issues (3)

Severity	Location	Issue
🟢 Info	README.md:1363-1367	The documentation introduces new configuration fields for static AWS credentials but does not include a security warning about the risks of storing sensitive keys in plaintext configuration files. 💡 Suggestion Add a security note to the documentation recommending the use of more secure authentication methods like IAM roles or environment variables where possible, and warning users about the risks of committing configuration files containing plaintext credentials to source control.
🟡 Warning	pumps/kinesis.go:78-82	The introduction of static AWS credentials (`access_key_id`, `secret_access_key`) directly in the configuration file increases the risk of sensitive data exposure. If this configuration file is accidentally committed to version control, shared, or otherwise exposed, it would lead to a credential leak. 💡 Suggestion While providing static credentials offers flexibility, it is a less secure practice compared to using the default AWS credential chain (like IAM roles for EC2/ECS or environment variables). The documentation in README.md should be updated to include a prominent warning about the risks of storing plaintext credentials and strongly recommend using more secure, dynamically provisioned credentials in production environments.
🟡 Warning	pumps/kinesis.go:78	The code checks for `AccessKeyID` and `SecretAccessKey` being non-empty, but it doesn't validate if only one is provided. If a user provides an `AccessKeyID` but not a `SecretAccessKey` (or vice-versa), the code will fall back to the default credential chain, which might be confusing and lead to unexpected behavior. 💡 Suggestion Improve the conditional logic to handle cases where only one of the two required credential fields is provided. For example, log a warning or return an error if the credential configuration is incomplete, making the behavior more explicit and preventing silent fallbacks.

✅ Performance Check Passed

No performance issues found – changes LGTM.

Quality Issues (4)

Severity	Location	Issue
🟢 Info	README.md:1388	There is a typo in the environment variable for the Kinesis stream name. `TYK_PMP_PUMPS_KINESIs_META_STREAMNAME` should be `TYK_PMP_PUMPS_KINESIS_META_STREAMNAME`. 💡 Suggestion Correct the typo to ensure consistency with other environment variables and prevent configuration errors for users. 🔧 Suggested Fix TYK_PMP_PUMPS_KINESIS_META_STREAMNAME=my-stream
🟡 Warning	pumps/kinesis.go:85	The `Init` method's use of `log.Fatalf` on configuration errors makes the function difficult to test reliably, as it terminates the entire process. The current tests can only verify the configuration parsing but cannot test the error-handling path within `Init` itself. 💡 Suggestion Refactor the `Init` method to return an `error` instead of calling `log.Fatalf`. This is a more idiomatic Go practice that allows callers (and tests) to handle errors gracefully. Tests could then assert that specific invalid configurations correctly produce an error.
🟡 Warning	pumps/kinesis.go:78-84	The configuration logic silently falls back to the default AWS credential chain if only one of the two required static credential keys (`AccessKeyID` or `SecretAccessKey`) is provided. This can lead to confusing behavior for users who might expect an error for an incomplete configuration. 💡 Suggestion Improve the conditional logic to be more explicit. If one key is provided without the other, log a warning to inform the user that the static credential configuration is incomplete and that the system is falling back to the default chain, or return an error to enforce a valid configuration.
🟡 Warning	pumps/kinesis_test.go:86-112	The test function `TestKinesisPump_StaticCredentials_Logic` is redundant as it primarily re-tests the configuration parsing already covered in `TestKinesisPump_StaticCredentials_ConfigurationParsing`. It does not effectively verify that the AWS client initialization logic within the `Init` function correctly selects the static credential provider versus the default chain. 💡 Suggestion Remove this redundant test. To properly test the credential selection logic, the `Init` function should be refactored to be more testable (e.g., by returning the created AWS config or by injecting the config loading dependency).

Style Issues (5)

Severity	Location	Issue
🟢 Info	README.md:1365	The description for `secret_access_key` is less detailed than the one for `access_key_id`. The `access_key_id` description helpfully lists examples of the default credential chain (`environment variables, shared credentials file, IAM roles, etc.`), but the `secret_access_key` description omits this. 💡 Suggestion For consistency and clarity, expand the description for `secret_access_key` to match the level of detail provided for `access_key_id`.
🟢 Info	README.md:1361-1367	The descriptions for `batch_size`, `access_key_id`, `secret_access_key`, and `session_token` all end with `(optional)`. However, the description for `batch_size` already begins with "Optional.", making the suffix redundant. 💡 Suggestion To improve conciseness, consider standardizing how optional fields are documented. Either start the description with "Optional." or end it with `(optional)`, but not both. Removing the suffix from all of them would make the documentation cleaner.
🟢 Info	pumps/kinesis.go:43	Similar to the `README.md` file, the code comment for `SecretAccessKey` in the `KinesisConf` struct is less descriptive than the one for `AccessKeyID`. 💡 Suggestion To ensure the code is self-documenting and consistent, update the comment for `SecretAccessKey` to include the same details about the default credential chain as the `AccessKeyID` comment. 🔧 Suggested Fix // AWS Secret Access Key for authentication. If not provided, will use default credential chain (environment variables, shared credentials file, IAM roles, etc.) -- Optional SecretAccessKey string `mapstructure:"secret_access_key"`
🟢 Info	pumps/kinesis_test.go:101-103	In `TestKinesisPump_StaticCredentials_ConfigurationParsing`, assertions are made on individual fields of the parsed configuration. While this works, it's slightly verbose and could miss unintended changes to other fields. 💡 Suggestion For a more concise and robust test, consider using a single assertion to compare the entire expected struct with the actual one. This ensures all fields are correctly parsed as expected. 🔧 Suggested Fix assert.Equal(t, tt.expected, conf)
🟡 Warning	pumps/kinesis_test.go:86-113	The test function `TestKinesisPump_StaticCredentials_Logic` is redundant. It re-validates the outcome of configuration parsing, which is already thoroughly covered by `TestKinesisPump_StaticCredentials_ConfigurationParsing`. It does not, however, test the core logic of *which* AWS credential provider is selected within the `Init` function, which is what its name implies. 💡 Suggestion Remove the `TestKinesisPump_StaticCredentials_Logic` function to eliminate redundancy and streamline the test suite. The existing `TestKinesisPump_StaticCredentials_ConfigurationParsing` provides sufficient coverage for the configuration aspect.

---

Powered by Visor from Probelabs

Last updated: 2025-10-01T16:10:08.448Z | Triggered by: synchronize | Commit: baf041b