Skip to content

feat: add Session class and rapida init command to authenticate#96

Merged
JinIgarashi merged 3 commits intomainfrom
feat/95-azure-authentication
Dec 17, 2024
Merged

feat: add Session class and rapida init command to authenticate#96
JinIgarashi merged 3 commits intomainfrom
feat/95-azure-authentication

Conversation

@JinIgarashi
Copy link
Contributor

closes #95

@iferencik
I likely don't have appropriate role to access to storage account while I only have reader and contributor roles. You looks having different role (Storage Blob Data Contributor) from mine. Could you try this branch to check if you can access blob?

make build # to build docker image
make shell # to enter container

az login # authenticate with Azure CLI

pipenv run rapida init # init command to authenticate

pipenv run python -m cbsurge.session # run my test script to download a catalog.json
permission error logs
root@cda20016d7fc:/workspace# pipenv run python -m cbsurge.session
INFO:azure.identity._credentials.environment:No environment configuration found.
INFO:azure.identity._credentials.managed_identity:ManagedIdentityCredential will use IMDS
DEBUG:__main__:<azure.storage.blob._blob_service_client.BlobServiceClient object at 0xffffaa56b230>
DEBUG:__main__:<azure.storage.blob._container_client.ContainerClient object at 0xffffaa56b7d0>
DEBUG:__main__:https://undpgeohub.blob.core.windows.net/stacdata/catalog.json
DEBUG:azure.identity._internal.decorators:EnvironmentCredential.get_token_info failed: EnvironmentCredential authentication unavailable. Environment variables are not fully configured.
Visit https://aka.ms/azsdk/python/identity/environmentcredential/troubleshoot to troubleshoot this issue.
Traceback (most recent call last):
  File "/workspace/.venv/lib/python3.12/site-packages/azure/identity/_internal/decorators.py", line 23, in wrapper
    token = fn(*args, **kwargs)
            ^^^^^^^^^^^^^^^^^^^
  File "/workspace/.venv/lib/python3.12/site-packages/azure/identity/_credentials/environment.py", line 182, in get_token_info
    raise CredentialUnavailableError(message=message)
azure.identity._exceptions.CredentialUnavailableError: EnvironmentCredential authentication unavailable. Environment variables are not fully configured.
Visit https://aka.ms/azsdk/python/identity/environmentcredential/troubleshoot to troubleshoot this issue.
INFO:azure.core.pipeline.policies.http_logging_policy:Request URL: 'http://169.254.169.254/metadata/identity/oauth2/token?api-version=REDACTED&resource=REDACTED'
Request method: 'GET'
Request headers:
    'User-Agent': 'azsdk-python-identity/1.19.0 Python/3.12.3 (Linux-6.10.11-linuxkit-aarch64-with-glibc2.39)'
No body was attached to the request
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): 169.254.169.254:80
DEBUG:azure.identity._internal.msal_managed_identity_client:ImdsCredential.get_token_info failed: ManagedIdentityCredential authentication unavailable, no response from the IMDS endpoint.
Traceback (most recent call last):
  File "/workspace/.venv/lib/python3.12/site-packages/azure/identity/_credentials/imds.py", line 86, in _request_token
    client.request_token(*scopes, connection_timeout=1, retry_total=0)
  File "/workspace/.venv/lib/python3.12/site-packages/azure/identity/_internal/managed_identity_client.py", line 146, in request_token
    response = self._pipeline.run(request, retry_on_methods=[request.method], **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/.venv/lib/python3.12/site-packages/azure/core/pipeline/_base.py", line 240, in run
    return first_node.send(pipeline_request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/.venv/lib/python3.12/site-packages/azure/core/pipeline/_base.py", line 96, in send
    response = self.next.send(request)
               ^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/.venv/lib/python3.12/site-packages/azure/core/pipeline/_base.py", line 96, in send
    response = self.next.send(request)
               ^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/.venv/lib/python3.12/site-packages/azure/core/pipeline/_base.py", line 96, in send
    response = self.next.send(request)
               ^^^^^^^^^^^^^^^^^^^^^^^
  [Previous line repeated 1 more time]
  File "/workspace/.venv/lib/python3.12/site-packages/azure/core/pipeline/policies/_retry.py", line 573, in send
    raise err
  File "/workspace/.venv/lib/python3.12/site-packages/azure/core/pipeline/policies/_retry.py", line 551, in send
    response = self.next.send(request)
               ^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/.venv/lib/python3.12/site-packages/azure/core/pipeline/_base.py", line 96, in send
    response = self.next.send(request)
               ^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/.venv/lib/python3.12/site-packages/azure/core/pipeline/_base.py", line 96, in send
    response = self.next.send(request)
               ^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/.venv/lib/python3.12/site-packages/azure/core/pipeline/_base.py", line 96, in send
    response = self.next.send(request)
               ^^^^^^^^^^^^^^^^^^^^^^^
  [Previous line repeated 1 more time]
  File "/workspace/.venv/lib/python3.12/site-packages/azure/core/pipeline/_base.py", line 128, in send
    self._sender.send(request.http_request, **request.context.options),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/.venv/lib/python3.12/site-packages/azure/core/pipeline/transport/_requests_basic.py", line 409, in send
    raise error
azure.core.exceptions.ServiceRequestError: <urllib3.connection.HTTPConnection object at 0xffffaac13e60>: Failed to establish a new connection: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workspace/.venv/lib/python3.12/site-packages/azure/identity/_internal/msal_managed_identity_client.py", line 162, in _get_token_base
    token = self._request_token(*scopes, claims=claims, tenant_id=tenant_id, enable_cae=enable_cae, **kwargs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/.venv/lib/python3.12/site-packages/azure/identity/_credentials/imds.py", line 96, in _request_token
    raise CredentialUnavailableError(error_message) from ex
azure.identity._exceptions.CredentialUnavailableError: ManagedIdentityCredential authentication unavailable, no response from the IMDS endpoint.
DEBUG:azure.identity._internal.decorators:ManagedIdentityCredential.get_token_info failed: ManagedIdentityCredential authentication unavailable, no response from the IMDS endpoint.
Traceback (most recent call last):
  File "/workspace/.venv/lib/python3.12/site-packages/azure/identity/_credentials/imds.py", line 86, in _request_token
    client.request_token(*scopes, connection_timeout=1, retry_total=0)
  File "/workspace/.venv/lib/python3.12/site-packages/azure/identity/_internal/managed_identity_client.py", line 146, in request_token
    response = self._pipeline.run(request, retry_on_methods=[request.method], **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/.venv/lib/python3.12/site-packages/azure/core/pipeline/_base.py", line 240, in run
    return first_node.send(pipeline_request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/.venv/lib/python3.12/site-packages/azure/core/pipeline/_base.py", line 96, in send
    response = self.next.send(request)
               ^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/.venv/lib/python3.12/site-packages/azure/core/pipeline/_base.py", line 96, in send
    response = self.next.send(request)
               ^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/.venv/lib/python3.12/site-packages/azure/core/pipeline/_base.py", line 96, in send
    response = self.next.send(request)
               ^^^^^^^^^^^^^^^^^^^^^^^
  [Previous line repeated 1 more time]
  File "/workspace/.venv/lib/python3.12/site-packages/azure/core/pipeline/policies/_retry.py", line 573, in send
    raise err
  File "/workspace/.venv/lib/python3.12/site-packages/azure/core/pipeline/policies/_retry.py", line 551, in send
    response = self.next.send(request)
               ^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/.venv/lib/python3.12/site-packages/azure/core/pipeline/_base.py", line 96, in send
    response = self.next.send(request)
               ^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/.venv/lib/python3.12/site-packages/azure/core/pipeline/_base.py", line 96, in send
    response = self.next.send(request)
               ^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/.venv/lib/python3.12/site-packages/azure/core/pipeline/_base.py", line 96, in send
    response = self.next.send(request)
               ^^^^^^^^^^^^^^^^^^^^^^^
  [Previous line repeated 1 more time]
  File "/workspace/.venv/lib/python3.12/site-packages/azure/core/pipeline/_base.py", line 128, in send
    self._sender.send(request.http_request, **request.context.options),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/.venv/lib/python3.12/site-packages/azure/core/pipeline/transport/_requests_basic.py", line 409, in send
    raise error
azure.core.exceptions.ServiceRequestError: <urllib3.connection.HTTPConnection object at 0xffffaac13e60>: Failed to establish a new connection: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workspace/.venv/lib/python3.12/site-packages/azure/identity/_internal/decorators.py", line 23, in wrapper
    token = fn(*args, **kwargs)
            ^^^^^^^^^^^^^^^^^^^
  File "/workspace/.venv/lib/python3.12/site-packages/azure/identity/_credentials/managed_identity.py", line 185, in get_token_info
    return cast(SupportsTokenInfo, self._credential).get_token_info(*scopes, options=options)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/.venv/lib/python3.12/site-packages/azure/identity/_internal/msal_managed_identity_client.py", line 142, in get_token_info
    return self._get_token_base(*scopes, options=options, base_method_name="get_token_info")
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/.venv/lib/python3.12/site-packages/azure/identity/_internal/msal_managed_identity_client.py", line 162, in _get_token_base
    token = self._request_token(*scopes, claims=claims, tenant_id=tenant_id, enable_cae=enable_cae, **kwargs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/.venv/lib/python3.12/site-packages/azure/identity/_credentials/imds.py", line 96, in _request_token
    raise CredentialUnavailableError(error_message) from ex
azure.identity._exceptions.CredentialUnavailableError: ManagedIdentityCredential authentication unavailable, no response from the IMDS endpoint.
DEBUG:azure.identity._persistent_cache:msal-extensions is unable to encrypt a persistent cache: "Unable to import module 'gi'
Runtime dependency of PyGObject is missing.
Depends on your Linux distro, you could install it system-wide by something like:
    sudo apt install python3-gi python3-gi-cairo gir1.2-secret-1
If necessary, please refer to PyGObject's doc:
https://pygobject.readthedocs.io/en/latest/getting_started.html
"
Traceback (most recent call last):
  File "/workspace/.venv/lib/python3.12/site-packages/msal_extensions/libsecret.py", line 18, in <module>
    import gi  # https://github.com/AzureAD/microsoft-authentication-extensions-for-python/wiki/Encryption-on-Linux  # pylint: disable=line-too-long
    ^^^^^^^^^
ModuleNotFoundError: No module named 'gi'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/workspace/.venv/lib/python3.12/site-packages/azure/identity/_persistent_cache.py", line 101, in _get_persistence
    return msal_extensions.LibsecretPersistence(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/.venv/lib/python3.12/site-packages/msal_extensions/persistence.py", line 314, in __init__
    from .libsecret import (  # This uncertain import is deferred till runtime
  File "/workspace/.venv/lib/python3.12/site-packages/msal_extensions/libsecret.py", line 20, in <module>
    raise ImportError("""Unable to import module 'gi'
ImportError: Unable to import module 'gi'
Runtime dependency of PyGObject is missing.
Depends on your Linux distro, you could install it system-wide by something like:
    sudo apt install python3-gi python3-gi-cairo gir1.2-secret-1
If necessary, please refer to PyGObject's doc:
https://pygobject.readthedocs.io/en/latest/getting_started.html

DEBUG:azure.identity._internal.decorators:SharedTokenCacheCredential.get_token_info failed: SharedTokenCacheCredential authentication unavailable. No accounts were found in the cache.
Traceback (most recent call last):
  File "/workspace/.venv/lib/python3.12/site-packages/azure/identity/_internal/decorators.py", line 23, in wrapper
    token = fn(*args, **kwargs)
            ^^^^^^^^^^^^^^^^^^^
  File "/workspace/.venv/lib/python3.12/site-packages/azure/identity/_credentials/shared_cache.py", line 110, in get_token_info
    return cast(SupportsTokenInfo, self._credential).get_token_info(*scopes, options=options)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/.venv/lib/python3.12/site-packages/azure/identity/_credentials/shared_cache.py", line 156, in get_token_info
    return self._get_token_base(*scopes, options=options, base_method_name="get_token_info")
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/.venv/lib/python3.12/site-packages/azure/identity/_credentials/shared_cache.py", line 186, in _get_token_base
    account = self._get_account(self._username, self._tenant_id, is_cae=is_cae)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/.venv/lib/python3.12/site-packages/azure/identity/_internal/decorators.py", line 67, in wrapper
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/workspace/.venv/lib/python3.12/site-packages/azure/identity/_internal/shared_token_cache.py", line 209, in _get_account
    raise CredentialUnavailableError(message=NO_ACCOUNTS)
azure.identity._exceptions.CredentialUnavailableError: SharedTokenCacheCredential authentication unavailable. No accounts were found in the cache.
DEBUG:azure.identity._internal.decorators:AzureCliCredential.get_token_info succeeded
DEBUG:azure.identity._internal.decorators:[Authenticated account] Client ID: 04b07795-8ddb-461a-bbee-02f9e1bf7b46. Tenant ID: b3e5db5e-2944-4837-99f5-7488ace54319. User Principal Name: [email protected]. Object ID (user): 3a2cd35f-040f-4b84-89eb-d6982fe8afb8
INFO:azure.identity._credentials.chained:DefaultAzureCredential acquired a token from AzureCliCredential
INFO:azure.core.pipeline.policies.http_logging_policy:Request URL: 'https://undpgeohub.blob.core.windows.net/stacdata/catalog.json'
Request method: 'GET'
Request headers:
    'x-ms-range': 'REDACTED'
    'x-ms-version': 'REDACTED'
    'Accept': 'application/xml'
    'User-Agent': 'azsdk-python-storage-blob/12.24.0 Python/3.12.3 (Linux-6.10.11-linuxkit-aarch64-with-glibc2.39)'
    'x-ms-date': 'REDACTED'
    'x-ms-client-request-id': '19c8f665-bc6e-11ef-b3a2-339b0f3fbda1'
    'Authorization': 'REDACTED'
No body was attached to the request
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): undpgeohub.blob.core.windows.net:443
DEBUG:urllib3.connectionpool:https://undpgeohub.blob.core.windows.net:443 "GET /stacdata/catalog.json HTTP/11" 403 279
INFO:azure.core.pipeline.policies.http_logging_policy:Response status: 403
Response headers:
    'Content-Length': '279'
    'Content-Type': 'application/xml'
    'Server': 'Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0'
    'x-ms-request-id': 'de175d6d-501e-00fe-247a-50ff1f000000'
    'x-ms-client-request-id': '19c8f665-bc6e-11ef-b3a2-339b0f3fbda1'
    'x-ms-version': 'REDACTED'
    'x-ms-error-code': 'AuthorizationPermissionMismatch'
    'Access-Control-Allow-Origin': 'REDACTED'
    'Date': 'Tue, 17 Dec 2024 11:57:33 GMT'
DEBUG:azure.storage.blob._generated._serialization:Ran into a deserialization error. Ignoring since this is failsafe deserialization
Traceback (most recent call last):
  File "/workspace/.venv/lib/python3.12/site-packages/azure/storage/blob/_generated/_serialization.py", line 1629, in failsafe_deserialize
    return self(target_obj, data, content_type=content_type)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/.venv/lib/python3.12/site-packages/azure/storage/blob/_generated/_serialization.py", line 1496, in __call__
    data = self._unpack_content(response_data, content_type)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/.venv/lib/python3.12/site-packages/azure/storage/blob/_generated/_serialization.py", line 1660, in _unpack_content
    raise ValueError("This pipeline didn't have the RawDeserializer policy; can't deserialize")
ValueError: This pipeline didn't have the RawDeserializer policy; can't deserialize
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/workspace/cbsurge/session.py", line 150, in <module>
    download_stream = blob_client.download_blob()
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/.venv/lib/python3.12/site-packages/azure/core/tracing/decorator.py", line 105, in wrapper_use_tracer
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/.venv/lib/python3.12/site-packages/azure/storage/blob/_blob_client.py", line 753, in download_blob
    return StorageStreamDownloader(**options)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/.venv/lib/python3.12/site-packages/azure/storage/blob/_download.py", line 403, in __init__
    self._response = self._initial_request()
                     ^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/.venv/lib/python3.12/site-packages/azure/storage/blob/_download.py", line 504, in _initial_request
    process_storage_error(error)
  File "/workspace/.venv/lib/python3.12/site-packages/azure/storage/blob/_shared/response_handlers.py", line 186, in process_storage_error
    exec("raise error from None")   # pylint: disable=exec-used # nosec
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 1, in <module>
azure.core.exceptions.HttpResponseError: This request is not authorized to perform this operation using this permission.
RequestId:de175d6d-501e-00fe-247a-50ff1f000000
Time:2024-12-17T11:57:34.6968747Z
ErrorCode:AuthorizationPermissionMismatch
Content: <?xml version="1.0" encoding="utf-8"?><Error><Code>AuthorizationPermissionMismatch</Code><Message>This request is not authorized to perform this operation using this permission.
RequestId:de175d6d-501e-00fe-247a-50ff1f000000
Time:2024-12-17T11:57:34.6968747Z</Message></Error>

@JinIgarashi JinIgarashi added the enhancement New feature or request label Dec 17, 2024
@JinIgarashi JinIgarashi self-assigned this Dec 17, 2024
@JinIgarashi JinIgarashi marked this pull request as draft December 17, 2024 11:59
@JinIgarashi JinIgarashi temporarily deployed to github container registry December 17, 2024 11:59 — with GitHub Actions Inactive
@JinIgarashi JinIgarashi temporarily deployed to azure container registry December 17, 2024 11:59 — with GitHub Actions Inactive
@JinIgarashi JinIgarashi temporarily deployed to azure container registry December 17, 2024 12:58 — with GitHub Actions Inactive
@JinIgarashi JinIgarashi temporarily deployed to github container registry December 17, 2024 12:58 — with GitHub Actions Inactive
@JinIgarashi JinIgarashi marked this pull request as ready for review December 17, 2024 12:59
@JinIgarashi JinIgarashi merged commit 1fef395 into main Dec 17, 2024
3 checks passed
@JinIgarashi JinIgarashi deleted the feat/95-azure-authentication branch December 17, 2024 13:05
iferencik pushed a commit that referenced this pull request Apr 10, 2025
* feat: add Session class and rapida init command to authenticate

* fix

* updated readme and session.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

login command with Azure Identity Client

1 participant