Skip to content

Load hook metadata from YAML without importing Hook class#63826

Merged
potiuk merged 10 commits intoapache:mainfrom
yuseok89:hook-meta-load-from-yaml
Apr 4, 2026
Merged

Load hook metadata from YAML without importing Hook class#63826
potiuk merged 10 commits intoapache:mainfrom
yuseok89:hook-meta-load-from-yaml

Conversation

@yuseok89
Copy link
Copy Markdown
Contributor

@yuseok89 yuseok89 commented Mar 17, 2026

Closes: #61815

Loading the Connection Add page is slow on the first uncached request because hook_meta_data() imports Hook classes for every connection type. With 129+ providers, this cost becomes significant before the response can be cached.

This PR changes the GET /ui/connections/hook_meta response so it no longer imports Hook classes and instead uses only the metadata from provider.yaml via _load_ui_metadata(). This was made possible by @amoghrajesh 's provider.yaml migration, which moved connection UI metadata into YAML.

In this change, hook-name is added to connection-types in provider.yaml for display in the connection type dropdown (falls back to connection_type when absent).

Before

Screen.Recording.2026-02-12.at.9.34.40.PM.mov

After

Screen.Recording.2026-03-18.at.11.04.44.PM.mov

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)
    • Cursor

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@boring-cyborg boring-cyborg bot added the area:API Airflow's REST/HTTP API label Mar 17, 2026
Copy link
Copy Markdown
Contributor

@amoghrajesh amoghrajesh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Direction looks fine, some comments but otherwise looks fine!

@potiuk
Copy link
Copy Markdown
Member

potiuk commented Apr 1, 2026

@yuseok89 This PR has a few issues that need to be addressed before it can be reviewed — please see our Pull Request quality criteria.

Issues found:

  • ⚠️ Unresolved review comments: This PR has 3 unresolved review threads from maintainers: @amoghrajesh (MEMBER): 3 unresolved threads. Please review and resolve all inline review comments before requesting another review. You can resolve a conversation by clicking 'Resolve conversation' on each thread after addressing the feedback. See pull request guidelines.

Note: Your branch is 313 commits behind main. Some check failures may be caused by changes in the base branch rather than by your PR. Please rebase your branch and push again to get up-to-date CI results.

What to do next:

  • The comment informs you what you need to do.
  • Fix each issue, then mark the PR as "Ready for review" in the GitHub UI - but only after making sure that all the issues are fixed.
  • There is no rush — take your time and work at your own pace. We appreciate your contribution and are happy to wait for updates.
  • Maintainers will then proceed with a normal review.

There is no rush — take your time and work at your own pace. We appreciate your contribution and are happy to wait for updates. If you have questions, feel free to ask on the Airflow Slack.

@yuseok89 yuseok89 requested a review from amoghrajesh April 1, 2026 21:41
@kaxil kaxil requested a review from Copilot April 2, 2026 00:45
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Copy link
Copy Markdown
Member

@choo121600 choo121600 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!
This part was causing flaky issues, so thanks for fixing it 🙌

@potiuk potiuk added the backport-to-v3-2-test Mark PR with this label to backport to v3-2-test branch label Apr 4, 2026
@potiuk potiuk added this to the Airflow 3.2.1 milestone Apr 4, 2026
@potiuk potiuk merged commit c4a209b into apache:main Apr 4, 2026
149 checks passed
github-actions bot pushed a commit that referenced this pull request Apr 4, 2026
…#63826)

* Load hook metadata from YAML without importing Hook class

* Add hook-name to all provider.yaml connection-types

* Add hook-name to connection types and regenerate get_provider_info.py

* Fix ruff import order in connections.py

* fix: import ProvidersManager at top level per review

* Fix provider connection hook display names

* Add iter_connection_type_hook_ui_metadata for connection UI hook metadata
(cherry picked from commit c4a209b)

Co-authored-by: Yuseok Jo <yuseok89@gmail.com>
@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 4, 2026

Backport successfully created: v3-2-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status Branch Result
v3-2-test PR Link

github-actions bot pushed a commit to aws-mwaa/upstream-to-airflow that referenced this pull request Apr 4, 2026
…apache#63826)

* Load hook metadata from YAML without importing Hook class

* Add hook-name to all provider.yaml connection-types

* Add hook-name to connection types and regenerate get_provider_info.py

* Fix ruff import order in connections.py

* fix: import ProvidersManager at top level per review

* Fix provider connection hook display names

* Add iter_connection_type_hook_ui_metadata for connection UI hook metadata
(cherry picked from commit c4a209b)

Co-authored-by: Yuseok Jo <yuseok89@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:API Airflow's REST/HTTP API backport-to-v3-2-test Mark PR with this label to backport to v3-2-test branch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Load connection type metadata on-demand to improve Add Connection page load time

5 participants