feat(py): MiddlewareV2 Implementation

[huangjeff5]

• Middleware v2 implementation in Python
• Implements several useful middlewares

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request implements a substantial architectural refactoring of the Python SDK, aimed at enhancing both internal maintainability and external usability. It reorganizes core components into a more structured, private internal hierarchy while simultaneously simplifying and consolidating the public-facing API. The changes introduce a more robust middleware system and improved type management, providing a cleaner and more powerful development experience for building AI applications.

Highlights

• Internal Module Refactoring: The Python SDK's internal module structure has undergone a significant overhaul, moving many components from genkit.blocks and genkit.core into new, private (_ prefixed) modules under genkit._ai and genkit._core.
• Public API Simplification: The public API has been streamlined, exposing core functionalities directly under the top-level genkit package and new dedicated modules like genkit.embedder, genkit.model, and genkit.middleware, leading to cleaner imports and usage patterns.
• Enhanced Middleware Capabilities: New and updated middleware functions are introduced, including validate_support, download_request_media, simulate_system_prompt, augment_with_context, retry, and fallback, providing more powerful ways to intercept and modify model requests and responses.
• Improved Developer Experience with Veneer Types: New veneer types (e.g., Message and Document wrappers) have been introduced, offering convenient properties and methods on top of the generated wire types, along with refined schema sanitization for better type management.
• Plugin Dependency Adjustment: The dev-local-vectorstore plugin has been removed from the core pyproject.toml dependencies, indicating a potential deprecation or relocation of this plugin.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• py/README.md
  • Updated references to internal module paths and public API imports.
• py/bin/generate_schema_typing
  • Adjusted the path for the generated typing file to reflect internal module moves.
• py/bin/sanitize_schema_typing.py
  • Enhanced schema sanitization with class exclusions for veneer types.
  • Renamed 'Message' to 'MessageData' in generated types.
  • Improved import and warning handling within the schema generation process.
  • Added logic to inline wrapper types in ModelRequest for better developer experience.
• py/docs/dev_ui_eventloop_model.md
  • Updated module import paths to reflect the new internal structure.
• py/docs/index.md
  • Updated module import paths to reflect the new internal structure.
  • Removed genkit.ai.Output and changed genkit.ai.FlowWrapper to genkit.Flow.
• py/docs/python_docs_roadmap.md
  • Updated module import paths to reflect the new internal structure.
• py/docs/types.md
  • Updated module import paths to reflect the new internal structure.
• py/engdoc/extending/api.md
  • Simplified Genkit import from genkit.ai to genkit.
• py/engdoc/extending/index.md
  • Updated module import paths to reflect the new internal structure.
• py/engdoc/index.md
  • Simplified Genkit import from genkit.ai to genkit.
  • Updated output parameter to output_schema in ai.generate calls.
• py/engdoc/user_guide/python/getting_started.md
  • Updated module import paths to reflect the new internal structure.
• py/packages/genkit/README.md
  • Simplified Genkit import from genkit.ai to genkit.
  • Updated output parameter to output_schema in ai.generate calls.
• py/packages/genkit/pyproject.toml
  • Removed dev-local-vectorstore from optional dependencies and tool.uv.sources.
• py/packages/genkit/src/genkit/init.py
  • Refactored public API imports, exposing new middleware and veneer types directly under the genkit namespace.
  • Introduced Flow = Action as an alias.
• py/packages/genkit/src/genkit/_ai/init.py
  • Added new _ai module for AI-related components.
• py/packages/genkit/src/genkit/_ai/_aio.py
  • Added core asynchronous API for Genkit.
• py/packages/genkit/src/genkit/_ai/_document.py
  • Moved document-related types to the _ai module.
• py/packages/genkit/src/genkit/_ai/_embedding.py
  • Added new module for embedder types and utilities.
• py/packages/genkit/src/genkit/_ai/_evaluator.py
  • Added new module for evaluator types and utilities.
• py/packages/genkit/src/genkit/_ai/_formats/init.py
  • Added new module for built-in output formats.
• py/packages/genkit/src/genkit/_ai/_formats/_array.py
  • Moved array format implementation to the _ai module and updated internal type references.
• py/packages/genkit/src/genkit/_ai/_formats/_enum.py
  • Moved enum format implementation to the _ai module and updated internal type references.
• py/packages/genkit/src/genkit/_ai/_formats/_json.py
  • Moved JSON format implementation to the _ai module and updated internal type references.
• py/packages/genkit/src/genkit/_ai/_formats/_jsonl.py
  • Moved JSONL format implementation to the _ai module and updated internal type references.
• py/packages/genkit/src/genkit/_ai/_formats/_text.py
  • Moved text format implementation to the _ai module and updated internal type references.
• py/packages/genkit/src/genkit/_ai/_formats/_types.py
  • Moved format type definitions to the _ai module and updated internal type references.
• py/packages/genkit/src/genkit/_ai/_generate.py
  • Moved generate action implementation to the _ai module and updated internal type references.
• py/packages/genkit/src/genkit/_ai/_messages.py
  • Moved message utilities to the _ai module and updated internal type references.
• py/packages/genkit/src/genkit/_ai/_middleware.py
  • Added new module for model middleware functions.
• py/packages/genkit/src/genkit/_ai/_model.py
  • Added new module for model veneer types and utilities.
• py/packages/genkit/src/genkit/_ai/_prompt.py
  • Added new module for prompt management and templating.
• py/packages/genkit/src/genkit/_ai/_resource.py
  • Moved resource management to the _ai module and updated internal type references.
• py/packages/genkit/src/genkit/_ai/_testing.py
  • Added new module for internal AI testing utilities.
• py/packages/genkit/src/genkit/_ai/_tools.py
  • Added new module for tool-specific types and utilities.
• py/packages/genkit/src/genkit/_core/init.py
  • Added new _core module for core foundations.
• py/packages/genkit/src/genkit/_core/_action.py
  • Added new module for defining and managing actions.
• py/packages/genkit/src/genkit/_core/_background.py
  • Moved background model definitions to the _core module and updated internal type references.
• py/packages/genkit/src/genkit/_core/_base.py
  • Added new base model for consistent serialization across Genkit types.
• py/packages/genkit/src/genkit/_core/_channel.py
  • Added new module for async streaming channels.
• py/packages/genkit/src/genkit/_core/_compat.py
  • Moved compatibility layer for asyncio to the _core module.
• py/packages/genkit/src/genkit/_core/_constants.py
  • Moved core constants to the _core module.
• py/packages/genkit/src/genkit/_core/_context.py
  • Moved context-related types to the _core module.
• py/packages/genkit/src/genkit/_core/_dap.py
  • Added new module for Dynamic Action Provider support.
• py/packages/genkit/src/genkit/_core/_environment.py
  • Added new module for environment detection.
• py/packages/genkit/src/genkit/_core/_error.py
  • Moved error classes and utilities to the _core module and updated internal type references.
• py/packages/genkit/src/genkit/_core/_extract.py
  • Added new module for JSON extraction utilities.
• py/packages/genkit/src/genkit/_core/_flow.py
  • Added new module for flow registration.
• py/packages/genkit/src/genkit/_core/_http_client.py
  • Added new module for shared HTTP client utilities.
• py/packages/genkit/src/genkit/_core/_logger.py
  • Added new internal logger module.
• py/packages/genkit/src/genkit/_core/_loop_cache.py
  • Moved loop-local cache to the _core module.
• py/packages/genkit/src/genkit/_core/_model.py
  • Added new module for model veneer types.
• py/packages/genkit/src/genkit/_core/_plugin.py
  • Added new abstract base class for Genkit plugins.
• py/packages/genkit/src/genkit/_core/_plugins.py
  • Moved plugin namespace extension to the _core module.
• py/packages/genkit/src/genkit/_core/_reflection.py
  • Added new module for reflection API server.
• py/packages/genkit/src/genkit/_core/_registry.py
  • Moved registry management to the _core module and updated internal type references.
• py/packages/genkit/src/genkit/_core/_schema.py
  • Added new module for schema utilities.
• py/packages/genkit/src/genkit/_core/_tracing.py
  • Moved telemetry and tracing functionality to the _core module.
• py/packages/genkit/src/genkit/_core/_typing.py
  • Moved core typing definitions to the _core module and updated base classes to GenkitModel.
• py/packages/genkit/src/genkit/_core/trace/init.py
  • Moved trace module for OpenTelemetry span management to the _core module.
• py/packages/genkit/src/genkit/_core/trace/_adjusting_exporter.py
  • Added new module for adjusting trace exporter.
• py/packages/genkit/src/genkit/_core/trace/_default_exporter.py
  • Moved default trace exporter to the _core module.
• py/packages/genkit/src/genkit/_core/trace/_path.py
  • Moved path utilities for tracing to the _core module.
• py/packages/genkit/src/genkit/_core/trace/_realtime_processor.py
  • Moved realtime span processor to the _core module.
• py/plugins/google-genai/src/genkit/plugins/google_genai/models/_deprecations.py
  • Renamed deprecations file.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces several powerful middleware capabilities to the Python Genkit library, including retry, fallback, media downloading, and request validation. A high-severity Server-Side Request Forgery (SSRF) vulnerability has been identified in the download_request_media middleware, as it downloads content from arbitrary URLs without sufficient validation, potentially exposing internal resources or sensitive metadata. Beyond this, there are also opportunities to enhance the robustness of the retry logic and simplify code within the new middleware functions. Implementing strict URL validation for the download_request_media middleware is crucial before merging.