Miscellaneous notes on i2

[thorwhalen]

This is to accumulate miscellaneous notes, observations, ideas, etc.

[thorwhalen]

Parameter.empty behavior in function definitions

See my relevant stackoverflow post.

inspect.Parameter.empty is used as the value of the optional parameter attributes kind and default.
Se that if you use it in your function definition, the function will sometimes behave like if you omitted it, and sometimes will not.

Consider the two following functions:

from inspect import Parameter, signature

empty = Parameter.empty

def chalk(x=empty, y=2):
    return True

def cheese(x, y=2):
    return True

The signatures (including signatures shown in help) are the same:

assert signature(chalk) == signature(cheese)
assert str(signature(chalk)) == str(signature(cheese)) =='(x, y=2)'

But from a parameter hints point of view (when, in a fairly rich IDE, you get hints when entering arguments in a function) they are not the same. In VsCode,

• chalk gives me (x: empty = empty, y: int = 2) -> Literal[True], which gives the impression that x has a default.
• cheese gives me (x: Any, y: int = 2) -> Literal[True], which gives the impression that x does not have a default.

inspect.signature basically gives us the value of the __signature__ attribute of the function, but that value is completely independent of the actual behavior of the function.

Where does VsCode source its signature (string)?
Is there a python function (builtin if possible) that will give me that true signature?

signatures can lie

To see that signature is actually a "lie", consider this:

def weirdo(a, b, c=3):
    return f"{a=}, {b=}, {c=}"

assert weirdo(1, 2) == 'a=1, b=2, c=3'
# 'a=1, b=2, c=3'

weirdo.__signature__ = signature(chalk)

Function's behavior doesn't change:

assert weirdo(1, 2) == weirdo(1, b=2) == weirdo(a=1, b=2) == 'a=1, b=2, c=3'

But the signature is different:

signature(weirdo)
# <Signature (x, y=2)>

But if we believe the (x, y=2) signature that weirdo now has, we will get an error:

weirdo(x=1, y=2)
# TypeError: weirdo() got an unexpected keyword argument 'x'

More behaviors

from i2 import Sig

def has_valid_signature(x=Sig.empty, y=2):
    return True

assert Sig(has_valid_signature) == Sig('(x, y=2)')

# Python will let me define this function
def does_not_have_valid_signature(x=2, y=Sig.empty):
    return True

# But not call it:
# Sig(does_not_have_valid_signature)  # Boom
# # ValueError: non-default argument follows default argument

# # TODO: we probably want to raise a InvalidSignatureError instead of a ValueError here

On the other hand, though the following definition should be basically saying "neither x nor y have defaults", the python interpreter won't even let me define the function:

def should_be_valid_but_not_parsable(x=Sig.empty, y):
    pass

validate_signature

This led me to write validate_signature, so that, when one does signature manipulations, one can have the means to get early errors when things are wrong.

[thorwhalen]

Defaults Source Object

I'll focus on defaults here, but it should be obvious that the same could be done with other often repeated values such as annotations, docstring argument descriptions, etc.

The context that got me into the rabbit-hole writing of Parameter.empty behavior in function definitions
] is that I was thinking that, in order to align and keep in synch with defaults of a framework when writing a facade for the framework (here, I was working on my oa facade of the openai package), it would be useful to construct an object that contained all these defaults. It would look something like this:

openai_defaults = gather_defaults(list_of_objects_that_have_signatures)
# or
openai_defaults = gather_defaults(openai)  # will have rules to gather the objects with signatures

The defaults object would then be a Mapping (and/or "struct", like types.SimpleNamespace) that could be used when defining functions:

def my_facade(x, model=default.model, endpoint=default.endpoint): ...

This is what led me to writing Parameter.empty behavior in function definitions
: See that if default.endpoint is empty, there's a problem with the "after defaulted arguments there should be ONLY defaulted arguments (or variadic keyword ones)".
It's easy to circumvent (for example, disallowing any of the defaults to be empty), but should be noted.

Note that such a tool would have to resolve conflicts, just like with signature merging. In fact, one way to make this object would be simply to merge all signatures of the list (after making them all be PK kind, and removing annotations, to not have any conflicts at that level).

[thorwhalen]

Already existing (sorta) finer types

I found this: https://github.com/python/typeshed/blob/main/stdlib/_typeshed/__init__.pyi

It's unfortunately not included in the standard lib itself per se, but is part of the Typeshed project, which provides type definitions for the Python standard library and third-party modules. These specific protocols (e.g., SupportsAiter, SupportsLenAndGetItem, SupportsItems, etc.) are defined in Typeshed’s internal _typeshed module, which is not a standalone importable module in the Python standard library but rather a collection of type hints used by static type checkers like Pylance, mypy, and pyright.

I'm not sure how I should use them besides just copying them over to where I want to use them, but I'd rather use these than those that I have been writing myself. There's things like:

...

# Mapping-like protocols

# stable
class SupportsItems(Protocol[_KT_co, _VT_co]):
    def items(self) -> AbstractSet[tuple[_KT_co, _VT_co]]: ...

# stable
class SupportsKeysAndGetItem(Protocol[_KT, _VT_co]):
    def keys(self) -> Iterable[_KT]: ...
    def __getitem__(self, key: _KT, /) -> _VT_co: ...

# This protocol is currently under discussion. Use SupportsContainsAndGetItem
# instead, if you require the __contains__ method.
# See https://github.com/python/typeshed/issues/11822.
class SupportsGetItem(Protocol[_KT_contra, _VT_co]):
    def __contains__(self, x: Any, /) -> bool: ...
    def __getitem__(self, key: _KT_contra, /) -> _VT_co: ...

# stable
class SupportsContainsAndGetItem(Protocol[_KT_contra, _VT_co]):
    def __contains__(self, x: Any, /) -> bool: ...
    def __getitem__(self, key: _KT_contra, /) -> _VT_co: ...

# stable
class SupportsItemAccess(Protocol[_KT_contra, _VT]):
    def __contains__(self, x: Any, /) -> bool: ...
    def __getitem__(self, key: _KT_contra, /) -> _VT: ...
    def __setitem__(self, key: _KT_contra, value: _VT, /) -> None: ...
    def __delitem__(self, key: _KT_contra, /) -> None: ...

StrPath: TypeAlias = str | PathLike[str]  # stable
BytesPath: TypeAlias = bytes | PathLike[bytes]  # stable
GenericPath: TypeAlias = AnyStr | PathLike[AnyStr]
StrOrBytesPath: TypeAlias = str | bytes | PathLike[str] | PathLike[bytes]  # stable

OpenTextModeUpdating: TypeAlias = Literal[
    "r+",
    "+r",
    "rt+",
    "r+t",

...

[thorwhalen]

💡 A Problem: Balancing Flexibility and Explicitness in Data Interfaces

In software design, interfaces must navigate a subtle trade-off: they should be robust and flexible for the user, yet clear and predictable for the developer.

The Conflict: Liberal Input vs. Explicit Code

On one hand, the Robustness Principle (Postel's Law) advises us to "be conservative in what you send, and liberal in what you accept."
This translates to making your functions highly accommodating of various input formats. For instance, a function expecting a Pandas DataFrame might be much more convenient if it could also seamlessly accept:

• A file path or URL pointing to the data.
• A list of dictionaries.
• Raw JSON or CSV text.

This flexibility eliminates annoying data preparation boilerplate and lets the user "just give it the data we have." It allows the application layer to focus on its primary concern, encapsulating data adaptation away.

On the other hand, the popular mantra "Explicit is better than implicit" warns against hidden complexity. Overly flexible interfaces can lead to code that is hard to debug and maintain because the required input preparation is not immediately obvious.

A Solution: A Flexible, Organized Input Adapter

i2.castgraph solves this by providing a controlled, organized mechanism for dynamic input adaptation. It allows you to build a network of permissible, multi-step transformations, making interfaces highly "liberal in what they accept" without forcing the application code itself to become a tangled mess of conversion logic.

castgraph is an organized system to:

• Enforce Postel's Law: By automatically routing an input (like a file path) through multiple transformations (e.g., path $\rightarrow$ bytes $\rightarrow$ JSON string $\rightarrow$ dictionary) until it matches the target kind (e.g., a DataFrame).
• Maintain Explicitness: By consolidating all transformation logic in a single, dedicated graph structure, keeping the main function clean and focused on its core business.

This results in interfaces that are both developer-friendly and maintainable.