RandomVariables raise ValueError when size would broadcast parameter

[ricardoV94]

We have a common check in multivariate distributions (+ categorical), that raises when a non-size is not strictly equal to the batched dimensions of the parameters, saying objects cannot be broadcast to a single shape.

aesara/aesara/tensor/random/basic.py

Lines 377 to 383 in 510a961

	if (
	0 < alphas.ndim - 1 <= len(size)
	and size[-alphas.ndim + 1 :] != alphas.shape[:-1]
	):
	raise ValueError(
	"shape mismatch: objects cannot be broadcast to a single shape"
	)

I don't know why we are imposing this limitation. Even if there is a good reason, the message is wrong, because the batched shapes may very well be broadcastable to size:

# Raises ValueError: shape mismatch: objects cannot be broadcast to a single shape
# However batch_shape=(1,) and size=(5,) are clearly broadcastable
at.random.dirichlet([[0.2, 0.3, 0.5]], size=5).eval()

The message for the CategoricalRV is more accurate:

aesara/aesara/tensor/random/basic.py

Lines 651 to 652 in 510a961

	if len(size) > 0 and size[-len(ind_shape) :] != ind_shape:
	raise ValueError("Parameters shape and size do not match.")

However I still don't get why we would not broadcast the parameter to match size, when this is would be valid. Otherwise, the following should also be invalid, no?

# Just fine
at.random.normal([0], size=5).eval()

[brandonwillard]

You can check the git blame results at those locations and find the relevant commit(s), and—through that—the associated PR: #446.

[ricardoV94]

I know that it came through that PR. It does not explain the interpretation with respect to broadcasting to size.

It explains why you cannot do at.random.dirichlet(np.ones((2, 5)), size=5), the same way one cannot do at.random.normal(np.ones((2,)), size=5)

It does not explain why parameters can broadcast to size in univariate and not in multivariate distributions. Also the categorical distribution is a univariate one, and has the same limitation as other multivariate distributions.

[brandonwillard]

It does not explain why parameters can broadcast to size in univariate and not in multivariate distributions.

For completeness, they can:

import aesara.tensor as at

at.random.dirichlet([[0.2, 0.3, 0.5], [0.5, 0.3, 0.2]], size=(4, 2)).eval().shape
# (4, 2, 3)

Again, what you were observing was just a bug.

[brandonwillard]

Also the categorical distribution is a univariate one, and has the same limitation as other multivariate distributions.

The categorical should work, too:

at.random.categorical([[0.2, 0.3, 0.5], [0.5, 0.3, 0.2]], size=(4, 2,)).eval().shape
# (4, 2)

[ricardoV94]

I disagree with converting this to a discussion. The original example clearly shows a different interpretation/inconsistency between the meaning of size in univariate and multivariate distributions.

These checks were introduced by the PR that intended to harmonize the meaning across the two classes of distributions.

It also shows the distinct interpretation being used for the univariate categorical distribution that just so happens to require a vector parameter.

It looks like (the independent dimensions of parameters) should be allowed to broadcast to size. If not, the last example with the normal should also fail (and similar checks should be introduced to all existing univariate distributions).

[brandonwillard]

Do not convert this into another issue until some sort of resolution is arrived upon via this discussion.

If you read the original PR (i.e. #446) and the associated discussion in #412, you'll see that the current approach is well justified. If you believe that there are benefits to a different approach, or errors/issues with the current, then you need to make the relevant comparisons and justifications clear in this discussion.

As of now, you haven't addressed the consistency issue that was explicitly addressed by the current approach, especially since what you're proposing is only a mild convenience in some cases and introduces an important inconsistency: it implements broadcasting in the opposite direction as the standard NumPy broadcasting rules.

More specifically, your example in #1056, i.e.

import numpy as np
import aesara.tensor as at

# Already allowed before
at.random.normal(np.ones((3, 1, 3)), size=(3, 10, 3)).eval().shape  # (3, 10, 3)

# Allowed now
at.random.dirichlet(np.ones((3, 1, 3)), size=(3, 10)).eval().shape  # (3, 10, 3)

it proposes that an array of shape (3, 1, 3) (i.e. matching the independent parameters given by np.ones((3, 1, 3))) should "broadcast" against the "size" (3, 10), but that's not how broadcasting works. Broadcasting goes from the rightmost dimensions—of both shapes—to the left, adding broadcast dimensions wherever one array is missing a dimension that the other has.

Your proposed change implements broadcasting in the opposite direction, and this is an inconsistency we don't need. The only way to reconcile this inconsistency is by saying that size works differently for multivariate distributions, but this is the kind of "special-casing" and unnecessary complexity that #412 was addressing.

I thought your "Already allowed before" example was somehow implying that the at.random.normal case should work with size=(3, 10) as well, like the dirichlet example that follows.

The original example clearly shows a different interpretation/inconsistency between the meaning of size in univariate and multivariate distributions.

The original example looks like it was just a bug, and shouldn't have anything to do with the interpretation/meaning of size between univariate and multivariate distributions.

[brandonwillard]

Wait, if you're only talking about broadcasting the dimensions other than the support dimension, then this might not be an issue, because it would be in-line with #446 and #412.

[brandonwillard]

Even if there is a good reason, the message is wrong, because the batched shapes may very well be broadcastable to size:

# Raises ValueError: shape mismatch: objects cannot be broadcast to a single shape
# However batch_shape=(1,) and size=(5,) are clearly broadcastable
at.random.dirichlet([[0.2, 0.3, 0.5]], size=5).eval()

This could simply be a bug in the current implementation of RandomVariable, and not a general issue with the way size is interpreted, because—for instance—at.random.dirichlet([[0.2, 0.3, 0.5]], size=(5,)).shape.eval() yields [5, 3]. Also, at.random.dirichlet([[0.2, 0.3, 0.5], [0.2, 0.3, 0.5]], size=(2,)).eval() works, so it seems to be a problem specific to broadcastable independent dimensions.

[brandonwillard]

Now, that the original example issue is resolved by #1056, it's clear that the problem was specific to the same copy-pasted code used by MvNormalRv.rng_fn, MultinomialRV.rng_fn, and DirichletRV.rng_fn, and not our handling of multivariate distribution parameters in general (e.g. in the RandomVariable methods or anywhere else).