Error on recursive opaque ty in HIR typeck

[compiler-errors]

"Non-trivially recursive" opaques are opaques whose hidden types are inferred to be equal to something other than themselves. For example, if we have a TAIT like type TAIT = impl Sized, if we infer the hidden type to be TAIT := (TAIT,), that would be a non-trivial recursive definition. We don't want to support opaques that are non-trivially recursive, since they will (almost!! -- see caveat below) always result in borrowck errors, and are generally a pain to deal with.

On the contrary, trivially recursive opaques may occur today because the old solver overagerly uses replace_opaque_types_with_inference_vars. This infer var can then later be constrained to be equal to the opaque itself. These cases will not necessarily result in borrow-checker errors, since other uses of the opaque may properly constrain the opaque. If there are no other uses we may instead fall back to () today.

The only weird case that we have to unfortunately deal with was discovered in #139406:

#![allow(unconditional_recursion)]

fn what1<T>(x: T) -> impl Sized {
    what1(x)
}

fn what2<T>(x: T) -> impl Sized {
    what2(what2(x))
}

fn print_return_type<T, U>(_: impl Fn(T) -> U) {
    println!("{}", std::any::type_name::<U>())
}

fn main() {
    print_return_type(what1::<i32>); // ()
    print_return_type(what2::<i32>); // i32
}

HIR typeck eagerly replaces the return type with an infer var, ending up with RPIT<T> = RPIT<RPIT<T>> in the storage. While we return this in the TypeckResults, it's never actually used anywhere.

MIR building then results in the following statement

let _0: impl RPIT<T> /* the return place */ = build<RPIT<T>>(_some_local);

Unlike HIR typeck MIR typeck now directly equates RPIT<T> with RPIT<RPIT<T>>. This does not try to define RPIT but instead relates its generic arguments

rust/compiler/rustc_infer/src/infer/relate/type_relating.rs

Lines 185 to 190 in b9856b6

	(
	&ty::Alias(ty::Opaque, ty::AliasTy { def_id: a_def_id, .. }),
	&ty::Alias(ty::Opaque, ty::AliasTy { def_id: b_def_id, .. }),
	) if a_def_id == b_def_id => {
	super_combine_tys(infcx, self, a, b)?;
	}

This means we relate T with RPIT<T>, which results in a defining use RPIT<T> = T

I think it's pretty obvious that this is not desirable behavior, and according to the crater run there were no regressions, so let's break this so that we don't have any inference hazards in the new solver.

In the future what2 may end up compiling again by also falling back to (). However, that is not yet guaranteed and the transition to this state is made significantly harder by not temporarily breaking it on the way. It is also concerning to change the inferred hidden type like this without any notification to the user, even if likely not an issue in this concrete case.