ECE Computation

[GloryyrolG]

Hi Sinz Lab,

Appreciate your code release for the inspiring work.

May I ask a question on the computation of ECE, in

https://github.com/sinzlab/cGNF/blob/b126121a40f96e8086a98c1a4f19604b1071db6f/notebooks/toy_problem/minMPJPE_and_ECE_landscapes.ipynb

Why is it

x_cdf_vals = np.abs(x_cdf_vals - 0.5) * 2

?

Thanks & Regards,

[GloryyrolG]

Um, I think you implemented an interval prop_type instead of a quantile one.

        prop_type: "interval" to measure observed proportions for centered prediction intervals,
                   and "quantile" for observed proportions below a predicted quantile.

Ref: https://github.com/uncertainty-toolbox/uncertainty-toolbox/blob/f7ca1ec948b71e5214577015b3f831a39785d707/uncertainty_toolbox/metrics_calibration.py#L98