[Tracking] Large customer account performance and UX support

[mountiny]

This is a tracking issue for proactive and direct support of one of our largest users. Our goal is to track their performance and metrics, monitor crashes and errors so that we can fix them proactively.

Their performance is going to suffer from the large amounts of data, but that should be a good example for us to address and improve iteratively using the spans and profiling from Sentry. They can also provide a tight feedback loop on the issues they are experiencing, which is a great opportunity for us.

⚠️ IMPORTANT: Please make sure not to post any customer data, domain names and names in general in the issue.

More details and sub-issues to come in here later

Issue Owner

Current Issue Owner: @TMisiukiewicz

[TMisiukiewicz]

I found a Sentry trace where user spent ~30 seconds in createOptionList. The profile is very fragmented but the common pattern is thousands of short executions of functions that iterate over all policies. This lines up with what we know about the account having a few thousand policies

I started scanning the codebase for places where we iterate over entire policies collection. The plan is to remove or optimize those where possible, and add unit + regression tests where it makes sense

As a next step, we could do a broader audit of the codebase to validate all usages of full-collection iteration and see where we can eliminate them entirely.

The Sentry profile originates from OptionsListContextProvider, which we plan to remove soon. As discussed with @mountiny, we want to offload that work to the community. That said, removing the provider alone likely won't fix this issue - the expensive full-collection iterations will still run regardless of the surrounding implementation when opening search screens

[youssef-lr]

I started scanning the codebase for places where we iterate over entire policies collection. The plan is to remove or optimize those where possible, and add unit + regression tests where it makes sense

@TMisiukiewicz I was thinking about this as well and I think to applies to other callections as well like reports, transaction, etc. Maybe we can start by investigating components that rely on a single policy/report/transaction, but they still subscribe to the whole collections?