Skip to content

perf: eliminate program join from enrollment count query DHIS2-20921#22972

Merged
teleivo merged 2 commits intomasterfrom
DHIS2-20921-eliminate-program-join
Feb 17, 2026
Merged

perf: eliminate program join from enrollment count query DHIS2-20921#22972
teleivo merged 2 commits intomasterfrom
DHIS2-20921-eliminate-program-join

Conversation

@teleivo
Copy link
Contributor

@teleivo teleivo commented Feb 16, 2026

Follow-up to #22970. Optimizes enrollment queries when a program is provided.

When a program is given, both data and count queries replace
p.type = 'WITH_REGISTRATION' and p.uid = :programUid with e.programid = :programId. The
p.type check is safe to drop because EnrollmentOperationParamsMapper validates the program is a
tracker program. The count query additionally drops the program and trackedentity joins (the
data query still needs them for SELECT columns). When a trackedEntity UID filter is provided in
the count query without the trackedentity join, a subquery is used instead
(e.trackedentityid in (select trackedentityid from trackedentity where uid in (...))).

SQL

Data query (when program is specified) -- joins unchanged, WHERE simplified:

-- before (#22970)
where p.type = 'WITH_REGISTRATION' and p.uid = :programUid ...

-- after
where e.programid = :programId ...

Count query (when program is specified) -- program and trackedentity joins removed:

-- before (#22970)
select count(*)
from enrollment e
inner join program p on p.programid = e.programid
inner join trackedentity te on te.trackedentityid = e.trackedentityid
inner join trackedentityprogramowner po on ...
inner join organisationunit ou on ...
inner join (...) as coc on ...
where p.type = 'WITH_REGISTRATION' and p.uid = :programUid
  and ...

-- after
select count(*)
from enrollment e
inner join trackedentityprogramowner po on ...
inner join organisationunit ou on ...
inner join (...) as coc on ...
where e.programid = :programId
  and ...

Database Performance

Sierra Leone DB with 10M tracked entities (10.9M enrollments). EXPLAIN ANALYZE on the generated SQL
queries, 4 warmup runs. All /enrollments requests use program=ur1Edk5Oe2n&pageSize=3. User
types: unmarked = normal user (restricted org unit scope, 2 facilities), admin = search-all
authority with root org unit, super = superuser.

Request #22970 This PR Change
fields=enrollment 32ms 17ms -48%
fields=enrollment (admin) 9ms 9ms --
fields=enrollment (super) 8ms 8ms --
orgUnitMode=DESCENDANTS (admin) 10ms 9ms --
orgUnitMode=SELECTED (admin) 10ms 7,921ms *
orgUnitMode=ALL (admin) 9ms 8ms --
status=ACTIVE (admin) 9ms 8ms --
followUp=true (admin) 9ms 9ms --
enrolledAfter&enrolledBefore (admin) 9ms 9ms --
updatedAfter (admin) 9ms 9ms --
updatedWithin=30d (admin) 9ms 10ms --
trackedEntity=<uid> (admin) 8ms 8ms --
enrollments=<uid>,<uid> (admin) 8ms 9ms --
includeDeleted=true (admin) 9ms 9ms --
order=enrolledAt:desc 398ms 859ms +116%
order=enrolledAt:desc (admin) 26,656ms 4,294ms -84%
order=enrolledAt:desc (super) 28,131ms 3,990ms -86%
order=createdAt:desc (admin) 28,535ms 4,133ms -86%
order=completedAt:desc (admin) 28,547ms 4,602ms -84%
order=updatedAt:desc 63ms 48ms --
order=updatedAt:desc (admin) 9ms 9ms --
totalPages=true count (admin) 14,783ms 4,923ms -67%
fields=* (admin) 9ms 9ms --
orgUnitMode=DESCENDANTS&status=ACTIVE&enrolledAfter&order=enrolledAt:desc (admin) 3,289ms 3,105ms --
status=ACTIVE&followUp=true&fields=* (admin) 9ms 9ms --
/trackedEntities?fields=enrollments (admin) 9ms 9ms --

* orgUnitMode=SELECTED returns 0 rows due to hierarchy level mismatch (pre-existing).

The multi-second times on order-by and totalPages queries come from the admin user's root org unit
scope matching all ~10.9M enrollments with no filters. The database must sort or count all of them.
Normal users scoped to a few facilities see sub-second times. The remaining bottleneck is missing
indexes on enrolledAt, createdAt, and completedAt -- unlike /trackedEntities where
enrolledAt lives in a different table, here all orderable fields are on the enrollment table
and indexable.

The order=enrolledAt:desc regression for the normal user (+116%, no filters besides ownership) is
a plan change: PostgreSQL picks a parallel Gather Merge with the simpler WHERE instead of a nested
loop. Both are sub-second (398ms vs 859ms). The same user with order=updatedAt:desc (indexed)
sees 48ms with no regression -- the index avoids the full sort entirely.

The last row shows the enrollment query triggered by
/trackedEntities?program=ur1Edk5Oe2n&orgUnit=ImspTQPwCqd&pageSize=3&fields=enrollments. Known
clients (web capture app, Android SDK) never call /enrollments as a list endpoint. They fetch
enrollments embedded in tracked entity responses, which runs the enrollment query filtered by
tracked entity UIDs (9ms, high selectivity).

When a specific program is provided, resolve its ID in Java and filter
with e.programid = :programId instead of joining the program table and
filtering on p.uid/p.type. The program type check is already validated
by EnrollmentOperationParamsMapper before reaching the store.

This removes one inner join from the count query for the common case
(program specified). The data query still joins program for SELECT
columns.
@teleivo teleivo force-pushed the DHIS2-20921-eliminate-program-join branch 2 times, most recently from 59af617 to df5a8f3 Compare February 17, 2026 08:02
…20921

When the program is known and its access level is OPEN, AUDITED, or
CLOSED, the trackedentity join is unnecessary in the count query. The
join was only needed for the PROTECTED temp owner check in the
ownership clause, which references te.trackedentityid.

For tracked entity UID filters, uses a subquery instead of a join to
avoid hashing all 10M rows.
@teleivo teleivo force-pushed the DHIS2-20921-eliminate-program-join branch from df5a8f3 to 7f492da Compare February 17, 2026 08:20
@sonarqubecloud
Copy link

@teleivo teleivo merged commit 6767b84 into master Feb 17, 2026
16 checks passed
@teleivo teleivo deleted the DHIS2-20921-eliminate-program-join branch February 17, 2026 17:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants