Precise timestamp filtering and sorting for created/modified fields

[nnpvaan]

Description of the issue/feature this PR addresses

This PR fixes timestamp filtering and sorting for API queries using created and modified fields. It ensures results are filtered and sorted with full second precision, even when catalog metadata does not include these fields

Current behavior before PR

API queries with created_since or modified_since could return records with timestamps before the specified threshold. Sorting by created and modified did not work if these fields were missing from catalog metadata

Desired behavior after PR is merged

API queries now filter records by the exact timestamp, and sorting by created and modified works reliably using application-side logic. Results are accurate to the second, regardless of catalog metadata configuration.

--
I confirm I have tested the PR thoroughly and coded it according to PEP8
standards.

[xispa]

Thanks @nnpvaan . I think that you can simplify a lot, please note the comment.

[xispa]

Instead of iterating over the brains and building a list of tuples like (brain, created, modified), I suggest to sort directly the brains using the sorted. Also note that you can rely on the get_creation_date and get_modification_date helpers from bika.lims.api, which extract the date from the brain first and fall back to the object only when the metadata is missing. The key advantage of this approach is that it avoids waking up the underlying objects unless absolutely necessary.

Something like this:

sort_on = query.get("sort_on")
reverse = query.get("sort_order") == "descending"

# DateIndex only supports minute-level precision, so we must manually sort the results
# to achieve second-level accuracy when ordering by creation or modification time.
if sort_on == "created":
   brains = sorted(results, key=lambda b: api.get_creation_date(b), reverse=reverse)
if sort_on == "modified":
   brains = sorted(results, key=lambda b: api.get_modification_date(b), reverse=reverse)

return brains

[nnpvaan]

Also note that you can rely on the get_creation_date and get_modification_date helpers from bika.lims.api, which extract the date from the brain first and fall back to the object only when the metadata is missing

We can’t reliably use get_creation_date or get_modification_date helpers on catalog brains, because those attributes may be Missing.Value if not present in the catalog metadata (that's why it does not work so far)

[xispa]

We can’t reliably use get_creation_date or get_modification_date helpers on catalog brains, because those attributes may be Missing.Value if not present in the catalog metadata (that's why it does not work so far)

The created and modified metadata retrieved from catalog brains should never return Missing.Value. All objects, both Dexterity (DX) and Archetypes (AT), provide created() and modified() functions, which always return a DateTime.

Therefore, if Missing.Value is encountered, it indicates that the metadata columns were added to the catalog (so Missing.Value was set by default), but objects were not reindexed afterwards. To resolve this, you can reindex the objects of the affected catalog directly from the Zope Management Interface (Site Setup > Management Interface). Navigate to the "Advanced" tab and click "Update Catalog".

[nnpvaan]

@xispa Thank you for your help. It works after I updated the catalog. I switched to using the get_creation_date and get_modification_date helpers