Prepare marks contract #87

Raytesnel · 2026-01-15T12:58:18Z

initial draft for the endpoint prepare marks.

I put TODO's for the implementation.

i need to look at it again, and see if i missed somthing (like a test or so).

@cfs-data / @SimoneAriens in the docstrings and the description of the input/output schemas i just wrote somthing that could be wrong. can you check if it is okay or missing somthing? after that i can format the docstrings.

src/constants.py

snregales · 2026-01-15T13:13:29Z

src/extractors/schemas.py

+class PrepareMarkResponse(ProcessedDataAccess):
+    """Response model for prepared mark data access."""
+
+    mark_mat: HttpUrl = Field(
+        ...,
+        description="Data for the prepared mark.",
+        examples=["http://localhost:8000/preprocessor/files/surface_comparator_859lquto/mark.mat"],
+        alias="mark_mat",
+    )
+    processed_mat: HttpUrl = Field(
+        ...,
+        description="Data for the processed mark.",
+        examples=["http://localhost:8000/preprocessor/files/surface_comparator_859lquto/processed.mat"],
+        alias="processed_mat",
+    )
+    profile_mat: HttpUrl = Field(
+        ...,
+        description="Data for the profile of the prepared mark.",
+        examples=["http://localhost:8000/preprocessor/files/surface_comparator_859lquto/profile.mat"],
+        alias="profile_mat",
+    )
+    levelled_mat: HttpUrl = Field(
+        ...,
+        description="Data for the levelled mark.",
+        examples=["http://localhost:8000/preprocessor/files/surface_comparator_859lquto/levelled.mat"],
+        alias="levelled_mat",
+    )


Don't know if it's already merge, but I created a model for holding just download links. ProcessedDataUrls. there is a lost of precision no key, value pair. just a list of urls.

we make a different PR, il keep this open. to remind me. and close it when a branch /ticket is made

src/preprocessors/router.py

src/preprocessors/schemas.py

snregales · 2026-01-15T13:27:25Z

src/preprocessors/schemas.py

+
+class PrepareMark(BaseParameters):
+    mark_typ: MarkType = Field(..., description="Type of mark to prepare.")
+    mask_array: list[list[float]] = Field(..., description="Array representing the mask for the mark.")


we know that mask/mask_array is a 2D array so we can at the most validate that with the tople type for instance I used tuple[tuple[float, ...], tuple[float, ...]] now I know at the minimum that it must be a nested sequence with exactly two sequences nested within it. and all containers are also immutable.

@snregales said:
we will (try) to use EncodedBytes pydantic type for mask.
https://docs.pydantic.dev/latest/api/types/#pydantic.types.Base64UrlEncoder.get_json_format

i will leave this i think for the implementation. and have small subtasks to fill in,
like make striation pipeline or make more efficient transfer of 2d array etc

tests/preprocessors/router/test_router.py

SimoneAriens · 2026-01-15T14:59:30Z

src/constants.py

 PREPROCESSOR_ROUTE = "/preprocessor"
+
+
+class StriationMarks(StrEnum):


these are also defined in scratch-core, can you take the MarkType and CropTypes from there?

those two are doing 2 different things,
this one has a distinction between 2 marks, impression and striation and a key for the code and value what comes from Java instead of both only as key
when an endpoint gets a str, it is mapped to one of those 2 enums, and we keep the string value of the enum.

the MarkType and CropTypes are enums that has values 1-x. where i still need to do a pattern match to find if it is impression or striation.

but i don't know the purpose of the other exactly. il come back, maybe we can update those. and merge then.

looks like the code is not doing anything with those types, we only use .scale function?
so we can merge it then.
but still i like the distinction between what in our code is used and what is agreed on in Java side. for example if the name got little bit different we only need to change the interface of the api and not the core package.

but it is a bit of duplication. @snregales do you have a opinion?

Raytesnel · 2026-01-15T16:05:25Z

src/extractors/schemas.py

+    """Response model for prepared mark data access."""
+
+    mark_mat: HttpUrl = Field(
+        ...,
+        description="Data for the prepared mark.",
+        examples=["http://localhost:8000/preprocessor/files/surface_comparator_859lquto/mark.mat"],
+        alias="mark_mat",
+    )
+    processed_mat: HttpUrl = Field(
+        ...,
+        description="Data for the processed mark.",
+        examples=["http://localhost:8000/preprocessor/files/surface_comparator_859lquto/processed.mat"],
+        alias="processed_mat",
+    )
+    profile_mat: HttpUrl = Field(
+        ...,
+        description="Data for the profile of the prepared mark.",
+        examples=["http://localhost:8000/preprocessor/files/surface_comparator_859lquto/profile.mat"],
+        alias="profile_mat",
+    )
+    levelled_mat: HttpUrl = Field(
+        ...,
+        description="Data for the levelled mark.",
+        examples=["http://localhost:8000/preprocessor/files/surface_comparator_859lquto/levelled.mat"],
+        alias="levelled_mat",


@SimoneAriens can you give more information on the description. instead of only "Data for <file_name>" I like to have a short summary of what is in the file.

github-actions · 2026-01-16T13:04:47Z

Diff Coverage

Diff: origin/main..HEAD, staged and unstaged changes

src/constants.py (100%)
src/extractors/schemas.py (100%)
src/preprocessors/controller.py (100%)
src/preprocessors/pipelines.py (100%)
src/preprocessors/router.py (100%)
src/preprocessors/schemas.py (100%)

Summary

Total: 88 lines
Missing: 0 lines
Coverage: 100%

github-actions · 2026-01-16T13:04:48Z

Package	Line Rate	Branch Rate	Health
.	96%	88%	✔
comparators	100%	100%	✔
container_models	99%	100%	✔
conversion	96%	89%	✔
conversion.leveling	100%	100%	✔
conversion.leveling.solver	100%	75%	✔
conversion.preprocess_impression	99%	92%	✔
extractors	95%	75%	✔
parsers	98%	67%	✔
parsers.patches	89%	60%	✔
preprocessors	100%	100%	✔
processors	100%	100%	✔
renders	98%	50%	✔
utils	91%	75%	✔
Summary	97% (1230 / 1264)	84% (133 / 158)	✔

Minimum allowed line rate is 50%

cfs-data · 2026-01-19T15:10:39Z

src/extractors/schemas.py

+        examples=["http://localhost:8000/preprocessor/files/surface_comparator_859lquto/profile.mat"],
+        alias="profile_mat",
+    )
+    levelled_mat: HttpUrl = Field(


Personally I think we should stick either with American English or British, but not mix it (e.g. levelled instead of leveled)

cfs-data · 2026-01-19T15:11:28Z

src/extractors/schemas.py

+        examples=["http://localhost:8000/preprocessor/files/surface_comparator_859lquto/mark.mat"],
+        alias="mark_mat",
+    )
+    processed_mat: HttpUrl = Field(


We are not returning .mat-files anymore, so we can maybe change the name here to mark_file, processed_file, etc?

cfs-data · 2026-01-19T15:12:32Z

src/constants.py

+
+
+class ImpressionMarks(StrEnum):
+    BREACH_FACE = "breach face impression mark"


Could we not use = auto() everywhere?

cfs-data · 2026-01-19T15:12:57Z

src/extractors/schemas.py

+        ...,
+        description="Data for the prepared mark.",
+        examples=["http://localhost:8000/preprocessor/files/surface_comparator_859lquto/mark.mat"],
+        alias="mark_mat",


alias can be removed here everywhere I think

Raytesnel requested a review from snregales January 15, 2026 12:58

Raytesnel self-assigned this Jan 15, 2026

Raytesnel marked this pull request as draft January 15, 2026 12:58

Raytesnel requested review from SimoneAriens and cfs-data January 15, 2026 12:58

snregales reviewed Jan 15, 2026

View reviewed changes

Raytesnel force-pushed the prepare-marks-contract branch from c8f02fd to 46b2438 Compare January 15, 2026 14:58

SimoneAriens reviewed Jan 15, 2026

View reviewed changes

Raytesnel commented Jan 15, 2026

View reviewed changes

Raytesnel force-pushed the prepare-marks-contract branch from 46b2438 to 8e3bd6d Compare January 15, 2026 16:12

Raytesnel and others added 6 commits January 16, 2026 11:26

define output schema

dc697e1

add input schema preprocessor

c030d2f

add endpoint contract

749d22a

update docstring route prepare_mark

7747c49

add contract test for prepare_mark

ab68e18

split endpoint in 2 endpoint, stration and impression endpoint

7c2a9bb

Raytesnel force-pushed the prepare-marks-contract branch from 6f12502 to 7c2a9bb Compare January 16, 2026 13:02

cfs-data reviewed Jan 19, 2026

View reviewed changes

		PREPROCESSOR_ROUTE = "/preprocessor"


		class StriationMarks(StrEnum):



		class ImpressionMarks(StrEnum):
		BREACH_FACE = "breach face impression mark"

Prepare marks contract #87

Are you sure you want to change the base?

Prepare marks contract #87

Uh oh!

Conversation

Raytesnel commented Jan 15, 2026

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Raytesnel Jan 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Jan 16, 2026

Diff Coverage

Diff: origin/main..HEAD, staged and unstaged changes

Summary

Uh oh!

github-actions bot commented Jan 16, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cfs-data Jan 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Raytesnel Jan 15, 2026 •

edited

Loading

cfs-data Jan 19, 2026 •

edited

Loading