Skip to content

Revert required optional fields on read/response data classes (DM-3585)#2490

Merged
haakonvt merged 25 commits intopysdk-release-v8from
revert-haveing-all-fields-required-in-init-on-read-classes
Feb 27, 2026
Merged

Revert required optional fields on read/response data classes (DM-3585)#2490
haakonvt merged 25 commits intopysdk-release-v8from
revert-haveing-all-fields-required-in-init-on-read-classes

Conversation

@haakonvt
Copy link
Contributor

@haakonvt haakonvt commented Feb 24, 2026

See slack thread for info: https://cognitedata.slack.com/archives/C082WRQ5HSL/p1771577763338209

Partially reverts some of the changes in 8k LOC PR: #2311 😓

https://cognitedata.atlassian.net/browse/DM-3585

@haakonvt haakonvt force-pushed the revert-haveing-all-fields-required-in-init-on-read-classes branch 3 times, most recently from 6c87eb5 to 861957f Compare February 26, 2026 23:12
@haakonvt haakonvt marked this pull request as ready for review February 26, 2026 23:15
@haakonvt haakonvt requested review from a team as code owners February 26, 2026 23:15
@gemini-code-assist
Copy link
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request refines the data class definitions throughout the SDK to better reflect the optionality of fields returned by the API. By making many constructor parameters optional, the SDK becomes more resilient to partial API responses and provides clearer guidance on truly required fields for resource creation. This change also includes specific adjustments to enforce required fields where data integrity is paramount and updates type hints for improved precision. The overall impact is a more robust and type-safe SDK experience.

Highlights

  • Data Class Field Optionality: Many fields in read/response data classes across various API resources (e.g., Agent, Asset, Event, FileMetadata, Function, Relationship, Sequence, ThreeDModel, Transformation, WorkflowTrigger) have been updated to be optional in their constructors. This aligns the SDK with API responses where these fields might not always be present.
  • Required Field Enforcement: Conversely, some fields have been explicitly made required in specific data classes, such as id and is_string in Datapoints and DatapointsArray, name and data_set_id in ExtractionPipeline, name in SecurityCategory, and interval in TransformationSchedule. This ensures critical data is always provided where necessary.
  • Type Hint Refinements: The conflict_mode parameter in Transformation and TransformationJob related classes has been updated from a generic str type hint to a more precise Literal['abort', 'delete', 'update', 'upsert'], improving type safety and clarity.
  • Simplified as_write Methods: Redundant ValueError checks for None values have been removed from as_write methods in several data classes (e.g., Annotation, Asset, ExtractionPipeline, FileMetadata, Function, Geospatial, Raw, Relationship, Sequence, ThreeDModel, Transformation). This streamlines the conversion process, as the optionality is now handled at the constructor level.
  • Test Suite Updates: Integration and unit tests have been updated to reflect the changes in data class constructors, particularly for Datapoints, Asset, DataSet, Relationship, and Sequence objects, ensuring compatibility with the new field optionality.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog
  • cognite/client/_api/transformations/init.py
    • Updated type hint for conflict_mode to Literal in __call__ and list methods.
  • cognite/client/_sync_api/transformations/init.py
    • Updated type hint for conflict_mode to Literal in __call__ and list methods.
    • Updated auto-generated file hash.
  • cognite/client/data_classes/agents/agents.py
    • Added @dataclass decorator to AgentUpsert and Agent.
    • Made tools parameter more flexible in AgentUpsert and Agent constructors.
    • Adjusted tools assignment logic in AgentUpsert and Agent to handle AgentToolUpsertList and AgentToolList instances directly.
    • Removed Returns: None: No description. from AgentCore docstring.
  • cognite/client/data_classes/annotations.py
    • Removed ValueError check for annotated_resource_id in as_write method.
  • cognite/client/data_classes/assets.py
    • Made child_count, depth, and path optional in AggregateResultItem constructor.
    • Made external_id, parent_id, parent_external_id, description, data_set_id, metadata, source, labels, geo_location, aggregates optional in Asset constructor.
    • Made root_id and name required in Asset constructor.
    • Removed root_id from Asset attributes.
    • Removed ValueError check for name in Asset.as_write.
  • cognite/client/data_classes/contextualization.py
    • Made error_message optional in Job and DiagramConvertJob constructors.
    • Made file_external_id, file_instance_id, annotations, page_range, page_count optional in DiagramDetectItem constructor.
    • Made error_message optional in DiagramDetectJob, VisionExtractJob, and ContextualizationJob constructors.
  • cognite/client/data_classes/data_sets.py
    • Made external_id, name, description, metadata optional in DataSet constructor.
  • cognite/client/data_classes/datapoints.py
    • Made id and is_string required in DatapointsArray and Datapoints constructors.
    • Changed id assignment to self.id: int = id with type ignore.
  • cognite/client/data_classes/datapoints_subscriptions.py
    • Made time_series_count, filter, name, description, data_set_id optional in DataPointSubscription constructor.
  • cognite/client/data_classes/events.py
    • Made external_id, data_set_id, start_time, end_time, type, subtype, description, metadata, asset_ids, source optional in Event constructor.
  • cognite/client/data_classes/extractionpipelines.py
    • Removed cast import.
    • Made name and data_set_id required in ExtractionPipelineCore constructor.
    • Made description, raw_tables, last_success, last_failure, last_message, last_seen, schedule, contacts, metadata, source, documentation, notification_config, created_time, last_updated_time, created_by optional in ExtractionPipeline constructor.
    • Added pre-processing for contacts in ExtractionPipeline._load.
    • Updated type hint for status to Literal in ExtractionPipelineRunCore and ExtractionPipelineRun constructors.
    • Added RuntimeError for ExtractionPipelineRun.as_write.
    • Made external_id required in ExtractionPipelineConfigCore constructor.
    • Made config and description optional in ExtractionPipelineConfig constructor.
    • Removed ValueError checks in ExtractionPipeline.as_write and ExtractionPipelineConfig.as_write.
  • cognite/client/data_classes/files.py
    • Made uploaded_time, external_id, instance_id, source, mime_type, metadata, directory, asset_ids, data_set_id, labels, geo_location, source_created_time, source_modified_time, security_categories optional in FileMetadata constructor.
    • Made name required in FileMetadata constructor.
    • Removed ValueError check in FileMetadata.as_write.
  • cognite/client/data_classes/functions.py
    • Added NoReturn import.
    • Made external_id, description, owner, secrets, env_vars, cpu, memory, runtime, runtime_version, metadata, error, last_called optional in Function constructor.
    • Made function_path default to HANDLER_FILE_NAME.
    • Removed ValueError check in Function.as_write.
    • Made function_id, function_external_id, description optional in FunctionSchedule constructor.
    • Added FunctionCall.as_write to raise RuntimeError.
    • Made timestamp optional in FunctionCallLogEntry constructor.
  • cognite/client/data_classes/geospatial.py
    • Changed RESERVED_PROPERTIES to frozenset.
    • Made properties required in FeatureTypeCore and FeatureType constructors.
    • Made data_set_id and search_spec optional in FeatureType constructor.
    • Removed ValueError checks in FeatureType.as_write, CoordinateReferenceSystem.__init__, CoordinateReferenceSystem.as_write, CoordinateReferenceSystemWrite.__init__.
  • cognite/client/data_classes/iam.py
    • Made is_deleted required in Group constructor.
    • Made source_id, capabilities, attributes, deleted_time optional in Group constructor.
    • Made name required in SecurityCategoryCore and SecurityCategory constructors.
    • Removed ValueError check in SecurityCategory.as_write.
    • Made client_id optional in Session constructor.
  • cognite/client/data_classes/labels.py
    • Made name required in LabelDefinitionCore and LabelDefinition constructors.
    • Removed ValueError check in LabelDefinition.as_write.
    • Refactored Label._load_list to use a static method convert_label.
  • cognite/client/data_classes/raw.py
    • Removed ValueError checks in Row.as_write, Table.as_write, Database.as_write.
    • Removed RowWrite.__init__ and Database.__init__ as they are implicitly handled by dataclass.
    • Made created_time optional in Table and Database constructors.
    • Removed ValueError check in Database.tables_async.
  • cognite/client/data_classes/relationships.py
    • Made source, target, start_time, end_time, confidence, data_set_id, labels optional in Relationship constructor.
    • Removed ValueError check in Relationship.as_write.
  • cognite/client/data_classes/sequences.py
    • Removed warnings import.
    • Made external_id and value_type required in SequenceColumnCore constructor.
    • Made name, description, metadata optional in SequenceColumnCore constructor.
    • Made created_time, last_updated_time, value_type required in SequenceColumn constructor.
    • Made name, description, metadata optional in SequenceColumn constructor.
    • Updated SequenceColumn._load to use resource["createdTime"] and resource["lastUpdatedTime"].
    • Removed ValueError check in SequenceColumn.as_write.
    • Updated SequenceColumnWrite docstring.
    • Made columns required in Sequence constructor.
    • Made name, description, asset_id, external_id, metadata, data_set_id optional in Sequence constructor.
    • Removed warnings and DeprecationWarning related code from Sequence.__init__.
    • Updated Sequence._load to use dict[str, Any] type hint.
    • Updated SequenceWrite docstring and constructor for columns.
  • cognite/client/data_classes/shared.py
    • Made value optional in AggregateUniqueValuesResult constructor.
  • cognite/client/data_classes/three_d.py
    • Made data_set_id and metadata optional in ThreeDModel constructor.
    • Removed ValueError check in ThreeDModel.as_write.
    • Made file_id and published required in ThreeDModelRevisionCore constructor.
    • Removed ThreeDModelRevisionCore._load method.
    • Made rotation, scale, translation, camera, metadata, thumbnail_threed_file_id, thumbnail_url optional in ThreeDModelRevision constructor.
    • Removed ValueError check in ThreeDModelRevision.as_write.
    • Made parent_id, properties, bounding_box optional in ThreeDNode constructor.
    • Made asset_id, tree_index, subtree_size optional in ThreeDAssetMapping constructor.
    • Removed ValueError check in ThreeDAssetMapping.as_write.
  • cognite/client/data_classes/time_series.py
    • Made external_id, instance_id, name, metadata, unit, unit_external_id, asset_id, description, security_categories, data_set_id optional in TimeSeries constructor.
  • cognite/client/data_classes/transformations/init.py
    • Updated type hint for conflict_mode to Literal in Transformation constructor.
    • Made source_oidc_credentials, destination_oidc_credentials, running_job, last_finished_job, blocked, schedule, data_set_id, source_nonce, destination_nonce, source_session, destination_session optional in Transformation constructor.
    • Removed ValueError check in Transformation.as_write.
    • Updated type hint for conflict_mode in TransformationFilter and TransformationFilter.__init__.
  • cognite/client/data_classes/transformations/jobs.py
    • Added Literal import.
    • Made uuid and created_time required in TransformationJob constructor.
    • Made error, started_time, finished_time, last_seen_time optional in TransformationJob constructor.
    • Updated type hint for conflict_mode to Literal in TransformationJob constructor.
  • cognite/client/data_classes/transformations/schedules.py
    • Made interval required in TransformationScheduleCore constructor.
    • Removed id_ and external_id logic from TransformationSchedule.as_write.
    • Updated is_paused to be required in TransformationSchedule._load.
  • cognite/client/data_classes/transformations/schema.py
    • Made element_type optional in TransformationSchemaArrayType constructor.
    • Made fields optional in TransformationSchemaStructType constructor.
  • cognite/client/data_classes/workflows.py
    • Made input and metadata optional in WorkflowTrigger constructor.
    • Reordered fields in WorkflowTrigger.dump.
    • Made workflow_execution_id and reason_for_failure optional in WorkflowTriggerLogEntry constructor.
  • tests/tests_integration/test_api/test_datapoints.py
    • Updated DatapointsArray instantiation to include id and is_string parameters.
  • tests/tests_integration/test_api/test_synthetic_time_series.py
    • Updated Datapoints instantiation to include id and is_string parameters.
  • tests/tests_unit/conftest.py
    • Made root_id and name required in asset fixture, providing default values.
  • tests/tests_unit/test_api/test_assets.py
    • Updated mock responses for assets to include required fields rootId and name.
  • tests/tests_unit/test_api/test_data_sets.py
    • Simplified DataSet instantiation in test_update_with_resource_class.
  • tests/tests_unit/test_api/test_datapoints.py
    • Updated Datapoints instantiation to include id and is_string parameters in various tests.
  • tests/tests_unit/test_api/test_relationships.py
    • Simplified Relationship instantiation in test_update_with_resource_class.
  • tests/tests_unit/test_api/test_sequences.py
    • Simplified Sequence instantiation in test_update_with_resource_class.
  • tests/tests_unit/test_api_client.py
    • Updated mock asset response to include required fields rootId and name.
  • tests/tests_unit/test_data_classes/test_agents/test_agents.py
    • Updated agent.tools assertion from [] to None for no tools.
    • Removed tests for empty and None tools.
  • tests/tests_unit/test_data_classes/test_functions.py
    • Simplified Function instantiation in empty_function and function fixtures.
  • tests/tests_unit/test_data_classes/test_repr.py
    • Updated Datapoints and DatapointsArray instantiations to include id and is_string parameters.
    • Updated Asset instantiation to include root_id.
Activity
  • The pull request is currently in DRAFT status, indicating ongoing development or initial review.
  • The author, haakonvt, has provided a Slack thread link for additional context and discussion regarding these changes.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@haakonvt haakonvt force-pushed the revert-haveing-all-fields-required-in-init-on-read-classes branch from 861957f to 2aa1610 Compare February 26, 2026 23:18
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a number of changes across various data classes, primarily to align their definitions with the API responses by making some fields optional and others required. This is a valuable cleanup that improves type safety and robustness. The changes are extensive but appear to be consistent and well-tested. I've identified one critical issue regarding a type hint in the Datapoints class that should be addressed.

timezone: datetime.timezone | ZoneInfo | None = None,
) -> None:
self.id = id
self.id: int = id # type: ignore [assignment]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

The type hint for self.id is int, but the assigned value id can be None (as per the __init__ signature id: int | None = None). This creates a discrepancy between the static type and the possible runtime type, which can lead to TypeError exceptions in code that relies on this type hint. The type hint for self.id should be int | None to accurately reflect its possible values. If the intention is to deprecate None for id, consider issuing a warnings.warn when id is None instead of using type: ignore.

Suggested change
self.id: int = id # type: ignore [assignment]
self.id: int | None = id

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah yeah yeah, this is on purpose my love

@haakonvt haakonvt changed the title DRAFT: Revert required optional fields on read/response data classes Revert required optional fields on read/response data classes Feb 26, 2026
@haakonvt haakonvt force-pushed the revert-haveing-all-fields-required-in-init-on-read-classes branch 3 times, most recently from e52e552 to a61622a Compare February 27, 2026 07:18
@codecov
Copy link

codecov bot commented Feb 27, 2026

Codecov Report

❌ Patch coverage is 72.94118% with 23 lines in your changes missing coverage. Please review.
✅ Project coverage is 64.59%. Comparing base (b268276) to head (f95e69a).
⚠️ Report is 29 commits behind head on pysdk-release-v8.

Files with missing lines Patch % Lines
cognite/client/data_classes/sequences.py 20.00% 4 Missing ⚠️
cognite/client/data_classes/agents/agents.py 40.00% 3 Missing ⚠️
cognite/client/data_classes/functions.py 0.00% 3 Missing ⚠️
cognite/client/data_classes/labels.py 80.00% 3 Missing ⚠️
cognite/client/data_classes/extractionpipelines.py 50.00% 2 Missing ⚠️
cognite/client/data_classes/iam.py 33.33% 2 Missing ⚠️
...nite/client/data_classes/transformations/schema.py 0.00% 2 Missing ⚠️
cognite/client/data_classes/geospatial.py 0.00% 1 Missing ⚠️
cognite/client/data_classes/shared.py 0.00% 1 Missing ⚠️
...ognite/client/data_classes/transformations/jobs.py 50.00% 1 Missing ⚠️
... and 1 more
Additional details and impacted files
@@                 Coverage Diff                  @@
##           pysdk-release-v8    #2490      +/-   ##
====================================================
- Coverage             64.59%   64.59%   -0.01%     
====================================================
  Files                   467      467              
  Lines                 47789    47729      -60     
====================================================
- Hits                  30871    30832      -39     
+ Misses                16918    16897      -21     
Files with missing lines Coverage Δ
cognite/client/_api/sequence_data.py 63.90% <ø> (ø)
cognite/client/_api/transformations/__init__.py 37.14% <ø> (ø)
cognite/client/_sync_api/sequence_data.py 21.62% <ø> (ø)
...gnite/client/_sync_api/transformations/__init__.py 22.22% <ø> (ø)
cognite/client/data_classes/annotations.py 48.38% <ø> (-0.04%) ⬇️
cognite/client/data_classes/assets.py 58.84% <ø> (+0.03%) ⬆️
cognite/client/data_classes/contextualization.py 44.54% <ø> (ø)
cognite/client/data_classes/data_sets.py 36.95% <ø> (ø)
cognite/client/data_classes/datapoints.py 57.23% <100.00%> (ø)
...te/client/data_classes/datapoints_subscriptions.py 32.91% <ø> (-0.43%) ⬇️
... and 37 more

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

erlendvollset
erlendvollset previously approved these changes Feb 27, 2026
Copy link
Contributor

@erlendvollset erlendvollset left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🦄

@erlendvollset
Copy link
Contributor

Great work! I'm sure this was a lot fun to do 😄

@haakonvt haakonvt force-pushed the revert-haveing-all-fields-required-in-init-on-read-classes branch from a61622a to f95e69a Compare February 27, 2026 11:21
@haakonvt haakonvt enabled auto-merge February 27, 2026 11:23
Copy link
Contributor

@erlendvollset erlendvollset left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🦄

@haakonvt haakonvt merged commit 9294726 into pysdk-release-v8 Feb 27, 2026
20 of 28 checks passed
@haakonvt haakonvt deleted the revert-haveing-all-fields-required-in-init-on-read-classes branch February 27, 2026 11:39
@haakonvt haakonvt changed the title Revert required optional fields on read/response data classes Revert required optional fields on read/response data classes (DM-3585) Feb 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants