Fix ISO-8601 date format preservation during data migration

[Copilot]

Date strings like "2023-12-19T00:00:00.000Z" were being changed to "2023-12-19T00:00:00Z" during Cosmos-to-Cosmos migration due to inconsistent serialization settings.

Changes

• CosmosDataSinkExtension.cs: CreateItemStream now uses RawJsonCosmosSerializer.GetDefaultSettings() instead of default JsonConvert.SerializeObject, ensuring consistent date handling between stream and non-stream write modes

• DataItemJsonConverter.cs (core layer): Added DateTimeOffset support alongside existing DateTime handling, using GetAsUnescaped() to preserve + in timezone offsets

// Before
var json = JsonConvert.SerializeObject(item);

// After
var json = JsonConvert.SerializeObject(item, RawJsonCosmosSerializer.GetDefaultSettings());

Tests Added

• CreateItemStream_WithDateString_PreservesFormat
• CreateItemStream_WithNestedDateString_PreservesFormat
• CreateItemStream_WithDateStringArray_PreservesFormat
• DateTimeOffset test cases in DataItemJsonConverterTests

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• http://crl3.digicert.com:443/DigiCertCSRSA4096RootG5.crl
  • Triggering command: /usr/bin/dotnet dotnet build CosmosDbDataMigrationTool.sln (http block)
• http://crl3.digicert.com:443/DigiCertTrustedG4TimeStampingRSA4096SHA2562025CA1.crl
  • Triggering command: /usr/bin/dotnet dotnet build CosmosDbDataMigrationTool.sln (http block)
• http://crl3.digicert.com:443/NETFoundationProjectsCodeSigningCA2.crl
  • Triggering command: /usr/bin/dotnet dotnet build CosmosDbDataMigrationTool.sln (http block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

Original prompt

---

This section details on the original issue you should resolve

<issue_title>ISO-8601 Date Format String is Changed After Data Migration</issue_title>
<issue_description>We are trying to migrate data from cosmosdb container A to B using the tool, one of the fields of the record is a ISO-8601 string as follows:

{
  "event_time": "2023-12-19T00:00:00.000Z"
}

However, after migrating to container B, the record is saved as follows:

{
  "event_time": "2023-12-19T00:00:00Z"
}

the content of event_time is changed, which is not expected. We found a similar issue reported before: #36</issue_description>

<agent_instructions>Can you check to make sure that date formats are preserved between source and sink. This should be the case for every extension. Try to make the fix for this in the core layer so as not to have to update every source and sink extension in this tool.</agent_instructions>

Comments on the Issue (you are @copilot in this section)

• Fixes ISO-8601 Date Format String is Changed After Data Migration #103

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

[philnach]

Looks good to me. Fixes a long standing issue.