Data Masking & String Truncation Improvements

[jongpie]

Bundled Data Masking Rules Bugfixes + New American Express (AMEX) Masking Rule

• Bugfix: Fixed issues in the regular expressions (regexes) used in the 3 existing LogEntryDataMaskRule__mdt records (stored in the field SensitiveDataRegEx__c). The previous regexes were too aggressive, and would sometimes incorrectly mask substrings of non-credit card & non-social security number values.
  • SocialSecurityNumber
  • MastercardCreditCardNumber
  • VisaCreditCardNumber
• Enhancement: Added a new bundled record in LogEntryDataMaskRule__mdt for American Express (AMEX) credit card numbers - AmericanExpressCreditCardNumber (similar to the existing rules for Visa & Mastercard)

Data Masking Bugfixes + Truncation Bugfixes + New *Truncated__c Boolean Fields

• Bugfix: Fixed Text truncation does not always properly truncate (especially when combined with data masking rules) #695 by adding another round of string truncation after applying data masking to fix issues where string values could still be too long for the corresponding fields

  • Context: Some data masking rules can (intentionally) cause a text value's length to increase. For example, social security numbers are formatted with - dashes when masked, so values like 123456789 (9 characters) are 2 characters longer when masked as ***-**-6789 (11 characters).
  • In previous versions of Nebula Logger, text was truncated, then masked. With a combination of long text values + matching data to mask, the resulting text value could be too long. This was especially a problem for the Message__c field in several orgs, but the same issue could/did happen with several other text fields.
  • In this release, the text is truncated again after masking to ensure the data will fit into the corresponding field.

• Optimization: Refactored some duplicated logic in LogEntryEventBuilder for truncating + masking various text values

  • There are several fields being masked + truncated (and more fields will probably be added in the future), and the existing code had a lot of code duplication happening.

• Enhancement: Added several new *Truncated__c boolean fields on both LogEntryEvent__e & LogEntry__c for some data points where it would be helpful to have the context that data has been truncated. There are 5 sets of new fields included:

  1. LogEntryEvent__e.HttpRequestBodyTruncated__c ➡️ LogEntry__c.HttpRequestBodyTruncated__c
  2. LogEntryEvent__e.HttpResponseBodyTruncated__c ➡️ LogEntry__c.HttpResponseBodyTruncated__c
  3. LogEntryEvent__e.RecordJsonTruncated__c ➡️ LogEntry__c.RecordJsonTruncated__c
  4. LogEntryEvent__e.RestRequestBodyTruncated__c ➡️ LogEntry__c.RestRequestBodyTruncated__c
  5. LogEntryEvent__e.RestResponseBodyTruncated__c ➡️ LogEntry__c.RestResponseBodyTruncated__c

• Enhancement: Updated the LogEntryRecordPage flexipage to add the new *Truncated__c boolean fields on LogEntry__c (mentioned above, and shown below)

  • These fields are configured to only show on the page when true, similar to the behavior used for the existing *Masked__c fields, like LogEntry__c.MessageMasked__c & LogEntry__c.RecordJsonMasked__c

  

  

  

  

  

Pipeline Improvements for extra-tests Directory

• DevOps/Pipeline Improvement: Added new pipeline-only test classes with integration tests to validate that the custom metadata type (CMDT) records bundled with Nebula Logger work as expected
  • This includes validating records in LogEntryDataMaskRule__mdt, LoggerParameter__mdt, LoggerSObjectHandler__mdt, and LogStatus__mdt
  • Since orgs can choose to modify or delete the bundled CMDT records, this level of testing needs to happen just in the pipeline
• Re-organized the Apex test classes + metadata in nebula-logger/extra-tests to make it a little more clear what everything is being used for. The number of files has grown a lot over the last few years, so a little cleanup was needed.
  • This directory's metadata is only used by the pipeline to more thoroughly test some features & metadata that's bundled with Nebula Logger - none of it is included when installing Nebula Logger, so these changes should not have any direct impact to people using Nebula Logger.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 91.49%. Comparing base (323fa77) to head (7b28450).

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #866      +/-   ##
==========================================
+ Coverage   86.19%   91.49%   +5.30%     
==========================================
  Files          17       58      +41     
  Lines        1318     6314    +4996     
  Branches      199      199              
==========================================
+ Hits         1136     5777    +4641     
- Misses        162      517     +355     
  Partials       20       20              

Flag	Coverage Δ
Apex	92.89% <100.00%> (?)
LWC	86.19% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.