Skip to content

Commit 061b0cc

Browse files
taylor-swansontaylor-swanson
authored
feat: Preserve event.original when errors occur in pipelines (#15806)
- Added append processor to global on_failure to preserve event original - Added append processor to default pipelines to preserve event original if error.message is set Affects the following integrations: - sophos - squid - stormshield - suricata - syslog_router - tetragon - watchguard_firebox - zeek
1 parent 5cc39ca commit 061b0cc

File tree

132 files changed

+819
-11
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

132 files changed

+819
-11
lines changed

‎packages/sophos/changelog.yml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,9 @@
11
# newer versions go on top
2+
- version: "3.16.0"
3+
changes:
4+
- description: Preserve event.original on pipeline error.
5+
type: enhancement
6+
link: https://github.com/elastic/integrations/pull/15806
27
- version: "3.15.4"
38
changes:
49
- description: Generate processor tags and normalize error handler.

‎packages/sophos/data_stream/utm/elasticsearch/ingest_pipeline/default.yml‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -226,6 +226,12 @@ processors:
226226
return false;
227227
}
228228
dropEmptyFields(ctx);
229+
- append:
230+
tag: append_preserve_original_event_on_error
231+
field: tags
232+
value: preserve_original_event
233+
allow_duplicates: false
234+
if: ctx.error?.message != null
229235
on_failure:
230236
- append:
231237
field: error.message
@@ -237,3 +243,7 @@ on_failure:
237243
- set:
238244
field: event.kind
239245
value: pipeline_error
246+
- append:
247+
field: tags
248+
value: preserve_original_event
249+
allow_duplicates: false

‎packages/sophos/data_stream/utm/elasticsearch/ingest_pipeline/dhcp.yml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -169,3 +169,7 @@ on_failure:
169169
- set:
170170
field: event.kind
171171
value: pipeline_error
172+
- append:
173+
field: tags
174+
value: preserve_original_event
175+
allow_duplicates: false

‎packages/sophos/data_stream/utm/elasticsearch/ingest_pipeline/dns.yml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -97,3 +97,7 @@ on_failure:
9797
- set:
9898
field: event.kind
9999
value: pipeline_error
100+
- append:
101+
field: tags
102+
value: preserve_original_event
103+
allow_duplicates: false

‎packages/sophos/data_stream/utm/elasticsearch/ingest_pipeline/http.yml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -324,3 +324,7 @@ on_failure:
324324
- set:
325325
field: event.kind
326326
value: pipeline_error
327+
- append:
328+
field: tags
329+
value: preserve_original_event
330+
allow_duplicates: false

‎packages/sophos/data_stream/utm/elasticsearch/ingest_pipeline/packetfilter.yml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -276,3 +276,7 @@ on_failure:
276276
- set:
277277
field: event.kind
278278
value: pipeline_error
279+
- append:
280+
field: tags
281+
value: preserve_original_event
282+
allow_duplicates: false

‎packages/sophos/data_stream/xg/elasticsearch/ingest_pipeline/antispam.yml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -163,3 +163,7 @@ on_failure:
163163
{{#_ingest.on_failure_processor_tag}}with tag '{{{ _ingest.on_failure_processor_tag }}}'
164164
{{/_ingest.on_failure_processor_tag}}in pipeline '{{{ _ingest.pipeline }}}'
165165
failed with message '{{{ _ingest.on_failure_message }}}'
166+
- append:
167+
field: tags
168+
value: preserve_original_event
169+
allow_duplicates: false

‎packages/sophos/data_stream/xg/elasticsearch/ingest_pipeline/antivirus.yml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -263,3 +263,7 @@ on_failure:
263263
{{#_ingest.on_failure_processor_tag}}with tag '{{{ _ingest.on_failure_processor_tag }}}'
264264
{{/_ingest.on_failure_processor_tag}}in pipeline '{{{ _ingest.pipeline }}}'
265265
failed with message '{{{ _ingest.on_failure_message }}}'
266+
- append:
267+
field: tags
268+
value: preserve_original_event
269+
allow_duplicates: false

‎packages/sophos/data_stream/xg/elasticsearch/ingest_pipeline/atp.yml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -144,3 +144,7 @@ on_failure:
144144
{{#_ingest.on_failure_processor_tag}}with tag '{{{ _ingest.on_failure_processor_tag }}}'
145145
{{/_ingest.on_failure_processor_tag}}in pipeline '{{{ _ingest.pipeline }}}'
146146
failed with message '{{{ _ingest.on_failure_message }}}'
147+
- append:
148+
field: tags
149+
value: preserve_original_event
150+
allow_duplicates: false

‎packages/sophos/data_stream/xg/elasticsearch/ingest_pipeline/cfilter.yml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -200,3 +200,7 @@ on_failure:
200200
{{#_ingest.on_failure_processor_tag}}with tag '{{{ _ingest.on_failure_processor_tag }}}'
201201
{{/_ingest.on_failure_processor_tag}}in pipeline '{{{ _ingest.pipeline }}}'
202202
failed with message '{{{ _ingest.on_failure_message }}}'
203+
- append:
204+
field: tags
205+
value: preserve_original_event
206+
allow_duplicates: false

0 commit comments

Comments
 (0)