Skip to content

Commit 3dc3838

Browse files
committed
Regenerate opensearch plugin documentation
1 parent 150430b commit 3dc3838

File tree

1 file changed

+22
-177
lines changed

1 file changed

+22
-177
lines changed

docs/plugins/fluentd/output/opensearch.md

Lines changed: 22 additions & 177 deletions
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,6 @@
22

33
Opensearch defines the parameters for out_opensearch plugin
44

5-
## Configuration Parameters
6-
7-
### Basic Connection
85

96
| Field | Description | Scheme |
107
| ----- | ----------- | ------ |
@@ -13,26 +10,11 @@ Opensearch defines the parameters for out_opensearch plugin
1310
| hosts | Hosts defines a list of hosts if you want to connect to more than one Openearch nodes | *string |
1411
| scheme | Specify https if your Opensearch endpoint supports SSL (default: http). | *string |
1512
| path | Path defines the REST API endpoint of Opensearch to post write requests (default: nil). | *string |
16-
17-
### Authentication
18-
19-
| Field | Description | Scheme |
20-
| ----- | ----------- | ------ |
21-
| user | Optional, The login credentials to connect to Opensearch | *[plugins.Secret](../secret.md) |
22-
| password | Optional, The login credentials to connect to Opensearch | *[plugins.Secret](../secret.md) |
23-
24-
### Index Configuration
25-
26-
| Field | Description | Scheme |
27-
| ----- | ----------- | ------ |
2813
| indexName | IndexName defines the placeholder syntax of Fluentd plugin API. See https://docs.fluentd.org/configuration/buffer-section. | *string |
2914
| logstashFormat | If true, Fluentd uses the conventional index name format logstash-%Y.%m.%d (default: false). This option supersedes the index_name option. | *bool |
3015
| logstashPrefix | LogstashPrefix defines the logstash prefix index name to write events when logstash_format is true (default: logstash). | *string |
31-
32-
### SSL/TLS Configuration
33-
34-
| Field | Description | Scheme |
35-
| ----- | ----------- | ------ |
16+
| user | Optional, The login credentials to connect to Opensearch | *[plugins.Secret](../secret.md) |
17+
| password | Optional, The login credentials to connect to Opensearch | *[plugins.Secret](../secret.md) |
3618
| sslVerify | Optional, Force certificate validation | *bool |
3719
| caFile | Optional, Absolute path to CA certificate file | *string |
3820
| clientCert | Optional, Absolute path to client Certificate file | *string |
@@ -41,178 +23,41 @@ Opensearch defines the parameters for out_opensearch plugin
4123
| sslVersion | Optional, You can specify SSL/TLS version (default: TLSv1_2) | *string |
4224
| sslMinVersion | Optional, Minimum SSL/TLS version | *string |
4325
| sslMaxVersion | Optional, Maximum SSL/TLS version | *string |
44-
45-
### Debugging & Error Handling
46-
47-
| Field | Description | Scheme |
48-
| ----- | ----------- | ------ |
49-
| logOs400Reason | Optional, Enable logging of 400 reason without enabling debug log level (default: false). **Critical for troubleshooting 400 errors!** | *bool |
50-
| reconnectOnError | Optional, Indicates that the plugin should reset connection on any error (reconnect on next send) (default: false) | *bool |
51-
| ignoreExceptions | Optional, List of exception classes to ignore | *string |
52-
| exceptionBackup | Optional, Backup chunk when ignore exception occurs (default: true) | *bool |
53-
54-
### Connection Management
55-
56-
| Field | Description | Scheme |
57-
| ----- | ----------- | ------ |
26+
| logOs400Reason | Optional, Enable logging of 400 reason without enabling debug log level (default: false) | *bool |
5827
| requestTimeout | Optional, HTTP request timeout in seconds (default: 5s) | *string |
28+
| reconnectOnError | Optional, Indicates that the plugin should reset connection on any error (reconnect on next send) (default: false) | *bool |
5929
| reloadConnections | Optional, Automatically reload connection after 10000 documents (default: true) | *bool |
6030
| reloadAfter | Optional, When ReloadConnections true, this is the integer number of operations after which the plugin will reload the connections (default: 10000) | *uint32 |
6131
| reloadOnFailure | Optional, Indicates that the opensearch-transport will try to reload the nodes addresses if there is a failure while making the request (default: false) | *bool |
62-
63-
### Performance Tuning
64-
65-
| Field | Description | Scheme |
66-
| ----- | ----------- | ------ |
67-
| compressionLevel | Optional, You can specify the compression level (default: no_compression). Options: no_compression, best_compression, best_speed, default_compression | *string |
32+
| maxRetryGetOsVersion | Optional, You can specify times of retry obtaining OpenSearch version (default: 15) | *uint32 |
33+
| failOnDetectingOsVersionRetryExceed | Optional, Indicates whether to fail when max_retry_get_os_version is exceeded (default: true) | *bool |
34+
| defaultOpensearchVersion | Optional, Default OpenSearch version (default: 1) | *uint32 |
35+
| verifyOsVersionAtStartup | Optional, Validate OpenSearch version at startup (default: true) | *bool |
36+
| templateOverwrite | Optional, Always update the template, even if it already exists (default: false) | *bool |
37+
| maxRetryPuttingTemplate | Optional, You can specify times of retry putting template (default: 10) | *uint32 |
38+
| failOnPuttingTemplateRetryExceed | Optional, Indicates whether to fail when max_retry_putting_template is exceeded (default: true) | *bool |
39+
| snifferClassName | Optional, Provide a different sniffer class name | *string |
40+
| selectorClassName | Optional, Provide a selector class name | *string |
6841
| httpBackend | Optional, You can specify HTTP backend (default: excon). Options: excon, typhoeus | *string |
6942
| httpBackendExconNonblock | Optional, With http_backend_excon_nonblock false, plugin uses excon with nonblock=false (default: true) | *bool |
43+
| compressionLevel | Optional, You can specify the compression level (default: no_compression). Options: no_compression, best_compression, best_speed, default_compression | *string |
7044
| preferOjSerializer | Optional, With default behavior, plugin uses Yajl as JSON encoder/decoder. Set to true to use Oj (default: false) | *bool |
71-
| bulkMessageRequestThreshold | Optional, Configure bulk_message request splitting threshold size (default: -1 unlimited) | *int32 |
72-
73-
### Record Handling
74-
75-
| Field | Description | Scheme |
76-
| ----- | ----------- | ------ |
45+
| suppressTypeName | Optional, Suppress '[types removal]' warnings on OpenSearch 2.x (default: true for OS2+) | *bool |
46+
| contentType | Optional, With content_type application/x-ndjson, plugin adds application/x-ndjson as Content-Type (default: application/json) | *string |
7747
| includeTagKey | Optional, Include tag key in record (default: false) | *bool |
7848
| tagKey | Optional, Tag key name when include_tag_key is true (default: tag) | *string |
7949
| idKey | Optional, Record accessor syntax to specify the field to use as _id in OpenSearch | *string |
8050
| removeKeys | Optional, Remove specified keys from the event record | *string |
8151
| removeKeysOnUpdate | Optional, Remove keys when record is being updated | *string |
8252
| writeOperation | Optional, The write operation (default: index). Options: index, create, update, upsert | *string |
8353
| emitErrorForMissingId | Optional, When write_operation is not index, setting this true will cause plugin to emit_error_event of records which do not include _id field (default: false) | *bool |
84-
85-
### Template & Version Management
86-
87-
| Field | Description | Scheme |
88-
| ----- | ----------- | ------ |
89-
| templateOverwrite | Optional, Always update the template, even if it already exists (default: false) | *bool |
90-
| maxRetryPuttingTemplate | Optional, You can specify times of retry putting template (default: 10) | *uint32 |
91-
| failOnPuttingTemplateRetryExceed | Optional, Indicates whether to fail when max_retry_putting_template is exceeded (default: true) | *bool |
92-
| verifyOsVersionAtStartup | Optional, Validate OpenSearch version at startup (default: true) | *bool |
93-
| maxRetryGetOsVersion | Optional, You can specify times of retry obtaining OpenSearch version (default: 15) | *uint32 |
94-
| failOnDetectingOsVersionRetryExceed | Optional, Indicates whether to fail when max_retry_get_os_version is exceeded (default: true) | *bool |
95-
| defaultOpensearchVersion | Optional, Default OpenSearch version (default: 1) | *uint32 |
96-
| applicationName | Optional, Specify the application name for the rollover index to be created (default: default) | *string |
97-
| indexDatePattern | Optional, Specify the index date pattern for creating a rollover index (default: now/d) | *string |
98-
| useLegacyTemplate | Optional, Use legacy template or not (default: false for composable templates) | *bool |
99-
100-
### Advanced Options
101-
102-
| Field | Description | Scheme |
103-
| ----- | ----------- | ------ |
104-
| suppressTypeName | Optional, Suppress '[types removal]' warnings on OpenSearch 2.x (default: true for OS2+) | *bool |
105-
| contentType | Optional, With content_type application/x-ndjson, plugin adds application/x-ndjson as Content-Type (default: application/json) | *string |
10654
| customHeaders | Optional, Custom headers in Hash format | *string |
10755
| pipeline | Optional, Pipeline name | *string |
10856
| utcIndex | Optional, UTC index (default: false for local time) | *bool |
10957
| suppressDocWrap | Optional, Suppress doc_wrap (default: false) | *bool |
110-
| snifferClassName | Optional, Provide a different sniffer class name | *string |
111-
| selectorClassName | Optional, Provide a selector class name | *string |
112-
113-
## Examples
114-
115-
### Basic Configuration
116-
117-
```yaml
118-
apiVersion: fluentd.fluent.io/v1alpha1
119-
kind: ClusterOutput
120-
metadata:
121-
name: opensearch-basic
122-
spec:
123-
outputs:
124-
- opensearch:
125-
host: opensearch.logging.svc.cluster.local
126-
port: 9200
127-
logstashFormat: true
128-
logstashPrefix: app-logs
129-
```
130-
131-
### Production Configuration
132-
133-
```yaml
134-
apiVersion: fluentd.fluent.io/v1alpha1
135-
kind: ClusterOutput
136-
metadata:
137-
name: opensearch-production
138-
spec:
139-
outputs:
140-
- opensearch:
141-
# Connection
142-
hosts: "opensearch-node1:9200,opensearch-node2:9200,opensearch-node3:9200"
143-
scheme: https
144-
145-
# Authentication
146-
user:
147-
valueFrom:
148-
secretKeyRef:
149-
name: opensearch-creds
150-
key: username
151-
password:
152-
valueFrom:
153-
secretKeyRef:
154-
name: opensearch-creds
155-
key: password
156-
157-
# SSL/TLS with mutual authentication
158-
sslVerify: true
159-
caFile: /etc/ssl/opensearch/ca.crt
160-
clientCert: /etc/ssl/opensearch/client.crt
161-
clientKey: /etc/ssl/opensearch/client.key
162-
sslMinVersion: TLSv1_2
163-
sslMaxVersion: TLSv1_3
164-
165-
# Index configuration
166-
logstashFormat: true
167-
logstashPrefix: kubernetes-logs
168-
169-
# CRITICAL: Enable 400 error debugging
170-
logOs400Reason: true
171-
172-
# Reliability
173-
reconnectOnError: true
174-
requestTimeout: 30s
175-
reloadConnections: true
176-
reloadOnFailure: true
177-
178-
# Performance
179-
compressionLevel: best_speed
180-
bulkMessageRequestThreshold: 20971520 # 20MB
181-
182-
# Document handling
183-
includeTagKey: true
184-
idKey: kubernetes.pod_id
185-
writeOperation: upsert
186-
```
187-
188-
### Development/Debug Configuration
189-
190-
```yaml
191-
apiVersion: fluentd.fluent.io/v1alpha1
192-
kind: ClusterOutput
193-
metadata:
194-
name: opensearch-debug
195-
spec:
196-
outputs:
197-
- opensearch:
198-
host: opensearch-dev.local
199-
port: 9200
200-
scheme: http
201-
202-
# Enable detailed error logging
203-
logOs400Reason: true
204-
205-
# Relaxed SSL for development
206-
sslVerify: false
207-
208-
# Simple index
209-
indexName: dev-logs
210-
211-
# Debug helpers
212-
includeTagKey: true
213-
suppressDocWrap: false
214-
215-
# Don't fail on errors during development
216-
failOnPuttingTemplateRetryExceed: false
217-
failOnDetectingOsVersionRetryExceed: false
218-
```
58+
| ignoreExceptions | Optional, List of exception classes to ignore | *string |
59+
| exceptionBackup | Optional, Backup chunk when ignore exception occurs (default: true) | *bool |
60+
| bulkMessageRequestThreshold | Optional, Configure bulk_message request splitting threshold size (default: -1 unlimited) | *int32 |
61+
| applicationName | Optional, Specify the application name for the rollover index to be created (default: default) | *string |
62+
| indexDatePattern | Optional, Specify the index date pattern for creating a rollover index (default: now/d) | *string |
63+
| useLegacyTemplate | Optional, Use legacy template or not (default: false for composable templates) | *bool |

0 commit comments

Comments
 (0)