DOC-5600 Pulsar connector pages

antora.yml

@@ -17,4 +17,16 @@ asciidoc:
     starlight-rabbitmq: 'Starlight for RabbitMQ'
     pulsar-reg: 'Apache Pulsar(TM)'
     pulsar: 'Apache Pulsar'
-    pulsar-short: 'Pulsar'
+    pulsar-short: 'Pulsar'
+
+    # Required for include::common partials that are shared with Astra Streaming
+    web-ui: 'Admin Console'
+    astra: 'Astra'
+    astra-db: 'Astra DB'
+    scb: 'Secure Connect Bundle (SCB)'
+    scb-short: 'SCB'
+    scb-brief: 'Secure Connect Bundle'
+    cass-reg: 'Apache Cassandra(R)'
+    cass: 'Apache Cassandra'
+    cass-short: 'Cassandra'
+    debezium-version: '1.7'

modules/ROOT/nav.adoc

@@ -23,6 +23,28 @@
 * xref:operations:auth.adoc[]
 * xref:operations:functions.adoc[]
 * xref:operations:functions-transform.adoc[]
-* xref:operations:io-connectors.adoc[]
+* I/O connectors
+** xref:connectors:index.adoc[]
+** Sinks
+*** xref:connectors:sinks/astra-db.adoc[]
+*** xref:connectors:sinks/cloud-storage.adoc[]
+*** xref:connectors:sinks/elastic-search.adoc[]
+*** xref:connectors:sinks/google-bigquery.adoc[]
+*** xref:connectors:sinks/jdbc-clickhouse.adoc[]
+*** xref:connectors:sinks/jdbc-mariadb.adoc[]
+*** xref:connectors:sinks/jdbc-postgres.adoc[]
+*** xref:connectors:sinks/jdbc-sqllite.adoc[]
+*** xref:connectors:sinks/kafka.adoc[]
+*** xref:connectors:sinks/kinesis.adoc[]
+*** xref:connectors:sinks/snowflake.adoc[]
+** Sources
+*** xref:connectors:sources/data-generator.adoc[]
+*** xref:connectors:sources/debezium-mongodb.adoc[]
+*** xref:connectors:sources/debezium-mysql.adoc[]
+*** xref:connectors:sources/debezium-oracle.adoc[]
+*** xref:connectors:sources/debezium-postgres.adoc[]
+*** xref:connectors:sources/debezium-sqlserver.adoc[]
+*** xref:connectors:sources/kafka.adoc[]
+*** xref:connectors:sources/kinesis.adoc[]
 * xref:operations:scale-cluster.adoc[]
 * xref:operations:troubleshooting.adoc[]

modules/ROOT/pages/faqs.adoc

@@ -81,7 +81,7 @@ https://github.com/datastax/pulsar-heartbeat[{pulsar-short} Heartbeat] monitors
 
 == What are the features provided by {company} {pulsar} Connector (`pulsar-sink`) that are not supported in `kafka-sink`?
 
-The https://pulsar.apache.org/docs/en/io-overview/[{pulsar-short} IO framework] provides many features that are not possible in Kafka, and has different compression formats and auth/security features. The features are handled by {pulsar-short}. For more, see xref:operations:io-connectors.adoc[Luna Streaming IO Connectors].
+The https://pulsar.apache.org/docs/en/io-overview/[{pulsar-short} IO framework] provides many features that are not possible in Kafka, and has different compression formats and auth/security features. The features are handled by {pulsar-short}. For more, see xref:connectors:index.adoc[Luna Streaming IO Connectors].
 
 The {company} {pulsar} Connector allows single-record acknowledgement and negative acknowledgements.
 

modules/ROOT/pages/index.adoc

@@ -17,7 +17,7 @@ In addition to the distribution of https://pulsar.apache.org/en/versions/[{pulsa
 
 * A xref:install-upgrade:quickstart-helm-installs.adoc[Helm chart] that deploys and manages {pulsar-short} on your current CNCF-conformant Kubernetes infrastructure
 
-* Cassandra, Elastic, Kinesis, Kafka, and JDBC xref:operations:io-connectors.adoc[connectors]
+* Cassandra, Elastic, Kinesis, Kafka, and JDBC xref:connectors:index.adoc[connectors]
 
 * The xref:components:starlight.adoc[Starlight suite of {pulsar-short} protocol handlers for Kafka, RabbitMQ, and JMS]
 

modules/ROOT/partials/sinks/edit.adoc

@@ -0,0 +1,17 @@
+To update a connector, pass the new configuration definition to the connector.
+For example, if you created the connector with a configuration file, you can pass an updated configuration file.
+
+You can include the entire configuration or only the properties that you want to change.
+Additionally, some properties can be modified with specific arguments, such as `--parallelism`.
+
+To get the current configuration, see xref:connectors:index.adoc#get-sink-connector-configuration-data[Get sink connector configuration data].
+
+[source,shell,subs="+attributes"]
+----
+./bin/pulsar-admin sinks update \
+  --sink-type {connectorType} \
+  --name "$SINK_NAME" \
+  --inputs "persistent://$TENANT/$NAMESPACE/$TOPIC" \
+  --tenant "$TENANT" \
+  --parallelism 2
+----

modules/ROOT/partials/sinks/get-started.adoc

@@ -0,0 +1,32 @@
+. Optional: If you are using the `pulsar-admin` CLI, set the following commonly-used environment variables:
++
+[source,shell,subs="+quotes"]
+----
+export TENANT="**TENANT_NAME**"
+export TOPIC="**INPUT_TOPIC_NAME**"
+export NAMESPACE="**NAMESPACE_NAME**"
+export SINK_NAME="**SINK_CONNECTOR_UNIQUE_NAME**"
+----
++
+`**SINK_NAME**` is the name for your new sink connector.
+{company} recommends a memorable, human-readable name that summarizes the connector's purpose.
+For example: `{connectorType}-sink-prod-us-east-1`.
+
+. Create the connector using JSON-formatted connector configuration data.
+You can pass the configuration directly or with a configuration file.
++
+[source,shell,subs="+attributes"]
+----
+./bin/pulsar-admin sinks create \
+  --sink-type {connectorType} \
+  --name "$SINK_NAME" \
+  --inputs "persistent://$TENANT/$NAMESPACE/$TOPIC" \
+  --tenant "$TENANT" \
+  --sink-config-file configs.json
+----
++
+.Example configuration data structure
+[source,json]
+----
+include::common:streaming:example$connectors/sinks/{connectorType}/sample-data.json[]
+----

modules/ROOT/partials/sources/edit.adoc

@@ -0,0 +1,17 @@
+To update a connector, pass the new configuration definition to the connector.
+For example, if you created the connector with a configuration file, you can pass an updated configuration file.
+
+You can include the entire configuration or only the properties that you want to change.
+Additionally, some properties can be modified with specific arguments, such as `--parallelism`.
+
+To get the current configuration, see xref:connectors:index.adoc#get-source-connector-configuration-data[Get source connector configuration data].
+
+[source,shell,subs="+attributes"]
+----
+./bin/pulsar-admin sources update \
+  --source-type {connectorType} \
+  --name "$SOURCE_NAME" \
+  --destination-topic-name "persistent://$TENANT/$NAMESPACE/$TOPIC" \
+  --tenant "$TENANT" \
+  --parallelism 2
+----

modules/ROOT/partials/sources/get-started.adoc

@@ -0,0 +1,32 @@
+. Optional: If you are using the `pulsar-admin` CLI, set the following commonly-used environment variables:
++
+[source,shell,subs="+quotes"]
+----
+export TENANT="**TENANT_NAME**"
+export TOPIC="**OUTPUT_TOPIC_NAME**"
+export NAMESPACE="**NAMESPACE_NAME**"
+export SOURCE_NAME="**SOURCE_CONNECTOR_UNIQUE_NAME**"
+----
++
+`**SOURCE_NAME**` is the name for your new source connector.
+{company} recommends a memorable, human-readable name that summarizes the connector's purpose.
+For example: `{connectorType}-source-prod-us-east-1`.
+
+. Create the connector using JSON-formatted connector configuration data.
+You can pass the configuration directly or with a configuration file.
++
+[source,shell,subs="+attributes"]
+----
+./bin/pulsar-admin sources create \
+  --source-type {connectorType} \
+  --name "$SOURCE_NAME" \
+  --destination-topic-name "persistent://$TENANT/$NAMESPACE/$TOPIC" \
+  --tenant "$TENANT" \
+  --source-config-file configs.json
+----
++
+.Example configuration data structure
+[source,json]
+----
+include::common:streaming:example$connectors/sources/{connectorType}/sample-data.json[]
+----