You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+27-27Lines changed: 27 additions & 27 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -17,25 +17,25 @@ A Kafka Connect Sink for Splunk features:
17
17
18
18
## Build
19
19
20
-
1. Clone the repo from https://github.com/splunk/kafka-connect-splunk
20
+
1. Clone the repo from https://github.com/splunk/splunk-kafka-connect
21
21
2. Verify that Java8 JRE or JDK is installed.
22
22
3. Run `bash build.sh`. The build script will download all dependencies and build the Splunk Kafka Connector.
23
23
24
-
Note: The resulting "kafka-connect-splunk-*.tar.gz" package is self-contained. Bundled within it are the Kafka Connect framework, all 3rd party libraries, and the Splunk Kafka Connector.
24
+
Note: The resulting "splunk-kafka-connect-*.tar.gz" package is self-contained. Bundled within it are the Kafka Connect framework, all 3rd party libraries, and the Splunk Kafka Connector.
25
25
26
26
## Quick Start
27
27
28
28
1.[Start](https://kafka.apache.org/quickstart) your Kafka Cluster and confirm it is running.
29
29
2. If this is a new install, create a test topic (eg: `perf`). Inject events into the topic. This can be done using [Kafka data-gen-app](https://github.com/dtregonning/kafka-data-gen) or the Kafka bundle [kafka-console-producer](https://kafka.apache.org/quickstart#quickstart_send).
30
-
3. Untar the package created from the build script: `tar xzvf kafka-connect-splunk-*.tar.gz` (Default target location is /tmp/kafka-connect-splunk-build/kafka-connect-splunk).
31
-
4. Navigate to kafka-connect-splunk directory `cd kafka-connect-splunk`.
30
+
3. Untar the package created from the build script: `tar xzvf splunk-kafka-connect-*.tar.gz` (Default target location is /tmp/splunk-kafka-connect-build/splunk-kafka-connect).
31
+
4. Navigate to splunk-kafka-connect directory `cd splunk-kafka-connect`.
32
32
5. Adjust values for `bootstrap.servers` and `plugin.path` inside `config/connect-distributed-quickstart.properties` to fit your environment. Default values should work for experimentation.
33
33
6. Run `./bin/connect-distributed.sh config/connect-distributed-quickstart.properties` to start Kafka Connect.
34
34
7. Run the following command to create connector tasks. Adjust `topics` to set the topic, and `splunk.hec.token` to set your HEC token.
35
35
36
36
```
37
37
curl localhost:8083/connectors -X POST -H "Content-Type: application/json" -d '{
See the [the Confluent doucumentation](https://docs.confluent.io/current/connect/managing.html#common-rest-examples) for additional REST examples.
@@ -98,11 +98,11 @@ Use the following connector deployment options:
98
98
### Connector in a dedicated Kafka Connect Cluster
99
99
Running the Splunk Kafka Connector in a dedicated Kafka Connect Cluster is recommended. Isolating the Splunk connector from other Kafka connectors results in significant performance benefits in high throughput environments.
100
100
101
-
1. Untar the **kafka-connect-splunk-*.tar.gz** package and navigate to the **kafka-connect-splunk** directory.
101
+
1. Untar the **splunk-kafka-connect-*.tar.gz** package and navigate to the **splunk-kafka-connect** directory.
102
102
103
103
```
104
-
tar xzvf kafka-connect-splunk-*.tar.gz
105
-
cd kafka-connect-splunk
104
+
tar xzvf splunk-kafka-connect-*.tar.gz
105
+
cd splunk-kafka-connect
106
106
```
107
107
108
108
2. Update config/connect-distributed.properties to match your environment.
@@ -118,26 +118,26 @@ Running the Splunk Kafka Connector in a dedicated Kafka Connect Cluster is recom
118
118
> Note: The below topics should be created by Kafka Connect when deploying the Splunk Connector. If the Kafka Connect cluster **does not have permission** to create these topics, create these manually before starting Kafka Connect cluster.
119
119
120
120
```
121
-
group.id=kafka-connect-splunk-hec-sink # consumer group id of Kafka Connect, which is used to form a Kafka Connect cluster
121
+
group.id=splunk-kafka-connect-hec-sink # consumer group id of Kafka Connect, which is used to form a Kafka Connect cluster
122
122
123
-
config.storage.topic=__kafka-connect-splunk-task-configs # kafka topic used to persistent connector task configurations
123
+
config.storage.topic=__splunk-kafka-connect-task-configs # kafka topic used to persistent connector task configurations
124
124
config.storage.replication.factor=3
125
125
126
-
offset.storage.topic=__kafka-connect-splunk-offsets # kafka topic used to persistent task checkpoints
126
+
offset.storage.topic=__splunk-kafka-connect-offsets # kafka topic used to persistent task checkpoints
127
127
offset.storage.replication.factor=3
128
128
offset.storage.partitions=25
129
129
130
-
status.storage.topic=__kafka-connect-splunk-statuses # kafka topic used to persistent task statuses
130
+
status.storage.topic=__splunk-kafka-connect-statuses # kafka topic used to persistent task statuses
131
131
status.storage.replication.factor=3
132
132
status.storage.partitions=5
133
133
```
134
134
135
-
4. Deploy/Copy the **kafka-connect-splunk** directory to all target hosts (virtual machines, physical machines or containers).
135
+
4. Deploy/Copy the **splunk-kafka-connect** directory to all target hosts (virtual machines, physical machines or containers).
136
136
5. Start Kafka Connect on all target hosts using the below commands:
> Note: The **KAFKA\_HEAP\_OPTS** environment variable controls how much memory Kafka Connect can use. Set the **KAFKA\_HEAP\_OPTS** with the recommended value stated in the example above.
0 commit comments