Step by Step Installation

A step by step guide from 0 to 1

Step 1: Prepare a base configuration

Streambased Enterprise requires only a single short config file to start. An example can be seen below:

# ---------- Kafka Configuration --------------- #
bootstrap.servers=pkc-12576z.us-west2.gcp.confluent.cloud:9092
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username='7FZAJHHSUUENVXPX' password='Mkh6SWV0EGxgQwwHo2Zpd+YJ25e+uuNIGIqRNSsmVzI+39cAWJTCmG4tIPcdaCWE';
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
client.dns.lookup=use_all_dns_ips
schema.registry.schema.registry.url=https://psrc-7nzodo.us-west2.gcp.confluent.cloud
schema.registry.basic.auth.credentials.source=USER_INFO
schema.registry.basic.auth.user.info=VIH5FYUKUPNHGK2U:rO17jCAiSnfDJMW/QzsFwH/P/ALLY5O/PW9CRsm5NujOZNzku8Q38xEUcgZjiQ24

board.size=1000
use.schema.registry=true

# ---------- Indexing Configuration --------------- #
field.extractors=purchasesExtractor

field.extractors.purchasesExtractor.class=io.streambased.index.extractor.JsonValueFieldsExtractor
field.extractors.purchasesExtractor.topic=purchases
field.extractors.purchasesExtractor.useSchemaRegistry=true

We provide a demo cluster for experimentation. To point at your own Kafka deployment re-point the Kafka connection properties to your environment.

This configuration file contains 2 sections:

  1. Kafka client configuration - This should contain connection properties (Java client style) for the Kafka cluster you wish to connect to.

  2. Indexing configuration - This should contain configuration for the topics you wish to index. For more information on this please see configuration

Save this example configuration as client.properties in your environment, make any changes necessary and mount as below.

Step 2: Start Streambased Enterprise

An indexer can be started with the following command:

docker run -v ${PWD}/client.properties:/etc/streambased/etc/client.properties -p 8080:8080 streambased/streambased-enterprise:latest

Step 3: Connect to Streambased Enterprise

Streambased exposes an interface that is 100% compatible with that which is exposed by the popular Trino database. This means connectivity can be established using components from the Trino ecosystem including:

Tutorials on connecting your favorite analytical applications to Streambased can be found here.

In addition to the based Trino functionality, Streambased supports the following extra session variables to aid interaction with Kafka. These can be set via the SET SESSION directive within a JDBC session.

streambased_connection

Used to pass a JSON map of Apache Kafka connection properties (typically security configs for impersonation). Streambased provides a handy tool to convert java properties to the correct format.

  • Type: string

  • Default: empty

  • Example:

SET SESSION streambased_connection = '{ "sasl.jaas.config":"org.apache.kafka.common.security.plain.PlainLoginModule required username=''restricted'' password=''restricted-secret'';"}';

Note: Remember to double up any single quotes in your Kafka connection configuration as shown above.

use_streambased

Enable or disable Streambased acceleration for the session.

  • Type: string

  • Default: true

  • Example:

SET SESSION use_streambased = true;

Step 4: Scale

Streambased Enterprise can scale alongside your Kafka infrastructure to practically unlimited capability. Please reach out to our team to schedule your free architecture assessment.

PreviousRequirementsNextConfiguration

Last updated