Configurations

Streambased I.S.K. is configured via a single configuration file mounted at /app/etc/directstream-config.properties in the Streambased deployment or by equivalent environment variables.

Configurations

port

The port on which Streambased I.S.K. should expose Kafka data as a S3 compatible filesyste.

• Type: integer

• Example:

  port=11000

• Environment Variable: ISK_PORT

---

catalog.port

The port on which Streambased I.S.K. exposes an Apache Iceberg REST catalog

• Type: integer

• Example:

  catalog.port=11001

• Environment Variable: ISK_CATALOG_PORT

---

external.hostname

The hostname on which Streambased I.S.K. exposes it's services

• Type: string

• Example:

  external.hostname=streambased-isk

• Environment Variable: ISK_EXTERNAL_HOSTNAME

---

config.provider

The method I.S.K. will use to determine Kafka connection details, options are service (connection details are managed by Slipstream) or properties (connection detail are provided in I.S.K. config)

• Type: string

• Example:

  config.provider=service

• Environment Variable: ISK_CONFIG_PROVIDER

---

config.service.url

The slipstream url from which I.S.K. can fetch connection details. This applies only when config.provider is set to service.

• Type: string

• Example:

  config.service.url=http://slipstream:3000/api/internal/cluster-config

• Environment Variable: ISK_CONFIG_SERVICE_URL

---

config.sources

A comma separated list of named prefixes containing connection configs. This applies only when config.provider is set to properties. Once listed here connection configs can be supplied using the prefix config.source.[name].

• Type: string

• Example:

  config.sources=source1,source2

• Environment Variable: ISK_CONFIG_SOURCES

---

coldset.client.coldset-type

The type of Iceberg catalog used to provide coldset data. Vendor managed catalogs have different auth, metadata column renames etc. Supported types are: * default - Minio + Rest catalog with Sigv4 authentication * iceberg-rest-fixture - Minio + Rest Catalog implemented by Iceberg Rest Fixture - Sigv4 auth, requires metadata.json relocation on Coldset / Merged set proxying * tableflow - Confluent Tableflow with Confluent managed Iceberg Catalog and storage * snowflake - Snowflake managed Iceberg

• Type: string

• Example:

  coldset.client.coldset-type=tableflow

• Environment Variable: ISK_COLDSET_CLIENT_COLDSET-TYPE

---

coldset.client.coldset-ns

Iceberg namespace to use for Coldset

• Type: string

• Example:

  coldset.client.coldset-ns=COLDSET

• Environment Variable: ISK_COLDSET_CLIENT_COLDSET-NS

Configuration prefixes

The remaining configurations represent prefixes that are used to determine configurations for external services used by Streambased

---

coonfig.source.[name].*

A prefix used to pass Kafka connection configurations to Streambased I.S.K. This applies only when config.provider is set to properties and [name] is included in config.sources.

• Example:

  config.source.source1.bootstrap.servers=kafka1:9092

• Environment Prefix: ISK_CONFIG_SOURCE_[name]_

---

coldset.client.*

Additional properties for connecting to an Iceberg provider for coldset data

• Example:

  coldset.client.rest.access-key-id=admin

• Environment Prefix: ISK_CONFIG_SOURCE_[name]_