Configurations

Streambased K.S.I. is configured by the standard Kroxylicious configuration scheme here or by equivalent environment variables documented here:

Environment Variables

KSI_BOOTSTRAP_SERVERS

Bootstrap servers for the underlying Kafka cluster

• Type: string

• Example:

  KSI_BOOTSTRAP_SERVERS=kafka1:9092

---

KSI_GATEWAY_BOOTSTRAP_ADDRESS

The address that clients should connect to to access the proxy.

• Type: string

• Example:

  KSI_GATEWAY_BOOTSTRAP_ADDRESS=ksi:9192

---

KSI_COLD_STORAGE_TYPE

The method in which K.S.I. will interact with Iceberg, supported values are:

* spark-connect

* iceberg

• Type: string

• Example:

  KSI_COLD_STORAGE_TYPE=spark-connect

---

KSI_SPARK_CONNECT_URL

The Url of Spark-Connect, this should only be used when KSI_COLD_STORAGE_TYPE is spark-connect.

• Type: string

• Example:

  KSI_SPARK_CONNECT_URL=sc://spark-iceberg:15002

---

KSI_MAX_COLD_STORAGE_RECORDS_PER_FETCH

The maximum number of Iceberg records to return for any signle FETCH response.

• Type: integer

• Example:

  KSI_MAX_COLD_STORAGE_RECORDS_PER_FETCH=1000

---

KSI_COLD_STORAGE_TIMEOUT_MS

The maximum time K.S.I. will wait for records from Iceberg before returning to Kroxylicious

• Type: integer

• Example:

  KSI_COLD_STORAGE_TIMEOUT_MS=30000

---

KSI_ALWAYS_RETURN_MINUS_ONE_START_OFFSET

Whether or not K.S.I. should return a -1 start offset for any topics it manages. A value of true indicates that this topic has infiinite retention and is managed by K.S.I.

• Type: boolean

• Example:

  KSI_ALWAYS_RETURN_MINUS_ONE_START_OFFSET=true

---

KSI_ICEBERG_NAMESPACE

The namespace that contains the Iceberg data

• Type: string

• Example:

  KSI_ICEBERG_NAMESPACE=coldset

---

KSI_SPARK_CATALOG_NAME

The catalog that contains the Iceberg data

• Type: string

• Example:

  KSI_SPARK_CATALOG_NAME=direct

---

KSI_SCHEMA_REGISTRY_URL

The Schema Registry url for retrieving topic schemas

• Type: string

• Example:

  KSI_SCHEMA_REGISTRY_URL=http://schema-registry:808