Key Concepts

Architecture

Streambased A.S.K. consists of a number of identical worker units monitored and instructed by a coordinator service. A.S.K. is a shared nothing architecture with any state persisted to Apache Kafka, this means that it can scale elastically according to query load to practically infinite size.

Schema Registry

A Schema Registry is a service that provides a central repository for storing and managing schemas used by Kafka applications. It ensures that data formats are consistent and allows for schema evolution over time. Key benefits include:

• Schema Versioning: Track changes to schemas over time.

• Compatibility Checks: Ensure compatibility between producers and consumers of data.

• Centralised Management: Simplify schema management across distributed systems.

JDBC (Java Database Connectivity)

JDBC is a Java-based API that allows applications to interact with databases. It provides methods for querying and updating data in a database.

• Database Access: Standardises database connectivity for Java applications.

• SQL Execution: Execute SQL queries and update statements.

• Cross-Database Compatibility: Works with various relational databases through JDBC drivers.

ODBC (Open Database Connectivity)

ODBC is a standard API for accessing database management systems (DBMS). It is language-agnostic and provides a uniform interface for database interaction. Key points include:

• Cross-Platform Support: Works with various operating systems and programming languages.

• Database Independence: Allows applications to interact with multiple DBMS using a common interface.

• SQL Execution: Execute SQL queries and updates, similar to JDBC.

SQLAlchemy

SQLAlchemy is a Python SQL toolkit and Object-Relational Mapping (ORM) library that provides a flexible and powerful framework for database interaction. Key points include:

• ORM Support: Map Python classes to database tables, allowing for object-oriented database interaction.

• Database Abstraction: Provides a consistent interface for interacting with different databases.

• Query Generation: Simplifies the process of generating complex SQL queries using Python.