Key Concepts

Architecture

Streambased A.S.K. consists of a number of identical worker units monitored and instructed by a coordinator service. A.S.K. is a shared nothing architecture with any state persisted to Apache Kafka, this means that it can scale elastically according to query load to practically infinite size.

Schema Registry

A Schema Registry is a service that provides a central repository for storing and managing schemas used by Kafka applications. It ensures that data formats are consistent and allows for schema evolution over time. Key benefits include:

  • Schema Versioning: Track changes to schemas over time.

  • Compatibility Checks: Ensure compatibility between producers and consumers of data.

  • Centralised Management: Simplify schema management across distributed systems.

JDBC (Java Database Connectivity)

JDBC is a Java-based API that allows applications to interact with databases. It provides methods for querying and updating data in a database.

  • Database Access: Standardises database connectivity for Java applications.

  • SQL Execution: Execute SQL queries and update statements.

  • Cross-Database Compatibility: Works with various relational databases through JDBC drivers.

ODBC (Open Database Connectivity)

ODBC is a standard API for accessing database management systems (DBMS). It is language-agnostic and provides a uniform interface for database interaction. Key points include:

  • Cross-Platform Support: Works with various operating systems and programming languages.

  • Database Independence: Allows applications to interact with multiple DBMS using a common interface.

  • SQL Execution: Execute SQL queries and updates, similar to JDBC.

SQLAlchemy

SQLAlchemy is a Python SQL toolkit and Object-Relational Mapping (ORM) library that provides a flexible and powerful framework for database interaction. Key points include:

  • ORM Support: Map Python classes to database tables, allowing for object-oriented database interaction.

  • Database Abstraction: Provides a consistent interface for interacting with different databases.

  • Query Generation: Simplifies the process of generating complex SQL queries using Python.

Last updated