Gleam Lab · Tag Archive
Tag: realtime-warehouse
5 articles collected by topic for tutorials, cases, engineering practice, and research notes.
Big Data 268 - Real-time Warehouse ODS Layer: Writing Kafka Dimension Tables into DIM
Kafka is a distributed streaming platform for high-throughput message passing. In ETL processes, Kafka serves as a data message queue or stream processing source.
Big Data 269 - Real-time Warehouse DIM, DW and ADS: Scala Pipelines to HBase
Original MySQL area table to HBase: Convert area table to region ID, region name, city ID, city name, province ID, province name, and write to HBase.
Big Data #266: Canal Integration with Kafka - Real-time Data Sync
This article introduces Alibaba's open-source Canal tool, which implements Change Data Capture (CDC) by parsing MySQL binlog.
Big Data 267 - Real-Time Warehouse ODS: Lambda and Kappa Architecture
In internet companies, common ODS data includes business log data (Log) and business DB data.
Big Data 261 - Real-Time Warehouse Business Table Structure
Realtime data warehouse is a data warehouse system that differs from traditional batch processing data warehouses by emphasizing low latency, high throughput.