Blog
Technical exploration and thoughts · 655 articles
Apache Kylin Cube Practice: Hive Load & Pre-computation A...
OLAP example: Generate dimension and fact data via Python, after Hive (wzk_kylin) load, design Cube in Kylin (dimensions/measures/Cuboids), and provide...
Apache Kylin Cube Practice: From Modeling to Build and Qu...
Apache Kylin (3.x/4.x) Cube setup and optimization: complete flow from DataSource → Model → Cube, covering dimension modeling, measure design, Cuboid...
From MapReduce to Spark: Big Data Computing Evolution
Systematic overview of big data processing engine evolution from MapReduce to Spark to Flink, analyzing Spark in-memory computing model, unified ecosystem and core components.
Apache Kylin Comprehensive Guide: MOLAP Architecture, Hiv...
Background, evolution and engineering practice of Apache Kylin, focusing on MOLAP solution implementation path for massive data analysis. Core keywords: Apache...
Apache Kylin 3.1.1 Deployment on Hadoop 2.9/Hive 2.3/HBas...
Complete deployment record of Apache Kylin 3.1.1 on Hadoop 2.9.2, Hive 2.3.9, HBase 1.3.1, Spark 2.4.5 (without-hadoop, Scala 2.12) and three-node...
Kafka Storage Mechanism: Log Segmentation & Retention
Deep analysis of Kafka log storage architecture, including LogSegment design, sparse offset index and timestamp index principles, message lookup flow, and log retention and cleanup strategy configu...
Kafka High Performance: Zero-Copy, mmap & Sequential Write
Deep dive into Kafka's three I/O technologies achieving high throughput: sendfile zero-copy, mmap memory mapping and page cache sequential write, revealing kernel-level optimization behind million ...
Kafka Replica Mechanism: ISR & Leader Election
Deep dive into Kafka replica mechanism, including ISR sync node set maintenance, Leader election process, and unclean election trade-offs between consistency and availability.
Kafka Exactly-Once: Idempotence & Transactions
Systematic explanation of how Kafka achieves Exactly-Once semantics through idempotent producers and transactions, covering PID/sequence number principle, cross-partition transaction configuration ...
Apache Druid Storage & Query Architecture: Segment/Chunk/...
Apache Druid data storage and high-performance query path: from DataSource/Chunk/Segment layering, to columnar storage, Roll-up pre-aggregation, Bitmap...
Apache Druid + Kafka Real-time Analysis: JSON Flattening ...
Scala Kafka Producer writes order/click data to Kafka Topic (example topic: druid2), continuous ingestion in Druid through Kafka Indexing Service. Since...
Apache Druid Real-time Kafka Ingestion: Complete Practice...
Complete practice of Apache Druid real-time Kafka ingestion, using network traffic JSON as example, completing data ingestion through Druid console's Streaming/Kafka wizard, parsing time column, se...
Apache Druid Architecture & Component Responsibilities: C...
Apache Druid component responsibilities and deployment points from 0.13.0 to current (2025): Coordinator manages Historical node Segment...
Apache Druid Cluster Deployment [Part 1]: MySQL Metadata ...
Apache Druid 30.0.0 deployable solution covering MySQL metadata storage (mysql-connector-java 8.0.19), HDFS deep storage and HDFS indexing-logs, plus Kafka...
Apache Druid Cluster Mode [Part 2]: Low-Memory Cluster Pr...
Low-memory cluster practice for Apache Druid 30.0.0 on three nodes: provides JVM parameters and runtime.properties key items for Broker/Historical/Router, explains off-heap memory and processing bu...
Kafka Topic, Partition & Consumer: Rebalance Optimization
Deep dive into Kafka Topic, Partition, Consumer Group core mechanisms, covering custom deserialization, offset management and rebalance optimization configuration.
Kafka Topic Management: Commands & Java API
Comprehensive introduction to Kafka Topic operations, including kafka-topics.sh commands, replica assignment strategy principles, and KafkaAdminClient Java API core usage.
Apache Druid Real-time OLAP Architecture & Selection Points
Apache Druid real-time OLAP practice: suitable for event detail with time as primary key, sub-second aggregation and high-concurrency self-service analysis.
Apache Druid Single-Machine Deployment: Architecture Over...
Apache Druid 30.0.0 for single-machine quick verification and engineering implementation, systematically reviewing Druid architecture (Coordinator, Historical,...
Flink Write to Kudu Practice: Custom Sink Full Process (F...
Complete runnable example for Kudu, based on Flink 1.11.1 (Scala 2.12)/Java 11 and kudu-client 1.17.0 (2025 test). Through RichSinkFunction custom sink,...