Distributed Communication: SOA/Dubbo, OpenFeign, Retry vs MQ

TL;DR

• Scenario: After distributed service splitting, inter-service communication evolved from synchronous calls to async decoupling, with retry, idempotency, and reliability issues
• Conclusion: Synchronous chains must have “bounded retry + observability + degradation”, cross-service side effects use “task-ization / MQ + idempotency + compensation” as fallback
• Output: A selection reference (RPC vs HTTP), sync/async implementation skeleton, and common fault quick reference

Distributed Architecture Communication

SOA (Service-Oriented Architecture)

In Service-Oriented Architecture (SOA), system design modularizes based on actual business requirements.

SOA Architecture Advantages:

1. Distributed features: Services can be deployed on different servers or containers
2. Loose coupling: Services interact through interface contracts
3. Flexible scaling: Individual services can be scaled based on business needs
4. Reusability: Services can be reused by multiple business systems

Dubbo + ZooKeeper Implementation:

• Dubbo as high-performance RPC framework
• ZooKeeper as distributed coordination service for service registration and discovery

Microservices

Feign Introduction

Feign is used in Spring Cloud to solve remote communication problems between services. Feign is a lightweight RESTful HTTP client.

Feign Main Features:

• Declarative REST client
• Integrates Ribbon for client-side load balancing
• Supports Hystrix circuit breaker mechanism

@FeignClient(name = "user-service")
public interface UserServiceClient {
    @GetMapping("/users/{id}")
    User getUserById(@PathVariable("id") Long id);
}

RPC vs HTTP Protocol Comparison

Feature	RPC	HTTP
Protocol	TCP/UDP	TCP
Connection	Long connection	Short connection
Serialization	Protobuf/Thrift	JSON
Performance	Higher	Lower
Debugging	More difficult	Easier

Scenarios Suitable for RPC:

• Large number of inter-system interfaces (>50)
• High-frequency interaction requirements (>1000 times/minute)
• Strict performance requirements

Scenarios Suitable for HTTP:

• Small number of inter-system interfaces (<20)
• Low interaction frequency (<100 times/minute)
• Need for rapid development and debugging

Distributed Synchronous Communication

Method 1: Synchronous Call

Long goodsId = addGoods(goods);
refreshInvertedIndex(goods);
refreshStaticPage(goods);

Synchronous Scheme with Retry:

private static final int MAX_RETRY = 3;

public Long saveGoods(Goods goods) {
    Long goodsId = addGoods(goods);
    boolean indexOk = retry(() -> indexService.refreshIndex(goods), MAX_RETRY, BASE_BACKOFF_MS);
    return goodsId;
}

Method 2: Async Task

Long goodsId = addGoods(goods);
goodsTaskService.cache(goods); // Enqueue/cache, worker refreshes later

Distributed Asynchronous Communication

Message Queue Advantages

1. System Decoupling: Producers and consumers don’t need to know each other’s existence
2. Recoverability: When consumer is temporarily unavailable, messages persist in queue
3. Heterogeneous System Support: Different tech stacks can interact through unified message protocol
4. Concurrent Processing: Multiple consumer instances can process messages in parallel
5. Traffic Peak Shaving: Bursts can be buffered by message queue

Message Queue Disadvantages

1. Middleware Bottleneck: Message queue itself may become performance bottleneck
2. Consistency Issues: Difficult to guarantee strict transactional consistency in distributed environment
3. Development Complexity: Need to handle message serialization, retry mechanism, dead letter queue issues
4. Operations Cost: Need to maintain message middleware cluster

Error Quick Reference

Symptom	Root Cause	Fix
Synchronous retry causes CPU spike/threads full	Unbounded retry, recursive calls, missing timeout and backoff	Change to for bounded retry; add timeout, backoff, circuit breaker
Recursive retry triggers StackOverflowError	Recursive without exit condition	Use iteration + max count; record failed tasks into compensation queue
Add product API RT large, throughput drops under concurrency	Main chain does index/static page side effects synchronously	Main chain only writes to DB; side effects become tasks
Async task lost	Enqueue and DB write not atomic	Outbox/transactional message: DB and task record in same transaction
Task repeated execution causing duplicate refresh	At-least-once delivery	Consumer idempotency: business unique key/deduplication table
MQ order scrambled causing state rollback	Inconsistent partition strategy, parallel consumption	Partition by business key; serialize same key processing