Big Data 183 - Elasticsearch Concurrency Conflicts & Optimistic Lock

1. Concurrency Conflict Scenario

Taking e-commerce inventory deduction as example:

  1. Thread A reads inventory: count = 10
  2. Thread B reads inventory: count = 10
  3. Thread A deducts: count = 9, writes
  4. Thread B deducts: count = 9, writes

Result: Inventory should be 8, actually 9, write overwrite caused data error.

2. Optimistic Lock vs Pessimistic Lock

2.1 Pessimistic Lock

  • Lock data every time
  • Low concurrency capability

2.2 Optimistic Lock

  • Document contains version field
  • Version number increments on each modification
  • Compare version numbers on write

3. ES Optimistic Lock Implementation

Use _seq_no and _primary_term for conditional write:

PUT /index/_doc/1?if_seq_no=5&if_primary_term=1
{
  "field": "value"
}
  • Success: Normal update
  • Conflict: Returns HTTP 409 version_conflict_engine_exception

4. Distributed Consistency

ES 5.0 before

?consistency=quorum  # Must have majority of replicas respond
?consistency=one      # Must have primary shard respond
?consistency=all      # Must have all replicas respond

ES 5.0 after

?wait_for_active_shards=2  # Must have 2 active replicas

5. Summary

  • Use optimistic lock in concurrency scenarios
  • ES implements conditional write through if_seq_no + if_primary_term
  • Distributed consistency controlled through wait_for_active_shards