Teams often jump to streaming because it feels modern — but ‘real-time’ is only valuable when it changes decisions. Streaming also changes your operational model: stateful processing, late events, exactly-once semantics, and new observability needs.
Use streaming when: the decision is time-sensitive
- Fraud detection and risk scoring
- Operational alerts (machine downtime, SLA breaches)
- User personalization during a session
- Inventory availability and logistics
Prefer batch when: the question is analytical
- Monthly financial reporting
- Marketing cohort analysis
- Most KPI dashboards (hourly is often enough)
Key design choices
- Event schema + versioning (contracts matter more in streaming).
- State management (windows, watermarks, dedupe).
- Backpressure and replay strategy.
- Sink semantics (exactly-once vs at-least-once + dedupe).
textLate events happen. Decide: drop? update? re-open windows? Your business requirement should drive the policy.
Hybrid pattern: micro-batch + SLAs
A common production sweet spot is micro-batching every 5–15 minutes: near-real-time visibility with far less complexity than true streaming, and easier backfills.