13/07/2020

In support of workload-aware streaming state management

Vasiliki Kalavri, John Liagouris

Keywords:

Abstract: Modern distributed stream processors predominantly rely on LSM-based key-value stores to manage the state of long-running computations. We question the suitability of such general-purpose stores for streaming workloads and argue that they incur unnecessary overheads in exchange for state management capabilities. Since streaming operators are instantiated once and are long-running, state types, sizes, and access patterns, can either be inferred at compile time or learned during execution. This paper surfaces the limitations of established practices for streaming state management and advocates for configurable streaming backends, tailored to the state requirements of each operator. Using state management, we achieve an order of magnitude improvement in p99 latency and 2x higher throughput.

 0
 0
 0
 0
This is an embedded video. Talk and the respective paper are published at HotStorage 2020 virtual conference. If you are one of the authors of the paper and want to manage your upload, see the question "My papertalk has been externally embedded..." in the FAQ section.

Comments

Post Comment
no comments yet
code of conduct: tbd

Similar Papers