Mozilla PDX

Join your comrade Rustaceans at the Portland Rust Hack Night!

An evening of Rust hacking and unstructured discussion at Mozilla's Portland office. All welcome, from beginners to experts.

This paper will be presented by Johannes Hoff.

This paper describes how Facebook stores 700 million data points (timestamp and value) per minute and keeps 26 hours of it available in memory for fast access. We will mainly look at the compression algorithms making this possible.

Here is the abstract:

"""

Large-scale internet services aim to remain highly available and responsive in the presence of unexpected failures. Pro-viding this service often requires monitoring and analyzing tens of millions of measurements per second across a large number of systems, and one particularly effective solution is to store and query such measurements in a time series database (TSDB).

A key challenge in the design of TSDBs is how to strike the right balance between efficiency, scalability, and relia-bility. In this paper we introduce Gorilla, Facebook’s in-memory TSDB. Our insight is that users of monitoring sys-tems do not place much emphasis on individual data points but rather on aggregate analysis, and recent data points are of much higher value than older points to quickly detect and diagnose the root cause of an ongoing problem. Gorilla op-timizes for remaining highly available for writes and reads, even in the face of failures, at the expense of possibly drop-ping small amounts of data on the write path. To improve query efficiency, we aggressively leverage compression tech-niques such as delta-of-delta timestamps and XOR’d floating point values to reduce Gorilla’s storage footprint by 10x. This allows us to store Gorilla’s data in memory, reduc-ing query latency by 73x and improving query throughput by 14x when compared to a traditional database (HBase)-backed time series data. This performance improvement has unlocked new monitoring and debugging tools, such as time series correlation search and more dense visualization tools. Gorilla also gracefully handles failures from a single-node to entire regions with little to no operational overhead.

"""

PDF: http://www.vldb.org/pvldb/vol8/p1816-teller.pdf

Reminder: Papers We Love PDX has a code of conduct.

Thanks to Mozilla for hosting!

This month's paper is "A Promising Semantics for Relaxed-Memory Concurrency" by Kang et al. and will be presented by Nick Fitzgerald.

PDF:

https://www.mpi-sws.org/~orilahav/papers/main.pdf

------------------------------------------

What was the last paper you read and loved within the realm of computing? What did it inspire you to build or tinker with? Come share the ideas in an awesome academic/research paper with fellow engineers, programmers, and paper-readers. Lead a session and show off code that you wrote that implements these ideas or just give us the lowdown about the paper. Or, just come, listen, and discuss!

------------------------------------------

Papers We Love has a code of conduct.