Wednesday, July 27, 2011

LevelDB Open-Sources by Google

LevelDB is a fast key-value storage engine written at Google that provides an ordered mapping from string keys to string values. We are pleased to announce that we are open sourcing LevelDB under a BSD-style license.

LevelDB is a C++ library that can be used in many contexts. For example, LevelDB may be used by a web browser to store a cache of recently accessed web pages, or by an operating system to store the list of installed packages and package dependencies, or by an application to store user preference settings. We designed LevelDB to also be useful as a building block for higher-level storage systems. Upcoming versions of the Chrome browser include an implementation of the IndexedDB HTML5 API that is built on top of LevelDB. Google's Bigtable manages millions of tablets where the contents of a particular tablet are represented by a precursor to LevelDB. The Riak distributed database has added support for using LevelDB for its per-node storage.

We structured LevelDB to have very few dependencies and it can be easily ported to new systems; it has already been ported to a variety of Unix based systems, Mac OS X, Windows, and Android.

LevelDB has good performance across a wide variety of workloads; we have put together a benchmark comparing its performance to SQLite and Kyoto Cabinet. The Riak team has compared LevelDB’s performance to InnoDB. A significant difference from similar systems like SQLite and Kyoto Cabinet is that LevelDB is optimized for batch updates that modify many keys scattered across a large key space. This is an important requirement for efficiently updating an inverted index that does not fit in memory.

LevelDB is available on Google Code, we hope you’ll find it useful for your projects.

Benchmarks available here at

# Posted via email from opportunity__cost

Tuesday, July 5, 2011

Do you like the taste of beer?

Among all our casual topics, whether someone likes the taste of beer is the single best predictor of if he or she has sex on the first date.

OkTrends is so awesome.

# Posted via email from opportunity__cost

Monday, July 4, 2011

Twitter is shifting into JVM

Last year the company announced that both its back-end message queue and Tweet storage had been re-written in Scala, and in the spring of 2010 the search team at Twitter started to rewrite the search engine. As part of the effort, Twitter changed the search storage from MySQL to a real-time version of Lucene. More recently the team announced that they were replacing the Ruby on Rails front-end for search with a Java server they called Blender. This change resulted in a 3x drop in search latencies.

# Posted via email from opportunity__cost

Saturday, July 2, 2011