Home

Introduction to JetBrains Xodus

Welcome to JetBrains Xodus. These wiki pages provide a brief introduction to Xodus concepts and features. Xodus stands for exodus, whatever that means, and is pronounced as exodus. Xodus is licensed under the Apache License, Version 2.0.

Overview
    Snapshot Isolation
    Garbage Collector
    Performance

[Getting Started](https://github.com/JetBrains/xodus/wiki#getting-started)

Overview

Xodus is a transactional schema-less embedded high-performing database written in Java. It is being successfully used in several JetBrains server-side products – YouTrack is one of them.

• It is written in Java which means that Xodus will run on any platform that is able to run a Java virtual machine.
• Xodus transactions have a full set of properties that guarantee reliability: atomicity, consistency, isolation & durability. Therefore Xodus is a general-purpose database that can be used in traditional database applications having high requirements for consistency and isolation.
• On the other hand, Xodus is schema-less which makes it different from traditional database applications that require a schema. Xodus is agile to avoid migrations, schema refactorings, etc. That makes developers lives much easier when applications are required to be compatible with different versions of the database.
• Embedded database runs inside your application. Main features of embedded database are zero deployment and zero administration. It requires no dedicated server to store and access data. Applications that use Xodus have no overhead on establishing connections with database server, SQL parsing, etc.

Snapshot Isolation

Xodus supports the only isolation level, snapshot isolation. It doesn't allow dirty reads, read-committed, repeatable-read or serializable isolation. In a transaction, snapshot isolation guarantees that all reads will see a consistent snapshot of the whole database.

Snapshot isolation follows from the log-structured design of Xodus. In log-structured databases, all changes are written sequentially to a log. In Xodus, this log is an infinite sequence of .xd files. Any data stored in the log will never be modified, any change is appended to the log creating new version of the data. Any committed transaction creates new snapshot (version) of the database, and any new transaction created just after commit holds (references) this snapshot. Thus, any Xodus database can be represented as a persistent functional data structure which naturally provides lock-free multi-version concurrency control (MVCC).

Garbage Collector

Due to append-only modifications, any modified data record becomes outdated, and it will no longer be used. Thus such records are just garbage, and the database should compact itself in order to keep its suitable physical size. In Xodus, you don't need to worry about that, since it collects garbage in background. In most cases, GC works seamlessly with default settings in a single background thread. GC tries to balance between the need to maintain the database size and the need to affect user transactions as little as possible. In particular cases, GC can be tuned with a set of additional properties.

Performance

Main JetBrains' YouTrack instance contains issues database for more than 10 years. Total number of issues is near one million, physical database size exceeds 80Gb. YouTrack runs on a moderate 8-CPU server with Java heap 20Gb. Xodus provides outstanding performance due to quite compact data storing, lock-free reads and lock-free optimistic writes, intelligent lock-free caching. Xodus is highly concurrent database since it has zero contention of read operations even if there are parallel write operations.

Getting started

There are three essentially different ways to deal with data, which give three different kinds of API or API layers:

1. Environment is a transactional key-value storage.
2. Entity Store describes data model as a set of typed entities with named properties (attributes) and named entity links (relations).
3. Virtual File System (VFS) transactionally deals with files and streams.

Before you start coding, chose API layer most suitable for your project needs. The choice will determine the set of artifacts that your project will depend on. Whichever API you chose, you have to create an instance of Environment. Entity Store and VFS both work on top of Environment.

Consider the simplest sample using the Environments layer. Start by creating an instance of Environment:

final Environment env = Environments.newInstance("/Users/me/.myAppData");

All the Environment data will be physically stored in the /Users/me/.myAppData directory. Create a named store that will contain your data:

final Store store = env.computeInTransaction(new TransactionalComputable<Store>() {
    @Override
    public Store compute(@NotNull final Transaction txn) {
        return env.openStore("MyStore", StoreConfig.WITHOUT_DUPLICATES, txn);
    }
});

Here a transactional closure is used as the simplest way to manage transactions and updates within transaction. Once you get Store object, you can put values by keys to it and get values by keys from it. On the Environment layer, all data is binary and untyped, and it is represented by the ByteIterable instances. ByteIterable is a kind of byte array or Iterable<Byte>. Prepare data and proceed with a closure to put it into the store:

final ByteIterable key = StringBinding.stringToEntry("myKey");
final ByteIterable value = StringBinding.stringToEntry("myValue");

env.executeInTransaction(new TransactionalExecutable() {
    @Override
    public void execute(@NotNull final Transaction txn) {
        store.put(txn, key, value);
    }
});

You can notice that here we used TransactionalExecutable closure instead of TransactionalComputable one. Unlike TransactionalComputable, TransactionalExecutable doesn't allow to return a value. Then you can get value by key:

env.computeInReadonlyTransaction(new TransactionalComputable<ByteIterable>() {
    @Override
    public ByteIterable compute(@NotNull final Transaction txn) {
        return store.get(txn, key);
    }
});

After you stopped using Environment invoke env.close().

Read more about Environments, Entity Stores and Virtual File Systems. See how to manage dependencies of your project.