Home

Introduction to JetBrains Xodus

Welcome to JetBrains Xodus. These wiki pages provide a brief introduction to Xodus concepts and features. Xodus stands for exodus, whatever that means, and is pronounced as exodus. Xodus is licensed under the Apache License, Version 2.0.

Overview

Xodus is transactional schemaless embedded pure-java high performant database. It is successfully used in several JetBrains server-side products, one of them is YouTrack.

• It is written in Java and only Java. So Xodus is postable as Java is, and any application written in Java that uses Xodus is portable to any platform which is able to run Java virtual machine.
• Xodus transactions have a full set of properties that guarantee that they are processed reliably: atomicity, consistency, isolation & durability. Therefore Xodus is a general-purpose database which can be used in traditional database applications having high requirements to consistency and isolation.
• On the other hand Xodus is schemaless, and that makes it disparate to traditional database applications which require database schema. Xodus is enough flexible to avoid migrations, schema refactorings, etc. That makes developer's life much easier when applications should be compatible with different versions of database.
• Embedded database exists within your application. Main features of embedded database are zero deployment and zero administration. It requires no dedicated server to store and access data (like SQL server) which is necessary to deploy and administer. Applications that use Xodus have no overhead on establishing connections with database server, SQL parsing, etc.

Snapshot Isolation

Xodus supports the only isolation level, snapshot isolation. It doesn't allow dirty reads, read-committed, repetable-read or serializable isolation. In a transaction, snapshot isolation guarantees that all reads will see a consistent snapshot of the whole database.

Snapshot isolation follows from the log-structured design of Xodus. In log-structured databases, all changes are written sequentially in a log. In Xodus, this log is an infinite sequence of .xd files. Any data stored in the log will never be modified. So any change is appended to the log creating new version of the data. Any committed transaction creates new snapshot (version) of the database, and any new transaction created just after commit holds (references) this snapshot. Thus, any Xodus database can be represented as a persistent functional data structure which naturally provides lock-free multi-version concurrency control (MVCC).

Performance

Main JetBrains' YouTrack instance contains issues database for more than 10 years. Total number of issues is near one million, physical database size exceeds 80Gb. YouTrack runs on a moderate 8-CPU server with Java heap 20Gb. Xodus provides outstading performance due to quite compact data storing, lock-free reads and lock-free optimistic writes, intelligent lock-free caching. Xodus is highly concurrent database since it has zero contention of read operations even if there are parallel write operations.

Kinds of API

There are three essencially different ways to deal with data, which give three different kinds of API.

1. Environments represent a transactional key-value storage placed on a physical storage.
2. Entity Stores describe data model as a kit of named entities with named properties and named entity relations.
3. Virtual File Systems transactionally deal with files and streams.

Getting started

Before you start coding, chose API layer most suitable for your application needs. The choice will determine the set of artifacts which your application will depend on. Whichever API you chose, you have to create an instance of Environment:

final Environment env = Environments.newInstance("/Users/me/.myAppData");

All the Environment data will be phisycally stored in the /Users/me/.myAppData directory. Then you are to create a named store that will contain your data:

final Store store = env.computeInTransaction(new TransactionalComputable<Store>() {
    @Override
    public Store compute(@NotNull final Transaction txn) {
        return env.openStore("MyStore", StoreConfig.WITHOUT_DUPLICATES, txn);
    }
});

Here a transactional closure is used as the simplest way to manage transactions and updates within transaction. Once you get Store object you can put/get data into/from it. On the Environment layer, all data is binary and untyped, and it is represented by the ByteIterable instances. ByteIterable is a kind of byte array or Iterable<Byte>. Prepare data and proceed with a closure to put it into the store:

final ByteIterable key = StringBinding.stringToEntry("myKey");
final ByteIterable value = StringBinding.stringToEntry("myValue");

env.executeInTransaction(new TransactionalExecutable() {
    @Override
    public void execute(@NotNull final Transaction txn) {
        store.put(txn, key, value);
    }
});

You may notice that here we used TransactionalExecutable closure instead of TransactionalComputable one. The only difference is that TransactionalExecutable doesn't allow to return a value whereas TransactionalComputable does. Then get data like this:

env.computeInReadonlyTransaction(new TransactionalComputable<ByteIterable>() {
    @Override
    public ByteIterable compute(@NotNull final Transaction txn) {
        return store.get(txn, key);
    }
});

Using Environments, you can manage transactions in a more sophisticated manner and implement more complex data interoperation and control flow. After you stopped using Environment invoke env.close().

Managing Dependencies

Xodus artifacts are published to Maven Central repository. Currently, only the "1.0-SNAPSHOT" version is available. To use snapshot versions, you have to specify url of the repository for snapshots. Suppose, you are using Gradle, then it is enough to define repositories in your project as follows:

repositories {
    mavenCentral()  // for release artifacts
    maven {         // for snapshot artifacts
        url 'https://oss.sonatype.org/content/groups/public'
    }
}

To build the sample above, you just have to define dependency on the xodus-environment artifacts:

dependencies {
    compile group: 'org.jetbrains.xodus', name: 'xodus-environment', version: '1.0-SNAPSHOT'
}

A shorter definition is also ok:

dependencies {
    compile 'org.jetbrains.xodus:xodus-environment:1.0-SNAPSHOT'
}

To work with Entity Stores, make you project dependent on the xodus-entity-store artifacts:

dependencies {
    compile 'org.jetbrains.xodus:xodus-entity-store:1.0-SNAPSHOT'
}

If your project essentially consists of API and implementation parts (projects, midules, etc.) you probably should define different dependecies for them. If you use Environments or Entity Stores, it is enough to make your API part dependent on the xodus-openAPI artifacts only:

dependencies {
    compile 'org.jetbrains.xodus:xodus-openAPI:1.0-SNAPSHOT'
}

Xodus openAPI contains definitions of interfaces and abstract classes sufficient to work with these two layers. As for Virtual File System, its interface is quite simple and self-contained and can be described in terms of java.io API, whereas implementation should depend on the xodus-vfs artifacts:

dependencies {
    compile 'org.jetbrains.xodus:xodus-vfs:1.0-SNAPSHOT'
}