v3.3.0

With the release of v3.3.0, R2R offers a completely RESTful API that covers everything you need for production RAG applications. The biggest change is our Git-like knowledge graph architecture, but we've also unified all the core objects you need to build real applications.

📚 Complete API Coverage:

1️⃣ Content & Knowledge
• Documents: Upload files, manage content, and track extraction status
• Chunks: Access and search vectorized text segments
• Graphs: Git-like knowledge graphs with:
↳ Entities & Relationships
↳ Automatic community detection
↳ Independent graphs per collection

2️⃣ Infrastructure
• Indices: Manage vector indices for search optimization
• Collections: Organize documents and share access
• Users: Built-in auth and permission management
• Conversations: Track chat history and manage branches

3️⃣ Retrieval & Generation
• RAG: Configurable retrieval pipeline with hybrid search
• Agents: Conversational interfaces with knowledge graph integration
• Search: Vector, keyword, and knowledge graph search

💻 Quick Example:

from r2r import R2RClient
client = R2RClient("http://localhost:7272")

# Document level extraction
client.documents.extract(document_id)

# Collection level graph management
client.graphs.pull(collection_id)

# Advanced RAG with everything enabled
response = client.retrieval.rag(
    "Your question here",
    search_settings={
        "use_hybrid_search": True,
        "graph_settings": {"enabled": True}
    }
)

All these components work together seamlessly - just configure what you need and R2R handles the rest. Perfect for teams building serious RAG applications.

🔗 Check the API: https://r2r-docs.sciphi.ai/api-and-sdks/introduction

We'd love feedback from folks building in production!

R2R V3.2.30

Among a number of bug fixes, scaling, and performance improvements, there are a few notable features and considerations.

Migrations
With this release, we introduce a number of changes that require migrations. To run migrations, you can update to the latest version of R2R with r2r update, upon serving R2R (e.g. r2r serve --docker) the schema will be checked, and you will be prompted to upgrade your database with r2r db upgrade.

Advanced GraphRAG
We're rolling out prompt tuning and contextual embeddings, which both offer significant improvements.

R2R V3.2.0

Key Features and Improvements:

• Introduced light and full installation modes to facilitate faster setup / prototyping
• Removed Neo4j dependency - GraphRAG is now done entirely in Postgres, we will be writing a blog post around our motivations for this change.
• Commitment to stable releases and migration - R2R is moving towards serious use cases and is committed to providing working migration scripts with each major change following 3.2.0. This will be the last painful migration.

Optimizations:

• Improved knowledge graph configuration and construction process
• Added new prompt endpoints and management features
• Renamed groups to collections for improved clarity
• Integrated Supabase authentication
• Simplified hybrid search functionality
• Improved error messaging and handling
• Enhanced Docker configuration and build process

Documentation:

• Updated and fixed documentation, including broken links in Mintlify
• Improved configuration documentation for various components

Bug Fixes:

• Resolved issues with Docker commands and configurations
• Fixed parsing config docs
• Addressed problems with app settings and prompt management
• Corrected various minor bugs and typos

Developer Experience:

• Updated dependencies and package versions
• Improved typing and pagination
• Enhanced JS SDK with auto-refresh capability

Full Changelog: V3.1.0...v3.2.0

R2R V3.1.0

New Features
Orchestration with Hatchet
Default ingestion provider set to Unstructured
Improved knowledge graph construction process

Bug Fixes
Minor bug fixes around config logic and other.

Full Changelog: v0.3.0...V3.1.0

R2R V3

We are very excited to introduce R2R V3. We have been working very intensely as a team over this past month to comprehensively address all the feedback we received after recent explosive growth in R2R usage.

In addition to modifying 420 files and tens of thousands of LOC in the core repository, we have refactored the documentation here - https://r2r-docs.sciphi.ai/introduction.

NOTE - This release has many breaking changes w.r.t to R2R v2 and so we are working on a bespoke migration script to help those already in production.

Highlights

• We've added full GraphRAG support as per the Microsoft GraphRAG paper. This implementation is built alongside our integration with Neo4j inside R2R so that you can build a non-ephemeral user-facing system. The system supports dynamic tuning of a number of query parameters so that you can optimize it for your given application, you can read more here!
• Added full group permissions over vector search, meaning you can create unique dynamic groupings of documents which are permissioned at the user level. Support is included for granular actions like create_group, update_group, add_user_to_group, add_document_to_group, add_user_to_group, and more. Get started with the cookbook here.
• We've doubled down on hybrid search by improving the full-text search component and its integration with vector search. The new system employs advanced techniques including text stemming, synonym expansion, phrase matching, and partial matching, all combined with a sophisticated ranking algorithm. Additionally, we've implemented runtime configurability through the VectorSearchSettings object. You can read more here.
• We've implemented a powerful RAG Agent that enables developers to add custom tools and behaviors tailored to specific use cases. This allows R2R developers to trivially go from RAG to Agentic RAG, allowing 1-click construction of full chat intelligence. You can read more here.
• Bug fixes and new focus on production R2R has matured significantly over the last quarter and has now answered over 1 million questions in production. As the system matures we are focusing more on supporting developers in production and carrying out our active development in a way that is minimally disruptive. This new release polishes a number of features that were rough / experimental in V2, while removing some features that were not seeing enough use, such as continuous evaluation.
• Slim CLI and Python SDK to reduce bloat required to launch / interact with R2R through docker.

What's Changed

• Feature/merge graphrag group mgmt by @emrgnt-cmplxty in #876
• Clean up API by @NolanTrem in #878
• Feature/merge dev owen changes by @emrgnt-cmplxty in #880
• Shreyas/graphrag test by @shreyaspimpalgaonkar in #881
• Feature/fix embedding pipe by @emrgnt-cmplxty in #882
• rm kg cruft by @emrgnt-cmplxty in #884
• tweak 2 by @emrgnt-cmplxty in #885
• Feature/fix retrieval endpoint cruft by @emrgnt-cmplxty in #887
• Python SDK by @NolanTrem in #886
• Separate out SDK, add js and go sdk to monorepo by @NolanTrem in #888
• Feature/merge w nolan by @emrgnt-cmplxty in #894
• fix rag stream by @emrgnt-cmplxty in #895
• add py r2r by @emrgnt-cmplxty in #896
• Clean up error handling by @NolanTrem in #897
• fix agent by @emrgnt-cmplxty in #898
• define RAGAgentResponse by @emrgnt-cmplxty in #899
• Shreyas/unstructured by @shreyaspimpalgaonkar in #900
• Feature/cleanup client obj logic by @emrgnt-cmplxty in #901
• Shreyas/tests by @shreyaspimpalgaonkar in #889
• Refactor CLI by @NolanTrem in #903
• Rm files readded by git by @NolanTrem in #904
• Remove Execution Wrapper by @NolanTrem in #905
• Feature/fix auth revive tests rebased by @emrgnt-cmplxty in #906
• fixup toml by @emrgnt-cmplxty in #908
• Feature/fix ingestion update by @emrgnt-cmplxty in #909
• Fix CLI Tests by @NolanTrem in #912
• Shreyas/kg runtime cfg by @shreyaspimpalgaonkar in #913
• minor fix: rename kgenrichmentresponse by @shreyaspimpalgaonkar in #914
• Feature/add nltk hybrid expansion rebased by @emrgnt-cmplxty in #917
• add script by @emrgnt-cmplxty in #918
• Fix bug in document chunks by @NolanTrem in #921
• Fix bug in update files by @NolanTrem in #923
• Shreyas/unstructured by @shreyaspimpalgaonkar in #922
• Feature/repair logging by @emrgnt-cmplxty in #925
• rm double logging by @emrgnt-cmplxty in #929
• Configs by @NolanTrem in #926
• Disable SSL when installing nltk wordnet by @NolanTrem in #930
• Fix Analytics by @NolanTrem in #931
• Update OpenAI sdk calls by @NolanTrem in #933
• Feature/revive advanced rag by @emrgnt-cmplxty in #932
• sync model by @emrgnt-cmplxty in #935
• Feature/remove version from ingestion end pt by @emrgnt-cmplxty in #936
• Move docker by @NolanTrem in #938
• Fix js tests, completion scoring by @NolanTrem in #939
• Shreyas/unstructured docker image by @shreyaspimpalgaonkar in #940
• Update JS by @NolanTrem in #941
• Sync JS Models by @NolanTrem in #942
• Feature/complete group logic by @emrgnt-cmplxty in #945
• Fix Dockerbuild, Symlink Readme by @NolanTrem in #944
• Shreyas/KG Search Result model by @shreyaspimpalgaonkar in #937
• refresh should not be gated by auth by @emrgnt-cmplxty in #946
• Sync Linting by @NolanTrem in #947
• Remove email in refresh by @NolanTrem in #948
• Feature/rm print cruft rebase by @emrgnt-cmplxty in #953
• Update Action by @NolanTrem in #954
• Update Github Actions by @NolanTrem in #956
• Shreyas/kgsearchresult model by @shreyaspimpalgaonkar in #957
• Login and refresh token bug by @NolanTrem in #959
• Point to Selfhosted runner by @NolanTrem in #960
• collection docs by @emrgnt-cmplxty in #955
• Feature/merge dev to main by @emrgnt-cmplxty in #962
• fix docker builds by @emrgnt-cmplxty in #963
• Running unstructured docker + code cleanups by @shreyaspimpalgaonkar in #964
• Small bugfixes on prompts, return types by @NolanTrem in #965
• Feature/fix dev tests by @emrgnt-cmplxty in #966
• Feature/merge dev and main by @emrgnt-cmplxty in #967

New Contributors

• @tkellogg made their first contribution in #883
• @mmtmr made their first contribution in #893
• @aphedges made their first contribution in #919
• @manuelciosici made their first contribution in #961

Full Changelog: v0.2.85...v3.0.0*

v0.2.85

A few important bug fixes in 0.2.85!

What's Changed

• Fix parsing config in example config files by @hvoecking in #864
• Allow BASE_URL and TRAEFIK_DASHBOARD_PORT (added) to be passed in from environment vars by @smig23 in #862
• Add agent entry to set local ollama/llama3.1 model by @smig23 in #866

New Features

• Environment variable support for BASE_URL and TRAEFIK_DASHBOARD_PORT

Bug Fixes

• Fixed parsing config in example config files
• Added agent entry in config for local ollama/llama3.1 model

New Contributors

• @hvoecking made their first contribution in #864

We'd like to extend a warm welcome to our new contributor and thank all those who made this release possible!

Full Changelog

For a detailed list of all changes, please see: v2.0.84...v0.2.85

---

To update to this version, run r2r update from your CLI. If you encounter any issues or have any questions, please don't hesitate to open an issue on Github!

Release v0.2.84

Release v0.2.84

We're excited to announce the release of version 0.2.84, which brings new features, bug fixes, and improvements to enhance your experience with R2R.

New Features

• Migration Support: Introduced Alembic for database migration support.
• Enhanced Logging: Added CompletionRecord to logs for richer observability.
• Completion Scoring: Users can now provide feedback on the responses they receive.
• User Tracking: Added user tracking to logs for improved analytics.

Bug Fixes

• Resolved issues related to CONFIG_PATH variables.
• Fixed incorrect providers in default configs.
• Various other minor bug fixes and improvements.

Breaking Changes

• Database Migration Required: Clone the repository and r2r migrate to update your local SQLite logging database. Note: this is a beta feature, and you may need to adjust the migration script to point to your local logs.
• User ID in Logs: User IDs are now included in logs. Logging without user ID is deprecated but will be supported until v0.3.0.

Configuration Updates

• Adjustments made to r2r/examples/configs/local_llm_neo4j_kg.toml.

Other Changes

• Multiple pull requests for patches and updates.
• Temporary changes that were reverted.

Documentation Updates

• Added instructions on how to score completions with the Python SDK.

Contributors

We'd like to thank all the contributors who made this release possible, with a special shoutout to @smig23 for their first contribution!

V0.2.83

Release v0.2.83

We're excited to announce the release of version 0.2.83, which brings new features, bug fixes, and improvements to enhance your experience with R2R.

New Features

• Migration Support: Introduced Alembic for database migration support.
• Enhanced Logging: Added CompletionRecord to logs for richer observability.
• Completion Scoring: Users can now provide feedback on the responses they receive.
• User Tracking: Added user tracking to logs for improved analytics.

Bug Fixes

• Resolved issues related to CONFIG_PATH variables.
• Fixed incorrect providers in default configs.
• Various other minor bug fixes and improvements.

Breaking Changes

• Database Migration Required: Run r2r migrate to update your local SQLite logging database.
• User ID in Logs: User IDs are now included in logs. Logging without user ID is deprecated but will be supported until v0.3.0.

Configuration Updates

• Adjustments made to r2r/examples/configs/local_llm_neo4j_kg.toml.

Other Changes

• Multiple pull requests for patches and updates.
• Temporary changes that were reverted.

Documentation Updates

• Added instructions on how to score completions with the Python SDK.

Contributors

We'd like to thank all the contributors who made this release possible, with a special shoutout to @smig23 for their first contribution!

V0.2.76

Summary of Changes

This release includes several significant updates and improvements:

• Bug fixes:

  • Resolved a PostgreSQL issue (#811)
  • Addressed PDF parsing problems with multilingual Unicode characters (#785)

• Feature enhancements:

  • Added support for external LiteLLM proxy (#815)
  • Implemented server stats (#823)
  • Introduced dynamic port selection (#833)

• Infrastructure improvements:

  • Refactored configuration to use TOML and JSON (#817)
  • Added Unstructured Docker support (#824, #826)
  • Implemented additional registry (#828)

• Developer experience:

  • Extended CLI provider checks (#816)
  • Improved test runner setup (#827, #829)
  • Split build flows for optimization (#835)

• User experience:

  • Enabled automatic navigation to dashboard on launch (#819)
  • Completed renaming to "agent" for consistency (#830)

• Performance enhancements:

  • Prevented Docker timeouts during report generation (#821)
  • Implemented caching improvements (#831)

• Flexibility improvements:

  • Made parsing and chunking providers configurable (#820, #832)

New Contributors

• @igor-pechersky made their first contribution (#815)

v0.2.74

0.2.74 - 2024-07-30

Added

• Docker Dev Image for CI/CD
• No-postgres mode
• Completion yield functionality
• Entity types and relationships for Knowledge Graph search

Changed

• Refactored CLI
• Updated concurrency and various configs
• Renamed 'local_ollama' to 'local_llm'
• Separated tool stream and LLM outputs

Fixed

• Health check dependency in Dockerfile
• Backoff for stream chat
• Runtime warning about version reading
• host.docker.internal connectivity issues

Improved

• R2R image configuration
• Local LLM configurations

Removed

• Postgres dependency from Docker (optional exclusion)

Contributors

• @houjibofa2050 made their first contribution