README.md

Parla - AI-driven Information Retrieval System

Parla is an AI-driven prototype designed to streamline the retrieval of information from the vast amount of documents managed by public administrations. It leverages artificial intelligence to simplify access to over 11,000 public documents, including answers to parliamentary inquiries and main committee procedures.

Why did we develop Parla?

The initiative for Parla emerged from the need to improve efficiency in public administration's handling of information. Traditional methods involve cumbersome manual searches through scattered and non-standardized data sources. Parla addresses this by providing a centralized, AI-supported platform for rapid and reliable information retrieval, benefiting not just the administration but also the public, companies, and various stakeholders.

How does Parla work?

Parla operates by scanning through a wide array of public documents available on the parliamentary documentation system, PARDOK. Users can query Parla, which then processes these inquiries using its AI to generate responses, ensuring each answer is accompanied by references to the source documents for transparency. This system not only aids in day-to-day administrative tasks but also supports broader access to governmental information.

Despite its potential, Parla remains a work in progress, with ongoing efforts to mitigate AI inaccuracies and expand its document base, mindful of data security and privacy constraints. As an experiment in AI application within public administration, Parla exemplifies the need for well-structured data and highlights the path toward more sophisticated, secure AI implementations in government processes.

Further information and contact

About the Parla project: https://citylab-berlin.org/en/projects/parla/

Blog article about Parla: https://citylab-berlin.org/en/blog/parla-intelligent-knowledge-management-for-administrative-documents/

Technical Deep Dive (german): https://citylab-berlin.org/de/blog/parla-technische-entwicklung-des-neuen-ki-tools/

For further information contact us at CityLAB Berlin: [email protected]

Parla (frontend)

This is the frontend for the project Parla. Currently we explore if we can make the parliamentary documentation provided by the Berlin House of Representatives as open data https://www.parlament-berlin.de/dokumente/open-data more accessible by embedding all the data and do search it using vector similarity search. The project is heavily based on this example from the supabase community. Built with Next.js deployed on vercel.com.

How it works with retrieval-augmented generation (RAG)

sequenceDiagram
    participant User as User
    participant Browser as Browser
    participant OurAPI as Our API
    participant OpenAI as OpenAI API
    participant Supabase as Supabase

    User ->> Browser: User writes question in frontend
    Browser ->> OurAPI: Question is sent to our API
    OurAPI ->> OpenAI: Our API moderates the question using OpenAI API
    OpenAI -->> OurAPI: OpenAI API returns moderated question
    OurAPI ->> OpenAI: Our API turns question to embedding through OpenAI API
    OpenAI -->> OurAPI: OpenAI API returns embedding
    OurAPI ->> Supabase: Our API compares question embedding to embeddings stored in database in Supabase
    Supabase -->> OurAPI: Top n most similar questions are retrieved from the database in Supabase
    OurAPI ->> OpenAI: Text from n most similar questions is used to augment a prompt and send to OpenAI API
    OpenAI -->> OurAPI: OpenAI GPT generates response
    OurAPI ->> Browser: Our API sends response back to user in frontend
    Browser -->> User: User sees response in browser

Loading

Prerequisites

• vercel.com account
• supabase.com account
• running instance of the related API and database https://github.com/technologiestiftung/parla-api
• Populated database. Using these tools https://github.com/technologiestiftung/parla-document-processor

Needed Environment Variables

NEXT_PUBLIC_PARLA_API_URL=https://domain-of-your-api-server.dev

Optional Environment Variables

To enable Matomo tracking, set the following environment variables:

NEXT_PUBLIC_MATOMO_URL=
NEXT_PUBLIC_MATOMO_SITE_ID=

Installation

npm ci

Deploy

Assuming you have a vercel.com account and you are logged in.

# does the first deployment and project creation
npx vercel
# add your env variables (interactive)
npx vercel env add NEXT_PUBLIC_PARLA_API_URL
# deploy again for production
npx vercel --prod

Development

First, run the development server:

npm run dev

Open http://localhost:3000 with your browser to see the result.

You can start editing the page by modifying src/app/page.tsx. The page auto-updates as you edit the file.

Tests

npm t

Contributing

Before you create a pull request, write an issue so we can discuss your changes.

Contributors

Thanks goes to these wonderful people (emoji key):

Fabian Morón Zirfas 💻 🚇 🎨

Ingo Hinterding 👀 🖋 🤔

Raphael.A 💻 👀 🐛

Lucas Vogel 👀 💻 🐛

Jonas Jaszkowic 💻 🐛 👀

This project follows the all-contributors specification. Contributions of any kind welcome!

Credits

Made by

A project by

Supported by

Related Projects

• https://github.com/technologiestiftung/parla-api
• https://github.com/technologiestiftung/parla-document-processor