Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/add docker image #384

Open
wants to merge 42 commits into
base: dev
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 29 commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
481fd55
qol: add attachments to cli interface
RLKRo Feb 11, 2024
973abaa
qol: make keywords upper case
RLKRo Feb 11, 2024
98692f9
qol: accept list of attachments as `Message.attachment`
RLKRo Feb 11, 2024
a9bbbb9
qol: make choice function accept responses as args
RLKRo Feb 11, 2024
d25fd30
qol: import script objects inside `dff/__init__.py`
RLKRo Feb 11, 2024
e8fc8f6
json import draft
RLKRo Feb 12, 2024
dfd61b5
add json import examples
RLKRo Feb 12, 2024
66b3bb0
add submodule aliases
RLKRo Feb 20, 2024
9637110
update resolve_target_object to use submodule aliases
RLKRo Feb 20, 2024
c4f04ba
add function to retrieve dff object mapping
RLKRo Feb 20, 2024
cfda4f7
small fixes
RLKRo Feb 22, 2024
0e5f72a
capitalize messages
RLKRo Mar 20, 2024
56755b9
improve import system for custom objects
RLKRo Mar 20, 2024
4b4cf69
make custom_dir standard way to address custom dir
RLKRo Mar 22, 2024
ce18579
prepare parser for dev release
RLKRo May 27, 2024
702ddc8
Merge branch 'master' into 'chore/slots2parser'
Ramimashkouk Jul 18, 2024
0c53232
Merge branch 'master' into chore/slots2parser
Ramimashkouk Jul 22, 2024
d787906
feat: Integrate slots parsing
Ramimashkouk Jul 22, 2024
8a4781d
chore: Add pre-transitions to script example
Ramimashkouk Jul 23, 2024
486a432
refactor: Use a consistent approach with slots
Ramimashkouk Aug 15, 2024
12c5913
feat: Add dockerfile image
Ramimashkouk Aug 15, 2024
cb0348d
refactor: Separate dockerfiles each in dir
Ramimashkouk Aug 22, 2024
8a49408
chore: Add chatsky to compose.yml
Ramimashkouk Aug 22, 2024
4f1c35f
chore: Add command to compose
Ramimashkouk Aug 28, 2024
f7faf1e
chore: Copy default pipeline to dockerfile
Ramimashkouk Aug 28, 2024
748aaec
feat: Add HTTPMessengerInterface
Ramimashkouk Aug 28, 2024
85f55ff
ci: Add workflow to build & publish docker image
Ramimashkouk Aug 28, 2024
3b8391b
fix: Make image workflow work
Ramimashkouk Sep 5, 2024
710d361
Merge branch 'feat/add-http-interface' into feat/add-docker-image
Ramimashkouk Sep 5, 2024
84a6630
chore: Publish docker image to ghrc
Ramimashkouk Sep 19, 2024
3e9e9d8
fix: Update dockerfile to find the built wheel
Ramimashkouk Sep 19, 2024
6af361b
Merge branch 'dev' into feat/add-docker-image
Ramimashkouk Sep 19, 2024
4a406c0
chore: Continue merge
Ramimashkouk Sep 19, 2024
7d15ab1
fix: Install dependencies in dokcerfile
Ramimashkouk Sep 19, 2024
6d17536
push poetry lock
Ramimashkouk Sep 19, 2024
a20de2c
chore: lock poetry
Ramimashkouk Nov 14, 2024
a0394eb
Merge branch 'dev' into feat/add-docker-image
Ramimashkouk Nov 14, 2024
d4296f0
fix: Import Message properly
Ramimashkouk Nov 14, 2024
029da89
chore: Get http_interface port from .env
Ramimashkouk Nov 18, 2024
51bc913
chore: Get port
Ramimashkouk Nov 18, 2024
584d06b
feat: Add http_interace health check
Ramimashkouk Nov 20, 2024
7e8f261
fix: Provide ctx_id for HttpInterface
Ramimashkouk Dec 2, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 59 additions & 0 deletions .github/workflows/build_and_publish_docker_image.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
name: build_and_publish_docker_image

on:
release:
types:
- published
workflow_dispatch:


concurrency:
group: ${{ github.workflow }}-${{ github.ref }}

jobs:
build_and_publish_image:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write

steps:
- name: Checkout
uses: actions/checkout@v4

- name: Log in to Docker Hub
uses: docker/login-action@v3
with:
registry: docker.io
username: ${{ vars.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Extract Docker metadata
uses: docker/metadata-action@v5
id: meta
with:
images: ${{ github.repository }}
flavor: |
latest=auto
tags: |
type=raw,value=latest,enable={{is_default_branch}}
type=semver,pattern={{version}}
Ramimashkouk marked this conversation as resolved.
Show resolved Hide resolved

- name: Test Dockerfile
run: docker build -t chatsky_image -f chatsky/utils/docker/chatsky/dockerfile_chatsky .

- name: Build and upload image
uses: docker/build-push-action@v5
with:
context: .
file: chatsky/utils/docker/chatsky/dockerfile_chatsky
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}

- name: Update repo description
uses: peter-evans/dockerhub-description@e98e4d1628a5f3be2be7c231e50981aee98723ae # v4.0.0
with:
username: ${{ vars.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
repository: ${{ github.repository }}
4 changes: 2 additions & 2 deletions .github/workflows/update_dashboard.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ jobs:
- name: Build and upload image
uses: docker/build-push-action@v5
with:
context: chatsky/utils/docker
file: chatsky/utils/docker/dockerfile_stats
context: chatsky/utils/docker/stats
file: chatsky/utils/docker/stats/dockerfile_stats
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
4 changes: 4 additions & 0 deletions chatsky/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,4 +13,8 @@
from chatsky.pipeline import Pipeline
from chatsky.script import Context, Script

from chatsky.msg import *

Script.model_rebuild()

import chatsky.__rebuild_pydantic_models__
1 change: 1 addition & 0 deletions chatsky/cnd.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
from chatsky.script.conditions import *
1 change: 1 addition & 0 deletions chatsky/lbl.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
from chatsky.script.labels import *
43 changes: 42 additions & 1 deletion chatsky/messengers/common/interface.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,10 @@
import abc
import asyncio
import logging
import uuid
from pathlib import Path
from tempfile import gettempdir
from typing import Optional, Any, List, Tuple, Hashable, TYPE_CHECKING, Type
from typing import Optional, Any, List, Tuple, TextIO, Hashable, TYPE_CHECKING, Type

if TYPE_CHECKING:
from chatsky.script import Context, Message
Expand Down Expand Up @@ -192,3 +193,43 @@ def on_request(
This method has the same signature as :py:class:`~chatsky.pipeline.types.PipelineRunnerFunction`.
"""
return asyncio.run(self.on_request_async(request, ctx_id, update_ctx_misc))


class CLIMessengerInterface(PollingMessengerInterface):
"""
Command line message interface is the default message interface, communicating with user via `STDIN/STDOUT`.
This message interface can maintain dialog with one user at a time only.
"""

def __init__(
self,
intro: Optional[str] = None,
prompt_request: str = "request: ",
prompt_response: str = "response: ",
out_descriptor: Optional[TextIO] = None,
):
super().__init__()
self._ctx_id: Optional[Hashable] = None
self._intro: Optional[str] = intro
self._prompt_request: str = prompt_request
self._prompt_response: str = prompt_response
self._descriptor: Optional[TextIO] = out_descriptor

def _request(self) -> List[Tuple[Message, Any]]:
return [(Message(input(self._prompt_request)), self._ctx_id)]

def _respond(self, responses: List[Context]):
print(f"{self._prompt_response}{responses[0].last_response.text}; attachments: {responses[0].last_response.attachments}", file=self._descriptor)

async def connect(self, pipeline_runner: PipelineRunnerFunction, **kwargs):
"""
The CLIProvider generates new dialog id used to user identification on each `connect` call.

:param pipeline_runner: A function that should process user request and return context;
usually it's a :py:meth:`~chatsky.pipeline.pipeline.pipeline.Pipeline._run_pipeline` function.
:param \\**kwargs: argument, added for compatibility with super class, it shouldn't be used normally.
"""
self._ctx_id = uuid.uuid4()
if self._intro is not None:
print(self._intro)
await super().connect(pipeline_runner, **kwargs)
30 changes: 30 additions & 0 deletions chatsky/messengers/http_interface.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
import uvicorn
from fastapi import FastAPI
from pydantic import BaseModel

from chatsky.messengers.common import MessengerInterface
from chatsky.script import Message


class HTTPMessengerInterface(MessengerInterface):
async def connect(self, pipeline_runner):
app = FastAPI()

class Output(BaseModel):
user_id: str
response: Message

@app.post("/chat", response_model=Output)
async def respond(
user_id: str,
user_message: str, # TODO: change type to Message after core rework is done
):
message = Message(text=user_message)
context = await pipeline_runner(message)
return {"user_id": user_id, "response": context.last_response}

uvicorn.run(
app,
host="0.0.0.0",
port=8020,
)
14 changes: 14 additions & 0 deletions chatsky/msg.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
from chatsky.script.core.message import (
Location,
Attachment,
Audio,
Video,
Image,
Document,
# Attachments,
# Link,
# Button,
# Keyboard,
Message,
# MultiMessage
)
42 changes: 41 additions & 1 deletion chatsky/pipeline/pipeline/pipeline.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
import asyncio
import logging
from typing import Union, List, Dict, Optional, Hashable, Callable
from collections import defaultdict

from chatsky.context_storages import DBContextStorage
from chatsky.script import Script, Context, ActorStage
Expand All @@ -25,7 +26,7 @@

from chatsky.messengers.console import CLIMessengerInterface
from chatsky.messengers.common import MessengerInterface
from chatsky.slots.slots import GroupSlot
from chatsky.slots.slots import GroupSlot, RegexpSlot, FunctionSlot
from ..service.group import ServiceGroup
from ..types import (
ServiceBuilder,
Expand All @@ -37,6 +38,7 @@
)
from .utils import finalize_service_group, pretty_format_component_info_dict
from chatsky.pipeline.pipeline.actor import Actor
from chatsky.pipeline.pipeline.script_parsing import JSONImporter, Path

logger = logging.getLogger(__name__)

Expand Down Expand Up @@ -271,6 +273,44 @@ def from_script(
components=[*pre_services, ACTOR, *post_services],
)

@classmethod
def from_file(
cls,
file: Union[str, Path],
# validation_stage: Optional[bool] = None,
condition_handler: Optional[Callable] = None,
# verbose: bool = True,
parallelize_processing: bool = False,
handlers: Optional[Dict[ActorStage, List[Callable]]] = None,
context_storage: Optional[Union[DBContextStorage, Dict]] = None,
messenger_interface: Optional[MessengerInterface] = None,
pre_services: Optional[List[Union[ServiceBuilder, ServiceGroupBuilder]]] = None,
post_services: Optional[List[Union[ServiceBuilder, ServiceGroupBuilder]]] = None,
):
pre_services = [] if pre_services is None else pre_services
post_services = [] if post_services is None else post_services
script = JSONImporter(file).import_script()

def to_tuple(i):
if isinstance(i, list):
return tuple(i)
return i

config = JSONImporter(file).replace_script_objects(script["CONFIG"])
params = {param: to_tuple(script["CONFIG"].get(param)) for param in config.keys() if param != "custom_dir"}
del script["CONFIG"] # todo: add support for CONFIG

return cls(
script=script,
**params,
condition_handler=condition_handler,
parallelize_processing=parallelize_processing,
handlers=handlers,
messenger_interface=messenger_interface,
context_storage=context_storage,
components=[*pre_services, ACTOR, *post_services],
)

def set_actor(
self,
script: Union[Script, Dict],
Expand Down
Loading