Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/vk interface #342

Draft
wants to merge 157 commits into
base: dev
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
157 commits
Select commit Hold shift + click to select a range
ae091ab
new telegram message proposal
pseusys Jan 22, 2024
b1c4da9
only required attachments remain
pseusys Jan 22, 2024
86c15c2
keyboard implemented
pseusys Jan 23, 2024
a0a859c
tests reworked
pseusys Jan 29, 2024
89fa2c7
telegram examples debugged
pseusys Jan 31, 2024
3745cff
optional attachment processing added
pseusys Jan 31, 2024
18ddd2c
Started implementing VK interface
NotBioWaste905 Feb 1, 2024
26dc2d6
download all removed, special downloading method created
pseusys Feb 4, 2024
57f418e
Attempts to implement asyncronous vk API usage
NotBioWaste905 Feb 5, 2024
bf1a81a
Added attachment management, started implementing keyboard
NotBioWaste905 Feb 5, 2024
7e65068
callback query added as an attachment
pseusys Feb 7, 2024
efece76
callback query conditions added
pseusys Feb 7, 2024
c947b4e
remove tg tests; move message tests to core tests
RLKRo Feb 22, 2024
25b8ac9
buttons pulled out of attachments list
pseusys Feb 28, 2024
bdda83e
Added multiple attachments
NotBioWaste905 Feb 28, 2024
a6c1172
merge
RLKRo Mar 7, 2024
b67b497
some errors fixed
pseusys Mar 12, 2024
5ad4715
Added basic attachments
NotBioWaste Mar 13, 2024
e4e6951
Fixed link
NotBioWaste Mar 13, 2024
29b11f0
Add/First version of tutorial
NotBioWaste905 Mar 14, 2024
95926d6
Fix/deleted sending image
NotBioWaste905 Mar 14, 2024
ede04a9
Delete bot_test.py
NotBioWaste905 Mar 18, 2024
68b4a37
review issues fixed
pseusys Mar 18, 2024
efde7c3
Merge remote-tracking branch 'origin/dev' into feat/telegram_interfac…
pseusys Mar 18, 2024
f9a9d79
Merge branch 'dev' into feat/telegram_interface_upgrade
pseusys Mar 18, 2024
6f8de0b
lock file updated
pseusys Mar 18, 2024
771370d
typing fixed
pseusys Mar 18, 2024
1c85052
circular import resolved
pseusys Mar 18, 2024
0cebb97
one more attempt to fix circular import
pseusys Mar 18, 2024
a5f0e02
populate attachment method made implemented
pseusys Mar 19, 2024
01947dd
CLI interface separated
pseusys Mar 19, 2024
5e35935
data message equation fixed
pseusys Mar 19, 2024
513f77a
lint applied
pseusys Mar 19, 2024
48bb29f
move callback query to script.conditions
RLKRo Mar 20, 2024
e2ea371
condition signature fix
pseusys Mar 21, 2024
ea66537
messenger interface parameter added to `get_bytes` method
pseusys Mar 21, 2024
21feb0c
Add/attachment_tutorial
NotBioWaste905 Mar 21, 2024
72e69fb
Fix/deleted match case
NotBioWaste905 Mar 21, 2024
db1de9a
async def removed from `extract_message_from_telegram` function
pseusys Mar 21, 2024
19d00f3
docs fixed
pseusys Mar 21, 2024
11cc1a3
condition import added
pseusys Mar 21, 2024
66f82b1
two attachment tutorials fixed
pseusys Mar 22, 2024
2773c9d
attachment tuples changed to sets
pseusys Mar 22, 2024
80fceb5
lint applied
pseusys Mar 22, 2024
a5fbe60
docs links fixed
pseusys Mar 22, 2024
6a21d35
Fixed missing document names when uploading
NotBioWaste905 Mar 22, 2024
8e970c9
Merge branch 'dev' into feat/telegram_interface_upgrade
pseusys Mar 27, 2024
3f815e7
Merge branch 'feat/telegram_interface_upgrade' of https://github.com/…
pseusys Mar 27, 2024
f8cecdf
telegram formatted
pseusys Mar 27, 2024
95019ad
lock update
pseusys Mar 27, 2024
8c90763
tuples replaced with sets everywhere
pseusys Mar 27, 2024
de1982f
Switched debug to logging, cleared unused code and imports, changed t…
NotBioWaste Mar 28, 2024
30db35a
telegram-specific request attachments added
pseusys Apr 4, 2024
1432893
attachment subclasses moved to extras
pseusys Apr 9, 2024
1bfe524
extras added
pseusys Apr 10, 2024
82e7a3e
lint fixed
pseusys Apr 11, 2024
2eb8f95
feat/Implemented VK_Bot class, working on VK_dummy test class
NotBioWaste905 Apr 11, 2024
1866008
Updated VK_dummy
NotBioWaste905 Apr 12, 2024
72bb23f
Added vk_dummy_example.py
NotBioWaste905 Apr 12, 2024
334cd74
ui replaced with keyboard
pseusys Apr 16, 2024
fb2e90a
Merge branch 'feat/telegram_interface_upgrade' of https://github.com/…
pseusys Apr 16, 2024
bc406f6
"check_if_" removed
pseusys Apr 16, 2024
b08e8f3
undefined message removed
pseusys Apr 16, 2024
c79a732
unnecessary type casting removed
pseusys Apr 16, 2024
7853dbc
args and kwargs removed
pseusys Apr 16, 2024
5b9992f
default poll option voter count added
pseusys Apr 16, 2024
a3beac9
telegram import check added
pseusys Apr 16, 2024
5dd0751
interface attachments sorted
pseusys Apr 16, 2024
b951772
temp attachment dir warning added
pseusys Apr 16, 2024
fe99c4c
attachment message clarified
pseusys Apr 16, 2024
595d55d
single attachment sending proposal + sticker
pseusys Apr 18, 2024
b31ea18
request and response attachments sorted
pseusys Apr 18, 2024
084ca65
Added dummy data to the requests
NotBioWaste Apr 18, 2024
c882abf
Merge branch 'feat/telegram_interface_upgrade' of https://github.com/…
pseusys Apr 18, 2024
dd57886
attachments tutorial added
pseusys Apr 22, 2024
33c72cb
advanced tutorial added
pseusys Apr 23, 2024
3035cf1
Redone VK_Bot, updated VK_dummy
NotBioWaste Apr 24, 2024
28959f4
Updated VKDummy, redone VK_Bot as VKWrapper
NotBioWaste Apr 25, 2024
17280e3
`cached_filename` field added
pseusys Apr 26, 2024
2a1f01a
Tutorial texts altered.
pseusys Apr 26, 2024
fb58bda
happy path removed, use cache flag added
pseusys Apr 26, 2024
2f3c528
Implementing tests with VKDummy
NotBioWaste Apr 27, 2024
a2e0ca3
literals added + tests started
pseusys May 1, 2024
decbef8
Added test data
NotBioWaste May 3, 2024
53e693e
Pre-merge commit
NotBioWaste May 6, 2024
1bd75de
Merge remote-tracking branch 'origin/feat/telegram_interface_upgrade'…
NotBioWaste May 6, 2024
08a7389
Added dummy_post function to VKDummy
NotBioWaste May 6, 2024
134a23f
Delete bot_test.py
NotBioWaste905 May 6, 2024
a07b73b
Attachment links updated
pseusys May 7, 2024
87001c0
new_attachment_tests_added
pseusys May 7, 2024
aabb3f8
Merge branch 'feat/telegram_interface_upgrade' of https://github.com/…
pseusys May 7, 2024
b4dd7fc
Merge branch 'dev' into feat/telegram_interface_upgrade
pseusys May 7, 2024
f478972
Merge branch 'feat/telegram_interface_upgrade' of https://github.com/…
pseusys May 7, 2024
f1a140a
parametrize fixed for telegram testing
pseusys May 7, 2024
3b04c03
Some lint fixed
pseusys May 7, 2024
4db3d6f
file path fixed
pseusys May 7, 2024
74a0402
attachments type fixed
pseusys May 8, 2024
9bc4b3c
supported attachment sets renamed
pseusys May 8, 2024
cb28fe3
final message test fixed
pseusys May 8, 2024
d30358b
dot path used instead of slash
pseusys May 8, 2024
eaad6d2
tutorial tests fixed
pseusys May 8, 2024
c77f41f
fixed getattribute stack overflow exception
pseusys May 8, 2024
28d7f8a
true transition fixed
pseusys May 8, 2024
786b3c7
async function marked
pseusys May 8, 2024
2277af7
cache dir clear and create reversed
pseusys May 8, 2024
6e218b6
pytest warning fixed
pseusys May 8, 2024
ff88c46
Working on populate attachments
NotBioWaste May 16, 2024
f0ea096
Resolving sync issue
NotBioWaste May 16, 2024
971e331
Merge telegram_interface into vk_interface
NotBioWaste May 16, 2024
e6d1226
tutorial testing done
pseusys May 21, 2024
eb07fae
last test step added
pseusys May 24, 2024
97be6b9
attachments migrated to wiki
pseusys May 24, 2024
33a5c73
pragma removed, inheritance added
pseusys May 27, 2024
140f014
some review parts fixed
pseusys May 27, 2024
47610a3
venv ignore added back
pseusys May 27, 2024
9c435d8
Merge remote-tracking branch 'origin/feat/telegram_interface_upgrade'…
NotBioWaste May 29, 2024
5435889
Fixed attachment parsing and tutorial attachments
NotBioWaste May 29, 2024
3e113b6
Added title extra field to doc type attachments
NotBioWaste May 29, 2024
0362fc8
some review comments updated
pseusys Jun 3, 2024
ea033b4
Created MockVK, splitted saving functions, added extra fields handling
NotBioWaste Jun 5, 2024
07d887e
erview notes taken into account
pseusys Jun 5, 2024
121b092
additional `Message` serialization test introduced
pseusys Jun 5, 2024
5172f0c
extra field verification added
pseusys Jun 7, 2024
27205bb
github action fixes #1
pseusys Jun 7, 2024
0de3e2b
github action fixes N2
pseusys Jun 7, 2024
83cb8a0
github action fixes N3
pseusys Jun 7, 2024
27678f5
misc typing fixed
pseusys Jun 7, 2024
f88e454
Attachment name changed (for windows)
pseusys Jun 7, 2024
85487c3
typing.Annotated fixed
pseusys Jun 7, 2024
e634436
Naming fixed once again
pseusys Jun 7, 2024
5ca6e20
telegram import in tests fixed
pseusys Jun 7, 2024
209d975
telegram import fix (again)
pseusys Jun 7, 2024
2f67816
PathStep quotes added
pseusys Jun 7, 2024
b61c42d
aiofiles dependency fixed
pseusys Jun 8, 2024
cc5772e
typing fixed
pseusys Jun 9, 2024
ae35fb9
naming error fixed
pseusys Jun 10, 2024
a1df33f
ok, we'll just use any I guess
pseusys Jun 10, 2024
d6fbaaf
telegram skip setup correctly
pseusys Jun 10, 2024
eb74d0a
Add files via upload
NotBioWaste905 Jun 10, 2024
c9231ff
Fixes in testing
NotBioWaste Jun 13, 2024
0c9d80d
Merge branch 'feat/vk_interface' of https://github.com/deeppavlov/dia…
NotBioWaste Jun 13, 2024
e12fbb4
Test old testing version
NotBioWaste Jun 13, 2024
c091dd1
Fixed path to test data
NotBioWaste Jun 13, 2024
572e913
Merge remote-tracking branch 'origin/feat/telegram_interface_upgrade'…
NotBioWaste Jun 13, 2024
7c1a9e4
Merged woth TG + fixed fixed path to test files
NotBioWaste Jun 13, 2024
df6a99e
Quickfix
NotBioWaste Jun 13, 2024
964c9e9
Fixed path
NotBioWaste Jun 13, 2024
b74152d
Retracted pyproject edit
NotBioWaste Jun 13, 2024
a1af719
Retracted pyproject to previous version (for real)
NotBioWaste Jun 13, 2024
edd5e93
Fixed message extraction
NotBioWaste Jun 14, 2024
9469846
Added outgoing testing and fixed vk_api_call
NotBioWaste Jun 17, 2024
be7a750
Fixed missing awaits
NotBioWaste Jun 17, 2024
c801175
testing outgoing attachments
NotBioWaste Jun 17, 2024
b06fd83
Fixed attachment testing
NotBioWaste Jun 21, 2024
fb59508
Switched to the url attachments
NotBioWaste905 Jun 23, 2024
26e4eea
Deleted old attachments
NotBioWaste Jul 1, 2024
11ec311
Merge branch 'feat/vk_interface' of https://github.com/deeppavlov/dia…
NotBioWaste Jul 1, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
2 changes: 0 additions & 2 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
*.DS_Store*
*.egg-info/
dist/
venv/
build/
docs/source/apiref
docs/source/_misc
Expand All @@ -22,7 +21,6 @@ GlobalUserTableAccessor*
memory_debugging*
opening_database*
_globals.py
venv*
.vscode
.coverage
.coverage.*
Expand Down
1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,6 @@ GlobalUserTableAccessor*
memory_debugging*
opening_database*
_globals.py
venv*
.vscode
.coverage
.coverage.*
Expand Down
51 changes: 51 additions & 0 deletions bot_test.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
from dff.script import conditions as cnd
from dff.script import RESPONSE, TRANSITIONS, Message, Context
from dff.pipeline import Pipeline

from dff.messengers.vk import PollingVKInterface
from tests.messengers.vk.vk_dummy import VKDummy



script = {
"greeting_flow": {
"start_node": {
RESPONSE: Message(),
TRANSITIONS: {"node1": cnd.exact_match(Message("Hi"))}
},
"node1": {
RESPONSE: Message(
text="Hi, how are you?"
),
TRANSITIONS: {
"start_node": cnd.exact_match(Message("Nice"))
},
},
"fallback_node": {
RESPONSE: Message(text="Sorry, I don't understand."),
TRANSITIONS: {
"start_node": cnd.true(),
}
}
}
}


# KEY = "vk1.a.BvmniZCwChZfKGPWTUtHiFOC1GOQ7M7ePur6y9cs9YpKuKXLKQLDGZDy5fxhiJAFWXNLAKazSRjFqpw5WByX1x-KiEU3HcIb4QHSG_utQMjnqQ4UOYXcT7pn-8nUeo4hnf-nH4BIB3fdwq0WhZLTcnlLAok2sxd_yR2q8LVzElc-7rhZAuyNDcLzOxFx9Z9P3xsg5EEhDFBcWRvGk_1R1Q"
# GROUP_ID = "224377718"


vk_dummy = VKDummy()
vk_interface = PollingVKInterface("KEY", "GROUP_ID")
vk_interface.bot = vk_dummy

pipeline = Pipeline.from_script(
script=script,
start_label=("greeting_flow", "start_node"),
fallback_label=("greeting_flow", "fallback_node"),
messenger_interface=vk_interface,
)
pipeline.run()
# vk_interface._respond(ctx)
print(vk_dummy.responses)

2 changes: 1 addition & 1 deletion dff/messengers/common/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# -*- coding: utf-8 -*-

from .interface import MessengerInterface, PollingMessengerInterface, CallbackMessengerInterface, CLIMessengerInterface
from .interface import MessengerInterface, PollingMessengerInterface, CallbackMessengerInterface
from .types import PollingInterfaceLoopFunction
75 changes: 29 additions & 46 deletions dff/messengers/common/interface.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,15 @@
import abc
import asyncio
import logging
import uuid
from typing import Optional, Any, List, Tuple, TextIO, Hashable, TYPE_CHECKING

from dff.script import Context, Message
from dff.messengers.common.types import PollingInterfaceLoopFunction
from pathlib import Path
from tempfile import gettempdir
from typing import Optional, Any, List, Tuple, Hashable, TYPE_CHECKING, Type

if TYPE_CHECKING:
from dff.script import Context, Message
from dff.pipeline.types import PipelineRunnerFunction
from dff.messengers.common.types import PollingInterfaceLoopFunction
from dff.script.core.message import Attachment, DataAttachment

logger = logging.getLogger(__name__)

Expand All @@ -27,6 +28,24 @@ class MessengerInterface(abc.ABC):
It is responsible for connection between user and pipeline, as well as for request-response transactions.
"""

supported_request_attachment_types: set[Type[Attachment]] = set()
supported_response_attachment_types: set[type[Attachment]] = set()

def __init__(self, attachments_directory: Optional[Path] = None) -> None:
tempdir = gettempdir()
if attachments_directory is not None and not str(attachments_directory.absolute()).startswith(tempdir):
self.attachments_directory = attachments_directory
else:
warning_start = f"Attachments directory for {type(self).__name__} messenger interface"
warning_end = "attachment data won't be cached locally!"
if attachments_directory is None:
self.attachments_directory = Path(tempdir) / f"dff-cache-{type(self).__name__}"
logger.warning(f"{warning_start} is None, so will be set to tempdir and {warning_end}")
else:
self.attachments_directory = attachments_directory
logger.warning(f"{warning_start} is in tempdir, so {warning_end}")
self.attachments_directory.mkdir(parents=True, exist_ok=True)

@abc.abstractmethod
async def connect(self, pipeline_runner: PipelineRunnerFunction):
"""
Expand All @@ -38,6 +57,9 @@ async def connect(self, pipeline_runner: PipelineRunnerFunction):
"""
raise NotImplementedError

async def populate_attachment(self, attachment: DataAttachment) -> bytes:
raise RuntimeError(f"Messanger interface {type(self).__name__} can't populate attachment {attachment}!")


class PollingMessengerInterface(MessengerInterface):
"""
Expand Down Expand Up @@ -119,7 +141,8 @@ class CallbackMessengerInterface(MessengerInterface):
Callback message interface is waiting for user input and answers once it gets one.
"""

def __init__(self):
def __init__(self, attachments_directory: Optional[Path] = None) -> None:
super().__init__(attachments_directory)
self._pipeline_runner: Optional[PipelineRunnerFunction] = None

async def connect(self, pipeline_runner: PipelineRunnerFunction):
Expand All @@ -142,43 +165,3 @@ def on_request(
This method has the same signature as :py:class:`~dff.pipeline.types.PipelineRunnerFunction`.
"""
return asyncio.run(self.on_request_async(request, ctx_id, update_ctx_misc))


class CLIMessengerInterface(PollingMessengerInterface):
"""
Command line message interface is the default message interface, communicating with user via `STDIN/STDOUT`.
This message interface can maintain dialog with one user at a time only.
"""

def __init__(
self,
intro: Optional[str] = None,
prompt_request: str = "request: ",
prompt_response: str = "response: ",
out_descriptor: Optional[TextIO] = None,
):
super().__init__()
self._ctx_id: Optional[Hashable] = None
self._intro: Optional[str] = intro
self._prompt_request: str = prompt_request
self._prompt_response: str = prompt_response
self._descriptor: Optional[TextIO] = out_descriptor

def _request(self) -> List[Tuple[Message, Any]]:
return [(Message(input(self._prompt_request)), self._ctx_id)]

def _respond(self, responses: List[Context]):
print(f"{self._prompt_response}{responses[0].last_response.text}", file=self._descriptor)

async def connect(self, pipeline_runner: PipelineRunnerFunction, **kwargs):
"""
The CLIProvider generates new dialog id used to user identification on each `connect` call.

:param pipeline_runner: A function that should process user request and return context;
usually it's a :py:meth:`~dff.pipeline.pipeline.pipeline.Pipeline._run_pipeline` function.
:param \\**kwargs: argument, added for compatibility with super class, it shouldn't be used normally.
"""
self._ctx_id = uuid.uuid4()
if self._intro is not None:
print(self._intro)
await super().connect(pipeline_runner, **kwargs)
49 changes: 49 additions & 0 deletions dff/messengers/console.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
from typing import Any, Hashable, List, Optional, TextIO, Tuple
from uuid import uuid4
from dff.messengers.common.interface import PollingMessengerInterface
from dff.pipeline.types import PipelineRunnerFunction
from dff.script.core.context import Context
from dff.script.core.message import Message


class CLIMessengerInterface(PollingMessengerInterface):
"""
Command line message interface is the default message interface, communicating with user via `STDIN/STDOUT`.
This message interface can maintain dialog with one user at a time only.
"""

supported_request_attachment_types = set()
supported_response_attachment_types = set()

def __init__(
self,
intro: Optional[str] = None,
prompt_request: str = "request: ",
prompt_response: str = "response: ",
out_descriptor: Optional[TextIO] = None,
):
super().__init__()
self._ctx_id: Optional[Hashable] = None
self._intro: Optional[str] = intro
self._prompt_request: str = prompt_request
self._prompt_response: str = prompt_response
self._descriptor: Optional[TextIO] = out_descriptor

def _request(self) -> List[Tuple[Message, Any]]:
return [(Message(input(self._prompt_request)), self._ctx_id)]

def _respond(self, responses: List[Context]):
print(f"{self._prompt_response}{responses[0].last_response.text}", file=self._descriptor)

async def connect(self, pipeline_runner: PipelineRunnerFunction, **kwargs):
"""
The CLIProvider generates new dialog id used to user identification on each `connect` call.

:param pipeline_runner: A function that should process user request and return context;
usually it's a :py:meth:`~dff.pipeline.pipeline.pipeline.Pipeline._run_pipeline` function.
:param \\**kwargs: argument, added for compatibility with super class, it shouldn't be used normally.
"""
self._ctx_id = uuid4()
if self._intro is not None:
print(self._intro)
await super().connect(pipeline_runner, **kwargs)
12 changes: 1 addition & 11 deletions dff/messengers/telegram/__init__.py
Original file line number Diff line number Diff line change
@@ -1,14 +1,4 @@
# -*- coding: utf-8 -*-

try:
import telebot
except ImportError:
raise ImportError("telebot is not installed. Run `pip install dff[telegram]`")

from .messenger import TelegramMessenger
from .abstract import telegram_available
from .interface import PollingTelegramInterface, CallbackTelegramInterface
from .message import TelegramUI, TelegramMessage, RemoveKeyboard, ParseMode
from .messenger import (
UpdateType,
telegram_condition,
)
Loading