feat: support for tools in `HuggingFaceAPIChatGenerator` #8661

anakin87 · 2024-12-19T10:12:09Z

Related Issues

part of Port Tools from experimental #8631

Proposed Changes:

basically a porting of HuggingFaceAPIChatGenerator from experimental
there are some slight changes, that I will try to comment in this PR to make them visible

How did you test it?

CI, several new tests

Checklist

I have read the contributors guidelines and the code of conduct
I have updated the related issue with new insights and changes
I added unit tests and updated the docstrings
I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test: and added ! in case the PR includes breaking changes.
I documented my code
I ran pre-commit hooks and fixed any issue

coveralls · 2024-12-19T10:46:33Z

Pull Request Test Coverage Report for Build 12413858648

Details

0 of 0 changed or added relevant lines in 0 files are covered.
11 unchanged lines in 2 files lost coverage.
Overall coverage increased (+0.04%) to 90.693%

Files with Coverage Reduction	New Missed Lines	%
components/generators/chat/hugging_face_api.py	4	96.4%
utils/hf.py	7	86.29%

Totals
Change from base Build 12410533994:	0.04%
Covered Lines:	8351
Relevant Lines:	9208

💛 - Coveralls

anakin87 · 2024-12-19T10:51:43Z

haystack/dataclasses/tool.py

@@ -216,6 +216,19 @@ def _remove_title_from_schema(schema: Dict[str, Any]):
                del property_schema[key]


+def _check_duplicate_tool_names(tools: List[Tool]) -> None:


this is a common check we do, so I moved it to the Tool module

anakin87 · 2024-12-19T10:54:28Z

haystack/utils/hf.py

@@ -270,6 +270,44 @@ def check_generation_params(kwargs: Optional[Dict[str, Any]], additional_accepte
            )


+def convert_message_to_hf_format(message: ChatMessage) -> Dict[str, Any]:


We should use this function in the HF Local Chat Generator too (I recently discovered that it does not work properly without the conversion), so I moved it to utils.

(I will open issue + PR to update the HF Local Chat Generator.)

anakin87 · 2024-12-19T10:56:39Z

haystack/components/embedders/hugging_face_api_document_embedder.py

@@ -14,7 +14,7 @@
 from haystack.utils.hf import HFEmbeddingAPIType, HFModelType, check_valid_model
 from haystack.utils.url_validation import is_valid_http_url

-with LazyImport(message="Run 'pip install \"huggingface_hub>=0.23.0\"'") as huggingface_hub_import:
+with LazyImport(message="Run 'pip install \"huggingface_hub>=0.27.0\"'") as huggingface_hub_import:


This is only a suggestion for users.
The ChatCompletionInputTool type was introduced recently, so it is better to install the latest version.

vblagoje

Looks good, I left a a few optional comments

vblagoje · 2024-12-19T13:16:16Z

haystack/components/generators/chat/hugging_face_api.py

@@ -159,6 +163,11 @@ def __init__(  # pylint: disable=too-many-positional-arguments
            msg = f"Unknown api_type {api_type}"
            raise ValueError(msg)

+        if tools:
+            if streaming_callback is not None:


Should we do this across the board for all CG - just something to write down and not forget

No. Most of Generators support Tools + Streaming, I think.

Theoretically also HF could support this but it is highly undocumented and does not behave consistently.
See deepset-ai/haystack-experimental#120 (comment)

vblagoje · 2024-12-19T13:20:06Z

haystack/components/generators/chat/hugging_face_api.py

+            "finish_reason": choice.finish_reason,
+            "index": choice.index,
+            "usage": {
+                "prompt_tokens": api_chat_output.usage.prompt_tokens,


Ugh, I've been burned b4 on these usage structures assuming they are always there. Please check HF pydantic classes to make sure they are not optional.

I have now taken a more cautious approach. Thank you!

vblagoje · 2024-12-19T13:22:25Z

haystack/dataclasses/tool.py

@@ -216,6 +216,19 @@ def _remove_title_from_schema(schema: Dict[str, Any]):
                del property_schema[key]


+def _check_duplicate_tool_names(tools: List[Tool]) -> None:
+    """
+    Check for duplicate tool names.


and raises ValueError if they are found.

anakin87 added 2 commits December 19, 2024 10:31

message conversion function

4db6f40

hfapi w tools

44b0103

github-actions bot added type:documentation Improvements on the docs topic:tests labels Dec 19, 2024

right test file + hf_hub version

3bc5374

github-actions bot added the topic:build/distribution label Dec 19, 2024

release note

39c3383

anakin87 commented Dec 19, 2024

View reviewed changes

Merge branch 'main' into hfapi-tools

39184a6

anakin87 marked this pull request as ready for review December 19, 2024 10:55

anakin87 requested review from a team as code owners December 19, 2024 10:55

anakin87 requested review from dfokina, Amnah199 and vblagoje and removed request for a team and Amnah199 December 19, 2024 10:55

anakin87 commented Dec 19, 2024

View reviewed changes

vblagoje approved these changes Dec 19, 2024

View reviewed changes

anakin87 mentioned this pull request Dec 19, 2024

fix: Make the HuggingFaceLocalChatGenerator compatible with the new ChatMessage; serialize chat_template #8663

Merged

feedback

6832e31

anakin87 merged commit 2bc58d2 into main Dec 19, 2024
19 checks passed

anakin87 deleted the hfapi-tools branch December 19, 2024 14:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: support for tools in `HuggingFaceAPIChatGenerator` #8661

feat: support for tools in `HuggingFaceAPIChatGenerator` #8661

anakin87 commented Dec 19, 2024 •

edited

Loading

coveralls commented Dec 19, 2024 •

edited

Loading

anakin87 Dec 19, 2024

anakin87 Dec 19, 2024

anakin87 Dec 19, 2024

vblagoje left a comment

vblagoje Dec 19, 2024

anakin87 Dec 19, 2024

vblagoje Dec 19, 2024

anakin87 Dec 19, 2024

vblagoje Dec 19, 2024

anakin87 Dec 19, 2024

		@@ -216,6 +216,19 @@ def _remove_title_from_schema(schema: Dict[str, Any]):
		del property_schema[key]


		def _check_duplicate_tool_names(tools: List[Tool]) -> None:

		@@ -270,6 +270,44 @@ def check_generation_params(kwargs: Optional[Dict[str, Any]], additional_accepte
		)


		def convert_message_to_hf_format(message: ChatMessage) -> Dict[str, Any]:

feat: support for tools in HuggingFaceAPIChatGenerator #8661

feat: support for tools in HuggingFaceAPIChatGenerator #8661

Conversation

anakin87 commented Dec 19, 2024 • edited Loading

Related Issues

Proposed Changes:

How did you test it?

Checklist

coveralls commented Dec 19, 2024 • edited Loading

Pull Request Test Coverage Report for Build 12413858648

Details

💛 - Coveralls

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vblagoje left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

feat: support for tools in `HuggingFaceAPIChatGenerator` #8661

feat: support for tools in `HuggingFaceAPIChatGenerator` #8661

anakin87 commented Dec 19, 2024 •

edited

Loading

coveralls commented Dec 19, 2024 •

edited

Loading