Added COSP #823
Added COSP #823
Conversation
![ellipsis-dev[bot]](https://avatars.githubusercontent.com/in/64358?s=60&v=4){kind=small}
There was a problem hiding this comment.
👍 Looks good to me! Reviewed everything up to c4b6731 in 28 seconds
More details
- Looked at
381lines of code in4files - Skipped
3files when reviewing. - Skipped posting
1drafted comments based on config settings.
1. mkdocs.yml:251
- Draft comment:
The navigation title forcosp.mdhas been changed from 'COSP' to 'Prioritize Consistent Examples'. If this change was intentional, please update the PR description to reflect this. Otherwise, revert the title change in the navigation. - Reason this comment was not posted:
Confidence of 0% on close inspection, compared to threshold of 85%.
Workflow ID: wflow_OcMJtsXCk7tzMPE9
You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.
Deploying instructor with
|
| Latest commit: |
5eab504
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://d9f28bc2.instructor.pages.dev |
| Branch Preview URL: | https://cosp.instructor.pages.dev |
There was a problem hiding this comment.
❌ Changes requested. Incremental review on bc0121d in 1 minute and 4 seconds
More details
- Looked at
268lines of code in3files - Skipped
0files when reviewing. - Skipped posting
0drafted comments based on config settings.
Workflow ID: wflow_BV2xYUn7E82Thxnq
Want Ellipsis to fix these issues? Tag @ellipsis-dev in a comment. You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.
There was a problem hiding this comment.
👍 Looks good to me! Incremental review on b2786d3 in 36 seconds
More details
- Looked at
314lines of code in5files - Skipped
0files when reviewing. - Skipped posting
1drafted comments based on config settings.
1. mkdocs.yml:251
- Draft comment:
The navigation title for COSP inmkdocs.ymlis 'Prioritize Consistent Examples', which might not clearly indicate the full technique name 'Consistency Based Self Adaptive Prompting'. Consider updating it to reflect the full name for clarity.
- Consistency Based Self Adaptive Prompting: 'prompting/ensembling/cosp.md'
- Reason this comment was not posted:
Confidence of 0% on close inspection, compared to threshold of 85%.
Workflow ID: wflow_8GFYdUM6SGAmCVeW
You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.
docs/prompting/ensembling/cosp.md
Outdated
|
|
||
|
|
||
| def score_repetitiveness(prediction: Response): | ||
| sentences = [item for item in prediction.chain_of_thought.split(".") if item] |
docs/prompting/ensembling/cosp.md
Outdated
|
|
||
| !!! Info "Quick Note" | ||
|
|
||
| Make sure to install `numpy` and `scikit-learn` before running the code, we need to use these two libraries to compute the cosine similarity |
There was a problem hiding this comment.
| Make sure to install `numpy` and `scikit-learn` before running the code, we need to use these two libraries to compute the cosine similarity | |
| Make sure to `pip install numpy scikit-learn` as we will need them for cosine computations |
can also implement a simple costine function in a list comprehension to avoid a install
There was a problem hiding this comment.
👍 Looks good to me! Incremental review on c9b3da6 in 52 seconds
More details
- Looked at
301lines of code in2files - Skipped
0files when reviewing. - Skipped posting
2drafted comments based on config settings.
1. docs/prompting/ensembling/cosp.md:96
- Draft comment:
Consider using thecosine_similarityfunction from a library likescikit-learninstead of a custom implementation to ensure optimized and accurate calculations.
from sklearn.metrics.pairwise import cosine_similarity
This suggestion applies to the entire custom implementation of cosine_similarity from lines 96 to 104.
- Reason this comment was not posted:
Confidence of 20% on close inspection, compared to threshold of 85%.
2. docs/prompting/ensembling/cosp.md:107
- Draft comment:
It's recommended to use thecosine_similarityfunction fromscikit-learnfor calculating repetitiveness to ensure optimized and accurate calculations. This would replace the custom implementation in thescore_repetitivenessfunction.
from sklearn.metrics.pairwise import cosine_similarity
This change should be applied to the entire score_repetitiveness function.
- Reason this comment was not posted:
Marked as duplicate.
Workflow ID: wflow_fPbH76ULH8ZufScO
You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.
There was a problem hiding this comment.
👍 Looks good to me! Incremental review on 5eab504 in 1 minute and 10 seconds
More details
- Looked at
13lines of code in1files - Skipped
0files when reviewing. - Skipped posting
1drafted comments based on config settings.
1. mkdocs.yml:258
- Draft comment:
The PR description mentions updates tomkdocs.ymlfor KaTeX support, but the diff provided does not show these changes. Please ensure that all intended changes are correctly committed. - Reason this comment was not posted:
Confidence of 0% on close inspection, compared to threshold of 85%.
Workflow ID: wflow_LeKWK9tlFkQNyh0p
You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.
Summary:
Added documentation and implementation details for COSP, updated
mkdocs.ymlfor KaTeX support, and made minor formatting fixes.Key points:
docs/javascripts/katex.jsfor rendering mathematical expressions using KaTeX.mkdocs.ymlto include KaTeX-related JavaScript and CSS files.docs/prompting/ensembling/cosp.md.mkdocs.ymlto reflect the new COSP documentation.docs/prompting/thought_generation/chain_of_thought_few_shot/uncertainty_routed_cot.md.instructor/dsl/partial.pyandinstructor/function_calls.py.Generated with ❤️ by ellipsis.dev