Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

swift 接入数据集下载后还是会不断请求网络 #1285

Open
fly-dragon211 opened this issue Jul 3, 2024 · 1 comment
Open

swift 接入数据集下载后还是会不断请求网络 #1285

fly-dragon211 opened this issue Jul 3, 2024 · 1 comment

Comments

@fly-dragon211
Copy link

Describe the feature
数据集最开始是下载好了,但是代码运行过程中好像会去请求数据集网络,有时网络会卡,造成训练中断。

[rank3]: OSError: Tunnel connection failed: 503 Service Unavailable[rank3]: The above exception was the direct cause of the following exception:[rank3]: urllib3.exceptions.ProxyError: ('Unable to connect to proxy', OSError('Tunnel connection failed: 503 Service Unavailable'))[rank3]: The above exception was the direct cause of the following exception:[rank3]: Traceback (most recent call last):[rank3]:   File "/data/miniconda3/envs/env-3.11.9/lib/python3.11/site-packages/requests/adapters.py", line 667, in send[rank3]:     resp = conn.urlopen([rank3]:            ^^^^^^^^^^^^^[rank3]:   File "/data/miniconda3/envs/env-3.11.9/lib/python3.11/site-packages/urllib3/connectionpool.py", line 843, in urlopen[rank3]:     retries = retries.increment([rank3]:               ^^^^^^^^^^^^^^^^^^[rank3]:   File "/data/miniconda3/envs/env-3.11.9/lib/python3.11/site-packages/urllib3/util/retry.py", line 519, in increment[rank3]:     raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type][rank3]:     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^[rank3]: urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='xingchen-data.oss-cn-zhangjiakou.aliyuncs.com', port=443): Max retries exceeded with url: /coco/2014/val2014/COCO_val2014_000000165638.jpg (Caused by ProxyError('Unable to connect to proxy', OSError('Tunnel connection failed: 503 Service Unavailable')))

版本:ms-swift 2.2.0.dev0

#982 这里说可以设置 --custom_train_dataset_path ,但是这个参数已经被废止:https://github.com/modelscope/swift/blob/main/docs/source/LLM/%E5%91%BD%E4%BB%A4%E8%A1%8C%E5%8F%82%E6%95%B0.md

@Jintao-Huang
Copy link
Collaborator

--train_dataset 直接指定本地路径就好了

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants