Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TPG创建数据时报错 #418

Open
liyifan20010515 opened this issue May 23, 2024 · 0 comments
Open

TPG创建数据时报错 #418

liyifan20010515 opened this issue May 23, 2024 · 0 comments

Comments

@liyifan20010515
Copy link

出现了多进程can‘t pickle问题
train_loader = DataLoader(train_dataset, sampler=LadderSampler(train_dataset, batch_size),
num_workers=num_workers, batch_size=batch_size,
collate_fn=lambda e: TPGDataset.collect_fn_quadkey(e, train_dataset,
sampler, self.QUADKEY,
self.loc2quadkey, k=num_neg, N_nearest=self.N_nearest, mix=mix, train=True, length=self.length))
报错信息如下:
EOFError: Ran out of input
return _MultiProcessingDataLoaderIter(self)
File "C:\some files\TPG-main\Bigscity-LibCity\venv\lib\site-packages\torch\utils\data\dataloader.py", line 914, in init
w.start()
File "C:\ruanjan\python 3.7\lib\multiprocessing\process.py", line 112, in start
self._popen = self._Popen(self)
File "C:\ruanjan\python 3.7\lib\multiprocessing\context.py", line 223, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "C:\ruanjan\python 3.7\lib\multiprocessing\context.py", line 322, in _Popen
return Popen(process_obj)
File "C:\ruanjan\python 3.7\lib\multiprocessing\popen_spawn_win32.py", line 89, in init
reduction.dump(process_obj, to_child)
File "C:\ruanjan\python 3.7\lib\multiprocessing\reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'TPGDataset.get_data..'
请问应该怎么修改呢,是不是不能这样使用

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant