Inappropriate implementation of Multi-head in IPT? (which only works on SR)

[hyunW3]

Hello, I'm interested in your work and try some simple things based on your work.
While i'm inspecting your code, I found some problem. I want to know whether it is real problem or misunderstanding

The passed argument to model in test time is degraded image and idx_scale as shown below

Pretrained-IPT/trainer.py

Line 42 in 675540e

sr = self.model(rain, idx_scale)

I thought "idx_scale" stands for task id, which denote "i" in $f_H = H^i(x)$ you mentioned in the paper 3.1 Head.

However, I found that "idx_scale" only varies depending on scale level in SR(Super Resolution), and "idx_scale" is set at 0 on denoising and deraining task.

Also, the number of heads is decided with "args.scale" list, not number of tasks.

Pretrained-IPT/model/ipt.py

Lines 33 to 39 in 675540e

	self.head = nn.ModuleList([
	nn.Sequential(
	conv(args.n_colors, n_feats, kernel_size),
	common.ResBlock(conv, n_feats, 5, act=act),
	common.ResBlock(conv, n_feats, 5, act=act)
	) for _ in args.scale
	])

I want to make sure this is right to improve your code (or repo)
Thank you reading this issue and looking forward your reponse.

[JGyoung-UCAS]

Hello, I'm interested in your work and try some simple things based on your work. While i'm inspecting your code, I found some problem. I want to know whether it is real problem or misunderstanding

The passed argument to model in test time is degraded image and idx_scale as shown below

Pretrained-IPT/trainer.py

Line 42 in 675540e

sr = self.model(rain, idx_scale)

I thought "idx_scale" stands for task id, which denote "i" in fH=Hi(x) you mentioned in the paper 3.1 Head.

However, I found that "idx_scale" only varies depending on scale level in SR(Super Resolution), and "idx_scale" is set at 0 on denoising and deraining task.

Also, the number of heads is decided with "args.scale" list, not number of tasks.

Pretrained-IPT/model/ipt.py

Lines 33 to 39 in 675540e

	self.head = nn.ModuleList([
	nn.Sequential(
	conv(args.n_colors, n_feats, kernel_size),
	common.ResBlock(conv, n_feats, 5, act=act),
	common.ResBlock(conv, n_feats, 5, act=act)
	) for _ in args.scale
	])

I want to make sure this is right to improve your code (or repo) Thank you reading this issue and looking forward your reponse.

I met with the same problem. Did you solve it?

[HantingChen]

Sorry for the unclear code. This implement is right since we only release the test code. Your implement is a right way to modify it into a training code.

[hyunW3]

@JGyoung-UCAS Actually, i'm not working with this code right now..
However, sharing my thoughts with you,
I think the implementation of the "self.head" should be concerning with "task_id", not "args.scale" to separate the head for each task. This might requires separation of "args.scale" variable into "task" and "scale in SR task"

@HantingChen Thank you for your response,
My question is whether the multi-head is well implemented regardless of training or test, since "idx_scale 0" head covers two task (denoising & deraining) with your implementation.

[JGyoung-UCAS]

@JGyoung-UCAS Actually, i'm not working with this code right now.. However, sharing my thoughts with you, I think the implementation of the "self.head" should be concerning with "task_id", not "args.scale" to separate the head for each task. This might requires separation of "args.scale" variable into "task" and "scale in SR task"

@HantingChen Thank you for your response, My question is whether the multi-head is well implemented regardless of training or test, since "idx_scale 0" head covers two task (denoising & deraining) with your implementation.

Thank you for the reminder. This code is very incomplete. Maybe you can check this code in the version of Minspore： https://gitee.com/mindspore/models/tree/master/research/cv/IPT. It seems to be more clear.