Has anyone tried implementing raytune for hyperparameter optimization?

[k-nayak]

Since most people here are having smaller datasets and struggling with playing around different hyperparameters, I came across Ray tune library for hyperparameter tuning, has anyone had any experience with it?

https://docs.ray.io/en/master/tune/index.html

[Sehjbir]

I have used Optuna: https://optuna.readthedocs.io/en/stable/index.html to tune my parameters. Ray also has an implementation of optuna. You can tune by making few changes in the train.py file

[k-nayak]

Thank you, i will look into it. Could you show how you have made changes in your code for U-net (train.py) ? Have you also changed the architecture and did it give better results?

[Sehjbir]

@k-nayak Sure i can share it. i haven't made any architecture changes as of now. were you able to implement attention-unet ? can you share it with me?
About improvement: I'm currently running the training process. Will let you know

[k-nayak]

Attention U-net gave a better Dice in my case and the loss was also better. In 50 epochs the change was not significant, i plan on trying RAUnet as well , based on the paper it performs better

@Sehjbir Sure i can share how i implemented Attn Unet. It would be great to see how you implemented Optuna, as i plan on Tuning my parameters on Attn Unet and see if it helps.

Let me know how you would want me to share the Code and you can do it as well.

[Sehjbir]

@k-nayak i can email you the code repo i updated. does that work ?

[k-nayak]

Sure @Sehjbir I can send you the code to your mail as well

Or create a private repo and i can maybe upload my code there.

[k-nayak]

@Sehjbir Thanks for the code. I have uploaded my Train, Unet_parts and Unet_model files. Give it a try and let me know if you need any support.

[Sehjbir]

@k-nayak while running the training i'm getting this error: RuntimeError: The size of tensor a (74) must match the size of tensor b (75) at non-singleton dimension 3. I'm using the same training and valid set i used to train U-net any suggestions?

[Sehjbir]

@k-nayak while running R-Unet: RuntimeError: Sizes of tensors must match except in dimension 2. Got 48 and 49 (The offending index is 0).

[Sehjbir]

@k-nayak while running U-net it's coming: unet_parts_updated.py", line 137, in forward
psi = self.relu(g1+x1)
RuntimeError: The size of tensor a (74) must match the size of tensor b (75) at non-singleton dimension 3

[Sehjbir]

@k-nayak any possible solution?

[sree3333]

Does anyone have this model implemented with Ray?