Showcasing Model Incorporation Workflow

[GemmaTuron]

Hi!

For the next rounds of internships with Outreachy (and other programs) It would be really helpful to have a one-page example of the model incorproration workflow, perhaps choosing one example and walking candidates through it?
We can use this discussions to craft a good document, which will then be uploaded into our GitBook

Some tips:

1. Work collaboratively and select one model that you all find is a good example
2. Do not use screenshots, ever. Write code manually if needed
3. Focus specially on how to debug a problem

[DhanshreeA]

A good reference as pointed out by Gemma is @carcablop's amazing work in describing the challenges she faced in this issue. #590

I would also like to add an example from incorporating ImageMol-HIV model wherein I imagined an issue while trying to review the results from the model's predictions, simply because of the statistics of the underlying dataset the model was trained on. Referencing it here: #532 (comment)

[DhanshreeA]

Hello @pauline-banye @DhanshreeA @karthikjetty @Femme-js Thank you @pauline-banye. Concerning what you mention about complex, I instead think that it can be a good example, mentioning the conversion of CUDA so that it uses CPU, precisely because there may be internals who have no experience with these ML libraries, as it was in my case, and this could save them time in coding and debugging the model, besides that it can be common to find this type of models that use neural networks, that are coded to run with cuda. We will try to document it in such a way that this is not so complex, providing clear examples and mentioning the official documentation.

@DhanshreeA has also proposed adding an example of the ImageMol-HIV model. I have seen that this model also uses a configuration similar to the one I used with the idl-ppb model to run the model with CPU and uses the pytorch library. I don't know if @DhanshreeA also had to change a lot of things in the original code so that it would be executed with CPU, but if it is not the case, we can also add it as an example, that is, add the steps that I did with the idl-ppb one. Your configuration to set the device to CPU is the same. https://github.com/ersilia-os/eos6hy3/blob/main/model/framework/code/main.py#L14. And the loading of the model is the same. I find that with this model we can incorporate what I did with idl-ppb and incorporate dhanshree's example. What do you think? We should think of a model that brings together several aspects not even mentioned in the documentation. With this model we find two aspects that I consider important, that's why I bring it up for discussion and we decide on the one that has more special characteristics in the model incorporation, that are not currently in the ersilia documentation.

What do you think? Shall we continue with the model that I worked on? Shall we propose the one with the ImageMol-HIV that Dhanhree worked on? Or do we discuss another model.?

@carcablop I see that you have already included code for IDL-PPB, and it looks good to me. If you or the others think it right we can change it to ImageMol-HIV. However, since you have taken the lead on this document, I really think we should use your model! 😃

Finally I think we will need to create a markdown file with the content from the Google Doc so that it can be commited to the Ersilia repo. I am not sure where it should go so @GemmaTuron can help with that.

[carcablop]

Thank you very much @pauline-banye and @DhanshreeA . I agree we continue with the model.
Regarding creating the markdown file, I think it's not necessary, because I think this is for some Ersilia gitbook session.

[carcablop]

@pauline-banye , I think that at this point, we should stop for a moment and think about the structure of the workflow, that is, establish the main points of the step by step, after each one is assigned a point to work on. What do you think?. I propose that the structure be like this:

1. Choose the model to incorporate into the Ersilia model Hub
2. Reading of the paper, for the interpretation of the model.
   In this process of reading and understanding what the model does, the parameters must be identified as the data set, type of model (pretrained, if it is necessary to retrain, etc.), verify the inputs, python version, code, the output type. Taking this into account, we can open a pull request to request the model.
3. Open a pull request to request the model.
4. Test the original model on the local machine. Here the dependencies are identified, the changes that must be made to the code can be identified and tested with eml_canonical.csv.
5. The output of the original model with the eml_canonical as input should be shared on the github issue of the model request. so that the ersilia coordinators can see if this model can really be incorporated or not, according to the analysis and interpretation of both input and output data.
6. After this, the model can be approved or not. In case it is approved. Then do the steps we know of forking the repository, cloning the repository, and modifying the code, etc.. Here also add the gitlfs.
   7.. The code must be executed successfully on our machine before merge with the ersilia repository. Therefore, at least one log of the fetch that the model was executed within ersilia successfully and that it was able to make the predictions must be shared in the github issue of the model. (This with the repo_path.).

What do you think of these steps? I believe that we must establish the steps well, so that we can work in a more organized way in the document. Let's take some time for this before continuing to edit the document.

[paulinebanye]

@carcablop I agree with this wholeheartedly 💯. I considered this and created an outline of the steps before I actually saw this message.

That's not a problem. You have tested and worked with this model so I feel you could point us in the right direction.

[paulinebanye]

I would suggest we take a page out of the Ersilia gitbook and explain the installation of the requirements like conda and git lfs before going into the steps for testing and the incorporation of the model

[carcablop]

Hello @DhanshreeA @pauline-banye @karthikjetty @Femme-js.
I think that for my part I have finished the document.
@pauline-banye, Could you please help me by modifying the style in the parts where I add code? Since I don't have the extension installed that you used. And take a look at the last thing I added. Thank you.
Everyone thanks for the work. The document remains open to make any modifications deemed necessary.
I think that we should eliminate part 1 of the document (1. Prerequisites), rather in one sentence we can mention that Ersilia should be installed on the system correctly, and provide the link to the documentation to install ersilia.
Also, remove the last part that says "Debugging steps". There is not adding relevant information to the workflow of the model.

[GemmaTuron]

I will close this discussion since the documentation is now up to date in our GitBook, many thanks for your help!