- Vision
- NLP
- Multimodal Images and Text
- Applications(ML+X)
- Theory
- Systems
- Interpretability
- Many more Alphageometry!
- Safety
- UG
- MS/PreDoc/RA
- PhD.
- Academia - Many profs at IITD in CSE have done B.Tech. from IITs
- Startup - AI Startup Boom
- Industry - Company Research Scientist
- Insanely competitive at the moment
- Too many papers, revolution every two weeks. Quite hectic to catch up.
- Too little compute - Need access to a very good machine or GPU
- Problems are amorphous - For example, trying to solve IMO Problems, not directly tacklable
- SURA - Very good to teach the basics of research. Here you are actually a part of the team and bring your ideas to the table. A lot to learn.
- Get your fundamentals right:
- COL341, COL774, ELL409 for Statistical Machine Learning.
- COL775 for Deep Learning.
- COL761, COL870 for Graph Neural Networks.
- COL772, ELL881 for Natural Language Processing.
- COL778, COL864 for Reinforcement Learning (Robotics oriented).
- Take a research project:
- DAIR (Prof. Mausam, Prof. Parag, Prof. Rohan).
- LCS2 (Prof. Tanmoy).
- Sayan Ranu.
- Try cool things:
- Experiment on your own.
- PreDoc has funding available.
- Consider taking COL774 alongside COL333 in your fifth semester.
- You'll have an advisor to help you choose a problem statement.
- Long-term commitment is common if you choose to stay.
- Identifying the kind of research you want to do - the field. Doing some research will give you a proper idea. Working with professors.
- Choose a lab based on it.
- A place that your advisor has done in the past or a place known for your field.
- Most important - Letters of Recommendation from previous professors. GPA too is very important. SoP is not very important. Either of the above two should be good.
The place and people are amazing at Berkeley. It's been 3-4 months. At IIT too I found nice people. I advise everyone not to take courses lightly. If you understand the AI/ML courses nicely, you'll unlock the ability to read interesting important papers which you wouldn't have otherwise.
Q. Is there a huge difference between 9.4 and 9.9?
A. It's not as simple. Frankly, yes. But if your research output has been good, it won't matter much. It's not about ranks or cutoffs now. But it sure helps to have a good GPA.
Q. What is the right time to start applying?
A. You don't really get to do good research before the third year, which seems late but it's fine. In my 5th semester, I had an idea that I'll do research, I did COL774 and liked it. COD got converted to B.Tech. project. I didn't have a paper published by the time of application, but was just in time for a deadline. Some residency programs have started offering a year to research. I applied to many and Berkley was the best I could get. Start by 6th semester for the best options. It's hard to develop genuine interest before then.
Q. How to choose research over industry?
A. I like thinking about problems and, in general, don't have a genuine reason. I just found it exciting. The kind of people you get to meet in research is very different from those in the industry. The peer group I had was also a deciding factor. Try out corporate as well; what you like matters.
Q. How did you decide the US above Europe and Canada?
A. US had Stanford, MIT and Berkeley
Q. After the 4th semester, what did you do?
A. COL215 labs were scheduled for the summer vacation. The internship was of 60-70 days. Didn't do research in the 6th sem summer.
Q. Is a paper in a known journal necessary?
A. Not mandatory but definitely helps. Research experience is what matters. You have to have people say for you that you have done decent research work.
Most of the current progress focuses on scaling, leading to a significant increase in productivity. However, deeper integration can only occur when these systems reach a level of proficiency to serve as personal agents, which they currently lack in contextual understanding. It's widely known that these systems can be systematically broken down, making deployment in heavily regulated sectors like education challenging. Productivity gains have been observed primarily in specific areas, such as composing effective emails, achieved by cleverly leveraging good data. Highlighting the importance of not just generative capabilities but also effective data utilization, it's essential to consider nuanced perspectives.
Q. Your timeline after the second year?
A. I used to work in the field of Human Computer Interation or HCI(Google Glass). Then I shifted to Multimodal research in the third year.
Q. How to do impactful work here? How to choose the problem statement? What issues did you find with your HCI experience in that space?
A. "LLM for X is a standard in this space." I believe that scale and data are working very well. For real impact, there are a lot of algorithmic improvements which people are not working on due to high risk, high reward being present. You must have a long-term vision around which your problem statement will have to revolve. With robotics and HCI in general, the problem is the lack of benchmarking systems in place as everyone claims to have made a good model.
Q. The safety or alignment aspect revolves around experiments and lack theory. Shed some light on it. If all your work is experimental, is it counted as good research?
A. Algorithmic improvements are something which needs more energy in this space. What the community decides to be good research is based on benchmarks already present.
Q. Which is better, GPT wrappers or core AI? As a student, what's your take on going into research?
A. GPT wrappers don't have a long-term vision and that is what matters. I believe they won't last long. The bigger industry people are anyway going to take over with better original features. It's better to join the bigger companies if you don't have a long-term vision of your own. You should be able to see a couple of years ahead to get into research. If you don't enjoy it, remember that you've to spend around 5 years doing a PhD.
Q. Corporate research v/s university research?
A. It all depends on your interests. Academia can't compare with the scales corporate has. Corporate is well when it comes to scaling. I would prefer an academia environment if interdisciplinary research is the one someone wishes to do, otherwise if scaling is what you wish to be a part of, corporate setting's the one.
Q. What does the Pre-Doctoral role mean?
A. The Pre-Doctoral role is an opportunity for individuals to engage in research activities, especially for those who may not have had research experience during their undergraduate studies. It's worth noting that prominent figures, such as the CEO of OLA, have previously served as research fellows at companies like Microsoft. Additionally, Google's research environment is highly regarded in the industry.
Q. When do you apply for a Pre-Doctoral?
A. Dec or Jan in 4th year. An SoP, Grades, and LoR are what you need.
Q. What after PhD.?
A. Research scientist in industry or a professor in academia. Research scientists are paid very well. The difference also lies in the choice of topic; in industry, the company makes this choice for you. The profit you bring to the company is what you're judged on. Google search results improvement is an example. It's expected in a Pre-Doctoral role that a product is an output.
Q. What is an ideal roadmap for getting into Google Deepmind or Microsoft Research (Pre Ph.D.)? Is it possible to do both - research and making good money?
A. It's very difficult for the Pre-Doctoral level. There are RSD positions available. In India, getting into such a position is easier than abroad. Connections in the lab really help along with past experience in the field. "Building the wrappers" is a good place to start with.
Q. Why did you choose Pre-Doctoral over Ph.D.?
A. Categories:
1. People who have done good research.
2. People with decent research but want to strengthen their CV.
3. People with no experience in research.
For the latter two, Pre-Doctoral is a good choice. I was in the second category. You should also consider the opportunities available to you.