How to get a cool thesis project (in Deep Learning)

2021, Aug 31    

Disclaimers:

• The following text is based on my experiences of German university standards. This might not be true for your particular university but the insights about your advisor/professor’s reasoning should be true and, thus, helpful regardless.
• Deep Learning is used as the buzzword it currently is, so you can replace it with whatever field is “hot” right now in your area of expertise.
• While the text is mostly targeted at a master thesis, it can be applied to any advisor-based project in a university setting.

Supply and demand

In smaller universities and, to some degree, less over-run fields (e.g., Mathematics) matching advisors and students is a piece of cake. Professors have some smaller projects that they want to outsource and students are eager to learn the craft of their studies or simply want to be done with their masters. This harmony enables both parties to advance their goals, without too much effort from either side. Professors are not reliant on students, but they are a healthy way to carry out some basic research and an easy opportunity to scout for potential Ph.D. candidates. From a professor’s perspective, supervising a student is pretty time-consuming, but it is on the one hand required for most professors and, on the other hand, having a few per semester is manageable and one usually distributes the work among more seasoned PhDs. From the student’s perspective, most don’t worry too much about their future advisor early in their studies and, usually, this is the right approach as preferences switch heavily with more time invested in their particular study program. There are always some more eager students that plan ahead and try to secure their spot with their favorite professor early, but most of the time this doesn’t matter. In the usual supply and demand scenario, this doesn’t pose to be a huge problem as students distribute themselves among the professors depending on their interest in each semester and everyone is happy.

This harmony is disrupted once a certain field “outshines” its peers in popularity. Deep Learning can be seen as such an example, which dominated previous approaches across multiple fields. This motivated more and more companies as well as universities to invest money, time, and resources into that topic. Soon, it was used as a buzzword in multiple news outlets, companies were eager to hire graduates with minimal knowledge to outcompete their rivals, and students wanted to get a piece of the cake as well by securing a high-paying job in this new market.

However, with such a high supply of both students and resources from companies, professors are under the position where they can and have to choose on whom they want to spend their, or their Ph.D.’s, time. Their selection process can be seen as quite confusing by students which leads to frustrating experiences if those are unable to secure their favorite topic and have to settle with lesser options.

In this post, we are going to highlight

• the motivation of professors to hire students in the first place,
• which options professors have at their disposal to make their “confusing” decisions, and
• what students can do to improve their chances to secure themselves their dream project.

External theses - the dread of any advisor

Let’s begin with a simple example, the external thesis where a student secures a “job” at a company and “only” needs to find a university advisor that enables them to use their research or work for a master thesis to finish their studies. As a student, this seems like a win-win situation: you get paid already while still studying, you are getting in contact with a prospective employer and can usually use the internal connection to apply for future jobs. The company, on the other hand, gets a cheap worker who they can employ on a trial basis so they don’t have to make graduates full-time employees right away.

On the other hand, a professor has to supervise a project where they usually have little to say about the general direction, still have to meet weekly/monthly with the student to check progress, and finally grade the finished thesis. If not for the exceptional case where, the chosen topic is the exact direction they want to explore themselves, it usually is a huge time sink for them with no substantial gains. So… why would one want to engage in such an advisor commitment? Sometimes companies have access to data or the topic is interesting for the professor so that it is time well spent. Other times, the professor gets actual funding from a company and agrees to supervise a certain number of students for this company each year.

That’s why you get those lame “I am sorry, I have too much on my plate” types of excuses from potential advisors, may it be PhDs, postdocs, or professors. This is especially true for a field like Deep Learning, where companies spend huge sums on chairs to get access to their research and, thus, professors are not reliant on funds where they are forced to take in external students.

So… tell me what you want, what you really, really want…

Ideally, an advisor-student relationship turns out like this: you start by meeting quite frequently in the beginning to set the foundation of required knowledge for the project as well as getting a student up to the required level. At some point, the student out-knowledges the advisor in the nitty-gritty details of said project and carries out independent research which requires a lesser amount of meetings to sort out questions/issues but still enables the advisor to track if the project direction is kept in mind and in line with the advisor’s goals.

In a perfect world, the produced insights outperform state-of-the-art and one can turn the thesis into a full-fledged out paper which is then published at a conference or journal in that field. Eager students’ eyes are now lighting up, but I can assure you that this usually is pretty unlikely and, ultimately, a pretty time-consuming task for the student even way past the thesis submission. Generally, though, I highly suggest looking for such opportunities if they are presented to students (I mean… why wouldn’t I ^^). Regardless of the outcome, the advisor gets his research question (partially) answered and the student was confronted with independent work and a research setting, which will help him in future university projects, his own research, or at his actual future job.

In contrast to the external thesis, an advisor is free to change the topic to his liking and is not required to compromise with a second party. In addition, if the collaboration was to the liking of both parties there is a non-zero chance for them to continue their work in the form of a Ph.D. employment, which is super beneficial for a professor as he gets a known highly motivated student as his employee that already has work experience (similar to the company side in the previous external thesis example, where most students first goal is to get employed by the said company). That’s why professors prefer students working in their lab from the get go.

“Why would he choose X over me, that is so unfair!”

So after figuring out the goals of advisors, let’s look at the metrics they have at their disposal to choose potential candidates. Ultimately, this boils down to:

• quantitative results, such as a student’s overall grade or specifically related exams, or
• qualitative reports, like a reference letter of a previous advisor/a person the professor values or a previous interaction of student and professor, like a prior guided research project.

(Quick rant about reference letters: a message telling me that “student was always in my classes and achieved very good results” is literally worthless as it only repeats grades. If you want a reference letter, it should be about a specific situation where your advisor can share personal experiences with you. Rant end.)

For most students in the usual situation, this will boil down to grades. However, since grade inflation is a thing and you can rarely estimate the difficulty of other exams, professors highly overvalue the results of their lectures (or colleagues whose exams they personally know, but those are rarer cases). Obviously, professors know that exams can be pretty random result-wise, especially in this specific case where you are left with 1-2 classes taught by said professor where you can show off your skills. A student can always have a bad day, be sick on the exam date or simply spend too much time on the wrong question, etc. And not all of those deficiencies translate to their future research performance. So why are advisors still preferring those with better grades? It is the supply and demand issue all over again: while I can’t know for sure that a student with a 2.7 is necessarily bad in a given subject, I am pretty convinced that one of my two 1.0 candidates out of a whole class of students should be pretty proficient. That doesn’t mean that he will make an excellent researcher but with the limited time most professors are willing to spend on this issue, given the large supply of potential candidates and the limited amount of information available to them, it is their best and preferred shot.

Personal recommendations will outperform other metrics, but usually getting into such a situation requires the grades to convince said advisor to begin with and we end up with a chicken-and-egg problem.

So… the solution is to simply get better grades? :(

Ghosting and other monstrosities

Before we talk about alternatives to the above conclusion, let me quickly explain seemingly negative common advisor behavior. You write up your proposal, attach your CV/grades and send the email out to your potential advisor. After several days… nothing. You get impatient but decide to wait it out. Two weeks have gone, still nothing. You send out another email, referencing the first attempt. After one week you finally get a reply! A one-liner: “Hi, sorry I am unable to take more students right now. Please try again later.”. What an a**hole… Sometimes, you actually get nothing.

Stories like the above a pretty common and I honestly, despite my best efforts, did the same thing to various students to whom I wholeheartedly apologize. In the end, you have to come to an understanding that they are pretty likely the more “hot” your research topic is right now. I know, it sucks pretty hard. You must understand the situation and after the initial frustration, you can start making actual plans to tackle the issue.

What can you actually do?

But in the end, it is not all that bad. Honestly, most of the time you will run into a nice advisor regardless, and even if it is not always your first choice, you will tackle your first own research problem and get the experience and lessons required to tackle bigger problems to aim for the Ph.D. or company position you desire. However, if you want to get picky, you have to put in the work in order to stand a chance. A harsh truth, but you have to know what you are in for. Most importantly, you must refrain from simple application mistakes to not get sorted out from the get-go.

Regardless of your ambitions, I advise the following:

• Research your potential advisor and get familiar with his research.
• Make connections from your previous studies and write up a nice explanation of why you can help him in his research and, in particular, what problems you are interested in. You don’t need to formulate a project goal from the get-go because usually you don’t have the competence and prefer to let your advisor pick the most suitable project he sees fit for you.
• Pack it all up in a personalized message together with your grades and highlight previous experiences that especially fit the current role, so that the advisor has an easier time identifying your strengths and how he can use them.

While sometimes quantity beats quality, this is not the case for such applications. So refrain from template-based applications. And while at this point you might think “who would do something like this?”, those are pretty common and I personally tend to not even answer them anymore as the student certainly didn’t spend a lot of time if he is praising the work of my colleague in the same email he wrote to me as well as the whole department (yes, we share gems like that between each other and it will only get you blacklisted without any substantial gains).

What if I want to go beyond? I really want to get to advisor X because I fancy applying for a Ph.D. candidate position there…

Finally, we reach the point that interests the eager students. Some students figure out the whole issue above by themselves early in their semesters (I didn’t, oops), but even if you get those things right you might wonder what else you can do to increase your odds. What you have to keep in mind is that your work might be for naught. Therefore, you should prioritize topics that actually interest you and where you don’t mind spending hours in the next couple of weeks because it makes you happy and challenges you. In fact, by doing your own research like this, you will take your first step in the independent working structure that dominates your master thesis and maybe your future Ph.D. time. It is an exciting period (to me at least) and I would only advise it to those that have a similar mindset. Otherwise, you will get frustrated by “useless” work and, to be honest, you might not be cut for the work required ahead of your path.

So what we know from the above is that we have to convince your advisor that we can help him with their research. The simplest way to do so is to read their papers and try to spot common methods/datasets/etc. in the field you aim to get into (most professors have a multitude of topics they are researching in so you have to focus on one of them). For Deep Learning, there is usually a shared dataset people are comparing themselves to in most specializations, which is the best-case scenario here. Your job is to start working ahead of time on this dataset/methods to understand and apply them yourself. In Deep Learning or any coding-related field having an open GitHub repository where you share your work is reaaaally valuable. Regardless of your work topic, it enables future advisors to check out your coding style and work methods if you additionally share your findings and taken steps in (multiple) README(S). Otherwise, reimplement existing architectures with a focus on those that don’t have a running implementation in your coding language already so that others can profit from it. If this is more difficult to figure out in your field or you want more confirmation if you are tackling the right problem, consider asking your professor after class in person (if that is possible in the current corona situation…) which eliminates the need for him to actually read your email. This especially works better, if you actively participate in the class by asking or answering questions so that your professor remembers your face.

Regardless if your favorite advisor will consider your application, carrying out your first own research, publishing it for others to scrutinize and, at best, learn or use it for their work will help you tremendously. It shows that you know the foundations already and this is super helpful for any advisor because that means he doesn’t have to spend the time teaching you those lessons.

Parting words

I hope this post gives you a reasonable insight into the minds of an advisor and, therefore, enables you to some degree understand their actions as well as their needs to increase your odds of getting the project or position you so desire. In the end, you have to decide for yourself if that is what you want and if you want to spend the time on yourself. Your utmost focus should be on your personal gains from both the content as well as the knowledge of your advisor. Spend time on the stuff you like and on topics that interest you. While you mostly can get through your studies without thinking about the future, selecting your (master) thesis profits from a vision from your side.

Or… just simply enjoy studying like myself and figure everything out after you are done. There is no right or wrong path, even if it might take you longer, as long as you enjoy your journey ;)