Expedia Group Director, Data Science
University of Utah PhD, Computing
Current Time 0:00
/
Duration Time -:-
Progress: NaN%

How did you get to where you are today? What is your story? What incidents and experiences shaped your career path?

Summarized By: Jeff Musk on Thu Dec 19 2019
I got my bachelor's in computer science in India, and I worked in a software services company for about three years building software for various insurance and banking clients. And, along the way, it's gonna figure out what I lack in the job is, building strong algorithmic solutions. And often that was because these were large banking systems for which all we had to do, was sort of maintain legacy code. And that wasn't very challenging for me. One of the things that I always loved doing was algorithms and I've taken a couple of basic machine learning classes when I was in undergrad, and that was one of the things that I thought I must in the things that I was doing. So I applied to grad school ten-eleven years ago And, when I did apply to grad school, I applied for a master's program because largely because I wanted a lay in the runoff from the theory and machine learning. I wasn't sure what exact problems I would be facing and Master's was a way for me to get my feet right. Trying to figure out what are the different opportunities that are available in graduate school. Eventually, I found myself at the University of Utah, who I worked with for a number of years, and I converted to a PhD program because I quickly figured out the list of problems that excited me and I started working on a number of unsupervised learning problems and defining the scope of Meta-Learning. One thing that helped me grow the graduate school part was to go for internships. Try and figure out what the problem is that the industry is excited about and to see if they can bring the Alogirthimic challenges back from the industry and trying to solve it, and that helps. That helped with my dissertation. I eventually ended up in a full-time job at Yahoo labs. I worked on different advertising scientists' problem for a while, moved on to a startup for a couple of years and now I'm at Expedia, where I manage medium-sized group of scientists, we work on a number of different problems from marketing dynamic pricing, the other some other natural language problems on and some vision problems.

What are the responsibilities and decisions that you handle at work? Discuss weekly hours you spend in the office, for work travel, and working from home.

Based on experience at: Director, Data Science, Expedia Group
Summarized By: Jeff Musk on Thu Dec 19 2019
When I started at Expedia, it was as an individual contributor. So in general, when you are in an industry, you have two major lines of a path as someone in the software industry. You can either be an individual contributor where you're working together with your team, you're collaborating, but you're still responsible for sort of building code, building algorithms, and building software. The other part is the managerial path where you can not only be involved in the technical design of algorithms, but you are also in charge of managing a team taking care of people's aspect as well. So, while I started as being an individual contributor or an IC, I recently moved on to managerial responsibility. And one of the things that I enjoy doing personally, it's to be able to get a small team together, go to find a problem, figure out the right approaches to the problem. Both help design and build solutions and then actively test the solutions to these problems. This path gave me an opportunity to sort of define everything from how do we define the problem to all the way to how do we test these algorithms in the right way. It's put me in a slightly different path than what I would have gone into a traditional IC. Something that I enjoy a lot. Working hours? Between 4 to 5 hours looking at collaborating with the scientists on the team, trying to define the problem, define approaches. About an hour or two is for meeting status updates and all that. the more history. And work from home? It's not something that people do every day of the week because of the commute and other personal reasons.

What tools (software programs, frameworks, models, algorithms, languages) do you use at work? Do you prefer certain tools more than the others? Why?

Based on experience at: Director, Data Science, Expedia Group
Summarized By: Jeff Musk on Thu Dec 19 2019
I'm a firm believer of having the right hammer for the nail. Often the simplest solution might just be enough to solve some problems. Not every business problem requires a mission learning solution. So when we start looking at the business problems that are sort of given to us and then you try to extract them out in clear problem of the machine, for which then try to find what the right algorithm is. Often they end up not being a Machine Learning solutions. Maybe it's a simple heuristic. Maybe it's a creative solution. Maybe the known off the shelf Machine learning algorithm. Occasionally it's a custom defining a new machine learning algorithm from the scratch. And so, from a frameworks software program's perspective, I believe whatever is the right tool to for the problem, what we have ended up using is anywhere from my perspective Pythons, Scala, from the framework perspective, anywhere from Spark and TensorFlow, also PyTorch occasionally. A lot of stuff that we do, the data and computation of the cloud, like that most internet services related companies. So having tools that could integrate with these in whether it be AWS or the Azure, the tools that are easily integrated with these cloud platforms we try and they can choose those. It depends on what we mean by tools If we talk about tools like IDE, I like Python a lot. I use PyCharm because it provides a lot of flexibility. There were days when people used to write codes in them. It was great and I do love them. There's a lot of advantages of using a modern IDE. If we're talking about IDE, PyCharm is one of my favorites

What things do you like about your job? Were there any pleasant surprises?

Based on experience at: Director, Data Science, Expedia Group
Summarized By: Jeff Musk on Thu Dec 19 2019
I think the one thing that I like a lot is the amount of time that I have in trying to define the future road map and future directions A lot of times, data learning scientists and machine learning practitioners. We're often walked down by a small scope of the problem. Maybe there's one model that we're building to predict the click-through rate of all the hotel listing that we show on our website. And I want to improve my accuracy from 95% to 96%. But, you know, maybe that's incrementally so much harder to do combat from maybe 80% to 90%. And that is great. And maybe a lot of people have tried to do because there's no one single model that can rule them all. Maybe I wanted to combine some of the models and try to gather that the strength from all these different models. Often data scientists would be bogged down by these little things. In my role as a director, I get to define the future road maps and, larger problem definitions that we want to go on the attack on. And I think that's one of the things that I really like about my current job. And, I personally enjoyed traveling. Ever, since I moved to the U. S., I have traveled quite a bit. And being in a company that can solve travel problems for millions and millions of people on a daily basis, that's quite a pleasant thing. I think it's a cliche when people say if your job becomes the fun, you don't work a day that sentence is fairly true.

What are the job titles of people you routinely work with inside and outside of your organization? What approaches do you find to be effective in working with them?

Based on experience at: Director, Data Science, Expedia Group
Summarized By: Jeff Musk on Thu Dec 19 2019
About 60% of the time I spent interacting with data scientists and machine learning practitioners both on my team and outside my team because they actively collaborate with other machine learners. Another about 20% of the time I end up talking to product managers. A lot of companies now have a product manager oversee how a specific product is being built. They become allies in between the engineering teams, the business teams. I'm interacting with the business teams, understanding their needs and directions and trying to define what are the next problems that you should resolve? What approach is working with them? One of my complaints a lot of time you go to meetings, there are in general are It's quite possible that you might walk into a meeting where there are a dozen people and there's no agenda. So one way of meeting to be very effective us too, along with the meeting, whoever meeting invite is being sent out to, to have a defined agenda. It could be three lines or paragraphs. But the point is, if you have an agenda then people can come prepared if people think that, the agenda doesn't match what they're doing, maybe they could drop out of it. it sends a message that I value your time and take this meeting seriously. So, specifically, in my current role, facilitating meetings and trying to get anything done, it's been really useful to set agendas ahead of the meeting. 

What are the recent developments in the field? How significant are these improvements over past work? What are their implications for future research & industry applications, if any?

Based on experience at: Director, Data Science, Expedia Group
Summarized By: Jeff Musk on Thu Dec 19 2019
There's a lot of sort of both industry and academic interest in mission learning, especially in the last 10 years or so more so than before. This is quite obvious if you see some of the machine learning conferences like NeurIPS and the size of this conference is the number of people who attend this conference is exploded quite a bit in the last few years. I was in NeurIPS recently last week, and I think they were over 14,000 entries this year which you know, it's quite a zoo at these conferences now. So there's no I think there were about 1400 papers that were published in NeurIPS. There's a lot of new areas people are interested in. Recently, there's a whole area of meta-learning that people have been publishing. If you have very few examples to learn from, there's a newish area of research called few short learning or zero short learning that there's been quite a buzz around. People have looked at GANs a lot for a lot of different NLP and vision-related problems. Deep learning, in general, has been taking off for the last 10 years or so. I think there's there's a lot of learnings for us, a travel company that looks at a variety of different kinds of data, into building different kinds of algorithms to learn from. I think as a lot of the machine learning tools are being Democratized, it helps people who previously did not have access to build machine learning solutions to be able to build them. One thing that we have to be careful about is, as things become democratized on, and more people have access to machine learning, it's it's equally important to make sure that the algorithms that people are being used in a fair and transparent fashion. In the recent past there's been increased focus on building fair, accountable,transparent and trustworthy machine learning. And I think you know, that's one thing that we have to be cognizant about when be build machine learning algorithms that touch people's lives day in, day out

What was the hiring process like for your job? What were the roles of people who interviewed you? What kind of questions were asked?

Based on experience at: Director, Data Science, Expedia Group
Summarized By: Jeff Musk on Thu Dec 19 2019
I think I could break this into two categories. One, the interview that I did for my IC position and my recent interview when I moved to the manager position. In the IC position, there's a phone interview followed by four or five technical interviews that are typically on-site. These interviews can range anywhere from coding exercises, whiteboard or online tools like Collabedit or CoderPad and other sorts of mission learning interview questions as well. There's usually one or so interviews that look at here is an open column how would you solve this problem? And there are some parts of the interview that go into depth of machine learning, either in the area of my expertise or other general areas to try and figure out how quickly I am able to think on my feet. This is a very typical data science interview process. The roles of the people who interviewed me were three people who were data scientist themselves. One hiring manager, who's also part of the interview. It's likely that there are other sort off elite teams like product manager, business team who would also like to be in the interview to try and figure out from outside the team perspective how does a candidate fit in. For the second interview there was a lot of, a long term vision kind of open problems as well, because as a director of the data sense group, I'm also responsible for defining directions rather than just build algorithms for tomorrow. And I think there were questions about that as well. So the people also remained the same except I had interviews from VP as well. 

What qualities does your team look for while hiring? How does your team interview candidates?

Based on experience at: Director, Data Science, Expedia Group
Summarized By: Jeff Musk on Thu Dec 19 2019
I think over the last couple of years, I've probably ended up doing 200 or so interviews, to hire candidates for the teams that I work here at Expedia and one of the qualities that the teams across the Expedia group in general, look for is how well these candidates try and fit into the culture that we defined here at Expedia and there are number of principles that Expedia group values. We look for these qualities in general when we're interviewing candidates. One of the pillars is a bias for action. And that's something that I try and look for in a candidate. The openness and being humble is another principle. While these are harder to measure, you know, these things are some of the things that we expect that the candidates bring in. But the questions are often focused around sort of mixture of coding questions and fundamental foundations of machine learning, as well as some sort of more focused on machine learning depth. Either In the idea of the students' expertise or often if they do really well, then we sort of move on to other areas as well. We've again typically done phone interviews to start with to try and pre-select the candidates and then five or so on-site interviews with the mixture of coding questions and machine learning.

What are different entry-level jobs and subsequent job pathways that can lead students to a position such as yours?

Based on experience at: Director, Data Science, Expedia Group
Summarized By: Jeff Musk on Thu Dec 19 2019
I think it largely depends on across the tech industry spectrum. It largely depends on you know where you want to start. I can specifically speak for sort of the data science are applied research perspective. The entry-level job for someone with masters to apply might be a junior data scientist position. Someone with PhD, uh, might get into one level above the junior data scientist position. Which might be if you think of a data science one, level one, PhD might be a level two.t The usual pathways across different companies and the Expedia as well, for the individual contributor path, look like this, you might have Data scientist level one, two three, Senior data scientist. Principal under distinguished, which would lead sort of all the way to the vice president level. The parallel path is the managerial path, which is a few people might do a switch from being an individual contributor. That part might look like a manager, Senior manager, directors, senior director, and Vice president. So again, this is really general across most of the tech companies, large companies. The typical career progression happens because of various reasons for how well you do in your job. Some of the companies also like their individual contributors and people who work on machine learning to be able to publish and disseminate their work to the broader community and being able to publish original work as well as advertise the work that you're doing on Blog posts such as a medium post. They certainly help the career advancement.

What were the responsibilities and decisions that you handled at work? Discuss weekly hours you spent in the office, for work travel, and working from home.

Based on experience at: Senior Data Scientist, Eastwind Networks
Summarized By: Jeff Musk on Thu Dec 19 2019
One thing that was different was that Eastwind thing was a start-up in Salt Lake City, Utah. We were 10 people company. So, there was one thing that was different was sort of have to have wear many hats. You go from talking to the engineers on the team to the people on the under sales and sales engineering team to understand what the customer needs are, and trying to define problems, different solutions to the problems that the customers might have. So being a start-up, the one additional thing that I had to do from all the things that I do today is to sort of wear multiple hats along with the data scientist role.

How did the program prepare you for your career? Think about faculty, resources, alumni, exposure & networking. What were the best parts?

Based on experience at: PhD, Computing, University of Utah
Summarized By: Jeff Musk on Thu Dec 19 2019
A lot of what happens in the PhD is how good is your relationship with your academic advisor. Often there are more than one advisors or mentor. But you know, there's a primary. There's always a primary person who oversees the work that you do. I think a strong relationship with the advisor for me was critical to me ending up where I am. My internship when I was in graduate school, Yahoo Labs, was something my academic advisor facilitated. And, you know, I'm grateful for that opportunity. And without looking at what problems that the industry might be solving it's hard to work on problems being siloed in academia, especially maybe 10 years ago when we had limited access to open-source data. The career fairs at Utah which were I think facilitated by the alumni association, if I'm not wrong, were excellent resources. We had a number of different companies from Google and Microsoft come on campus, and talk to candidates. That was a great opportunity to try and chat with people who were in the industry and be better prepared for from the interview perspective, as well as, be better prepared for a full-time position in the industry once I graduated. From a network perspective, I think that that's one place where, we could've done better, being not close to the big tech companies. A lot of that is changing in Utah now, but maybe 10 years ago, being not close to that that companies was one of the challenges in trying to network with the right people. Although being able to go to conferences, academy conferences was always a great opportunity because you had all the relevant people, people from the tech industry, come to these conferences and it was a great opportunity to network there.

Do you have any parting advice for students and professionals starting out in your field? What three mistakes they should avoid? What three things would help them the most?

Based on experience at: Director, Data Science, Expedia Group
Summarized By: Jeff Musk on Thu Dec 19 2019
My top advice would be especially to students who are preparing to be professionals, my top advice would be to be well prepared for interviews. You might be the best in the research field that you're working on. But often, in the industry, it's a lot more than that. So preparing for interviews, one thing that helps to prepare for interviews is to give mock interviews. If you have people in your network who are in the industry, then you know, reach out to them and ask them if they could give you mock interviews? Do coding exercises? Because often while you know there's only so much that coding interview can accomplish, it's critical for the hiring process. People write production code, robust production code in the industry day in and day out and it's critical for the hiding, manages to evaluate the ability of the students who are looking for full-time jobs to write code and sometimes coding exercises, given the limited amount of time can be a challenge in the preparation often is the key. The other advice that I would give is to sort of be generally knowledgeable about the breadth of the subject area that they're focused on, for example. I worked on a lot of machine learning problems and I take some time out every week to try and read some of the latest updates in the field, and this is particularly useful even, you know, just not just for the coffee chat on a conversation out on the water filter, but, you know, during interviews, it comes in quite handy. Three mistakes that students must avoid. I would say, the one being underprepared for interviews. Number two, at least for my interview perspective, I've seen some candidates who might end up talking a lot about what they have specifically done, instead of being focused on answering questions that were being asked in the interview. That can sometimes not work in your favor. Another mistake throughout the graduate school that you could avoid is to be only be focused on grades all the time. While grades are important, it is not all that the industry cares about. Jumping onto the three things that would help the students the most. One thing that is useful is to have industry exposure. Trying to find out if you could spend a summer interning at an Industry position. That exposes you not only to the problems that the industry might be interested, but also the work culture, and the large scale data that the industry might be using day in, day out. Another thing that might help students the most is to start being collaborative early on because everything that you might end up doing is a full-time employee in the tech industry is collaborative. So think of capturing projects. Think of a project that it may be working on, try and be collaborative because people might have different strengths. So as a team of three can certainly accomplish more than what three individuals can. And that's one thing that will definitely help along with good grades. Grades certainly help, but these are things that are probably better than just good grades on paper.