Walmart Technology Lead Machine Learning Engineer/Data Scientist at Walmart Technology
University of Illinois at Urbana-Champaign Doctor of Philosophy (PhD), Neuroscience
Current Time 0:00
/
Duration Time -:-
Progress: NaN%

How did you get to where you are today? What is your story? What incidents and experiences shaped your career path?

Summarized By: Jeff Musk on Tue Jan 28 2020
So, uh, I started my career. I guess you could say, doing an undergrad in computer science at Texas Tech. Um, I after leaving Texas second, I took a job as a software engineer at a small company that did automotive software. Uh, and I started my first day. It was a fairly terrible experience. I ended up lasting about three and 1/2 weeks and then quitting that job, which turned out to be one of the best decisions I ever made. So, uh, it was a little bit of a shovel for maybe four or five months after that, looking for what I wanted to do. Um, you know, normally you arranged your job, uh, over you graduate. So you know what? You're gonna go, Do you when you leave school? And so having that sort of swept off the table, it was a little tough, but ended up finding a really amazing job at a company called National Instruments. That does test and measurements. Azan embedded in vision systems engineer. So that was a great opportunity. And a company really took a lot of time to invest in their employees, so I had a lot of chances toe, learn and thio to grow my skills, both communication skills and technical skills. And I think really looking back, that choice to leave that first job was probably the most pivotal choice, but I've ever made in terms of both my personal life and my career, because tthe e job of national insurance required me to move to another city of Austin, Texas, from Houston, where the first job was. And basically everything great that's happened to me since then has been shaped by that by by the things went down there. So when I moved to Austin, I started looking for non profit opportunities, places to volunteer and found a great animal shelter in town that was treating a disease called Carbo. So started volunteer doing that in my free time. I ended up meeting, uh, who would become my wife, uh, doing that volunteer activity. So that's obviously pretty important to me. And, um, at the same time started delving into Data Analytics on behalf of the shelter. I think your science background. So that was terribly difficult. And that sort of set me off on a path thinking about data data science in a little bit of a different light in a more practical setting than what I was seeing in my actual day job, which was around that vision systems a TTE the same time I had always had a strong entrance interest in how brains working out humans function intelligence. Artificial intelligence is an area that I focused on during my undergrad. And so I was looking for opportunities to expand my knowledge. That area read some textbooks, uh, worked on just expanding. My sort of fundamental knowledge is have a great works. And ultimately, at some point my wife decided she wanted to go to that school. It had been a lifelong dream for her, so it wasn't a terribly hard decision, and we decided to apply to, oh, clapping schools. So I applied to competition Romero science programs and shaped like the best schools. We only applied the places that had both of those options on DSO. Ultimately, we both ended up at University of Illinois, and I started doing my PhD in neuroscience. There, with a focus on computational cognitive neuroscience is just a fancy way of saying, trying to write algorithms and math that reflects human behavior. Eso studying people who have what's called hippocampal damage of those folks. If you've ever seen the movie Memento, that's basically what happened. Campbell damage looks like when you have it very severely, you can't remember things, and it's It's like you're always living in the now, but you never actually create new, episodic memories. So I took a bit of an artificial intelligence player on all of that, and ultimately that led me to look for more data science oriented position so I could do a eye for eye left grad school. It was a little bit of a tough decision. Toe decided not to stay in academia. What ultimately sort of led to me making my choice. I knew I wanted to come back to Austin, so I limited my view what I was applying to for that there was a couple of real equal opportunities, even a lap that I was a trusted going to in Austin. But when it came to, uh, what I wanted my career to be going forward, I really wanted to grow my technical computational skills. I've been spending a lot of time focusing on behavior and other things like that, so I wanted to get back to writing code every day on. So I picked an opportunity that gave you the best chance to do those sorts of things. As I got in the Walmart, just the job I started after my PhD I found We work in my in my office for sort of internal customers. We don't work on consumer data or anything like that. So I found all sorts of interesting problems on really found. There is a huge opportunity to contribute so unique set of skills that I developing over over the last eight years of my career on. So now I need the team of data. Scientists here in the Austin office got a group of about eight people doing data science and really enjoying a ll the challenges and opportunities that come with working for such a big company. Uh, I'm also still sort of exploring some of the ideas related to my PhD and on the side. I get a lot of freedom to work on the projects that I think are interesting. So taking that opportunity to continue to explore the theoretical aspects of intelligence and memory and neuroscience that's my story.

What are the responsibilities and decisions that you handle at work? Discuss weekly hours you spend in the office, for work travel, and working from home.

Based on experience at: Lead Machine Learning Engineer/Data Scientist at Walmart Technology, Walmart Technology
Summarized By: Jeff Musk on Tue Jan 28 2020
I like to describe my primary job as applying vision to date a science project. So the way you learn data science is often by being handed a nice, clean, technical description of a problem. That technical description does not appear out of nowhere and often is very hard to take up. So you have to be on effective communicator with non technical people to sort of act as a translator for what their intentions are, what their objectives are, and then figuring out what data sources might necessary have access to those assembling all that together. And then finally, you get to end up with some sort of problem justification just where they are degree toe to come into play. And so it's all those soft skills are things that you pick up as you gain more experience, how to ask the right questions, the politics of some of those things. And then you construct on, I construct a vision for how a particular project should be executed in order to accomplish whatever thing is happening, and that includes all the way down to the details of how are you gonna beat in the modeling, how they're gonna deploy it to a cloud service of these. A lot of cloud tools in our office. What validation methods are you going to use? What you gonna do with all that goes wrong if it works the way you expect to on. And so all of all of those aspects I lead on any of the projects that come through the office, which also involves a lot of inter ship. You know, obviously I want to grow the people on my team so that they're able to do one day the things that I'm doing right now. And so that involves explaining why I'm making choices the way I am and really working through the rationale of not just solving a data science problem but solving a problem where that problem is usually very ambiguous, able to find at the beginning in terms of office hours and working sort of what my day looks like. Uh, I the culture here is one which really supports you working the way that works best for you. I've never been a morning person. I hate waking up early in the morning. Sometimes you still have to go to any day of meeting here or there. But on most days, I could get away with coming in at 10 30 or 10 and started my day a little bit late, which lets me dodge traffic. And so that's a bit of an optimization I get to do in my life. I don't have to deal with rush hour traffic in the morning on. Then I'll sometimes do the same on the way home, so I'll make you leave it for and work my last hour. Two of work at home, Um, getting to dodged in the five o'clock rush hour traffic. It's a lot of flexibility in terms of office hours. If I need to take a day off, I can do that. There's not a strict sort of mechanism through which I need Thio manage my time off, but it's all about getting the job done, and the more cedar you get, I think the more that's the pattern, So if you learn to be efficient in the management of your time, that means you get free time to do things like I still volunteer it. Austin pets alive and work on data science and research for them. So, um, it's, uh that would be how you typically described. My schedule is sort of start late into early work from home, a little bit as needed for for those sorts of things. In terms of travel, there are lots of travel opportunities. I tend to stay to the high priority travel, So I just got done doing a lot of travel. I want to double in to help starting office there, doing data science. I have to go to the Neurex conference back in December, recently, which was a wonderful conference experience and something that I encouraged to do if I have the time to do it and then sometimes a little smaller tricks to meet with cloud providers or other things like that to try and speck out project, so the travel is variable. I try to limit it to no more than 10 to 15% of my time spent traveling. I do work from home often once a week, sometimes once every other week, depending on what's needed for my job. But I think those air important days to take because when you're in the office, there's a lot of distractions. There's a lot of things misdirecting you from your programming or other holes. And so, uh, taking a work from home day once a week is something that's pretty encouraged in our office.

What tools (software programs, frameworks, models, algorithms, languages) do you use at work? Do you prefer certain tools more than the others? Why?

Based on experience at: Lead Machine Learning Engineer/Data Scientist at Walmart Technology, Walmart Technology
Summarized By: Jeff Musk on Tue Jan 28 2020
Yeah. So we for data science, we pretty much use python for everything. It depends on the exact task, but so I could learn totally critical for any basic modelling efforts. Algorithm wise, Random forest seemed to come back up repeatedly as being our simple bottles go, uh, one of the better classifier methods we do use auto metal framework. So I think that's becoming a a really nice way to sort of check if you build a hand designed model. How well are you doing? Really? It's a good benchmark toe face some of what you're doing on. And if you have access to Ben Cloud, compute resource is on a metal eyes very quick and easy to example. Train. So, uh, that's sort of the starting points in terms of In Terms of tools. Stack as the increasing complexity way do a lot of natural language processing in our office. So Houston six SpaceIy is a great library for natural language processing, different sorts of embedding methods. We really did try to stay in here that cutting edge on stuff like in beddings, and so we use also toe workout of Google and Facebook and another a place like that as well as trying to build custom models and things like tensorflow. Uh, very Schelotto encoders, etcetera as needed. That's our typical sort of tool sack. On the data side side, we do like deploying things and azure so as yours is where we tend to, a lot of our cloud computes, and my personal preference towards a lot of these things is basically the stuff I just described. I do think that there are times when you need some additional organization and will use things like Scala or even see uh, or, you know, seitan or things like that as needed for optimization. The reasoning behind Python probably pretty self explanatory if you're doing data science already, which is that that's where most of vegetables are. It's what became the de facto standard during the past 10 years or so and so if you want to find out tutorial, you find a library to do something, it's probably gonna be a python. Um, then there are some reasons you might want to go to something like C sharp, as I said, for optimization purposes or for scam little enterprise style code, and we'll do integrations as necessary for those other libraries. So when you're working with the software engineer, you may deliver them some python code that they find a way to integrate it command liner or via a service, arrest, FBI or something like that as needed for the different projects.have used H 20 pretty extensively. It is not easiest to work with of all of the autumn l frameworks, but it is definitely very scalable. And so when you're working on giant data sets that you can imagine with Walmart Scales is pretty critical. And so HBO has been one that we've consistently found success with. Autumn Ellis is an interesting choice for depending up, depending on the size of the data set you're working with and the level at which you're willing to be hands off with with things like validation of performance. There are some tools and g, C, p and an azure that do auto mellow. We tend to stay away from once that are highly coupled to the cloud provider, just in case we need to switch providers at some point for some reason on then for local auto mel on just your laptop or something. Auto S K lard is great. Teapot is great. Both of those libraries are ones that I found pretty good success with

What things do you like about your job? Were there any pleasant surprises?

Based on experience at: Lead Machine Learning Engineer/Data Scientist at Walmart Technology, Walmart Technology
Summarized By: Jeff Musk on Tue Jan 28 2020
really. I was pretty nervous starting thinking Giant company wasn't super clear on what the expectations were gonna be. We were a new office at the time, though, and so a lot of how I became sort of lead data scientists was that I realized by being knew it gave me an opportunity. Just take control what I wanted my job to look like. And so I was very open and honest. We communicated the things that I wanted and the things that I enjoyed doing and sort of built a personal brand around those things that I actually wanted out of my job, which includes telling people I don't wanna have morning meetings and communicating those sort of softer aspects of the job on the pleasant surprises that everyone's an unbelievably supportive of all of that. So, uh, there's there's way more freedom than I expected, and this is something that I think is very counter to folks coming from an academic background. But my experience in both of my more extended, uh since in in industry is that actually have a lot more freedom to try out the ideas. I want an industry that I did in academia the freak. The freedom seemed to be there because I had all the time in the world to spend on stuff. But I didn't have the resources or I had other pressures driving my decision making. And a lot of those pressures go away when you're in an industry, because they should if you're performing to the level that you need to. And so you know, I get no complaints about my performance. I'm giving a lot of freedom ticks for different things, like reinforcement, learning that I find to be a very fascinating fund set of techniques.

What are the job titles of people you routinely work with inside and outside of your organization? What approaches do you find to be effective in working with them?

Based on experience at: Lead Machine Learning Engineer/Data Scientist at Walmart Technology, Walmart Technology
Summarized By: Jeff Musk on Tue Jan 28 2020
engineers are probably the most important folks to us on in the data science space. They really help us get the data on, get access to things in a scalable way as well, right out what we're doing and communicating with them is really all about being clear on exactly what you need. Exactly what delivery able you expect to get soft engineers similar sort of thing. We often have to communicate what we have so that they can understand the best way to deliver it, either in the form of a front end. So front engine engineers will design the front of aspects of those. We don't make huge distinctions between front and back and engineers. People often like to call themselves full stack engineers. Uh, a lot of that is just is just semantics and B wants The reality is you're gonna be given some self programming related tasks. Having you ex user experience designers, I think it's great. I think every company needs for U ex people, because user experience often is not not the starting point for some of these projects, and I think it should be because ultimately it's the user who has to do something with a lot of these things. And then and then we have what are called product owners or product project managers on their role is to sort of coordinate all of these folks together. They're often nontechnical Andi. That means communicating with them involves really trying to stick away from jargon and try to stay with things like the liberals and timelines and what you need exactly being very specific and precise in your communications about those things. There's a mother more, uh, rare titles, things like cloud engineering, stuff like that. But I think the ones that I just described, the key ones

What major challenges do you face in your job and how do you handle them? Can you discuss a few accomplishments?

Based on experience at: Lead Machine Learning Engineer/Data Scientist at Walmart Technology, Walmart Technology
Summarized By: Jeff Musk on Tue Jan 28 2020
So I think the two biggest challenges are ambiguity. So having people come to you and they want something, but they don't know how to describe it to you. They don't know the state of the art in the field or what's possible. And so there's a huge amount of ambiguity where you have to be comfortable making choices and testing out your resolutions to that ambiguity, uh, yourself. Um, the other really big issue, which is related to that is thehe bility to access the data that will actually answer the question at hand. Oftentimes that data access is the slowest part of the project, and as a result, it's very easy to take that element of ambiguity, which is how to write, get the data and how to use it to my advantage and use it as a reason toe idol and not move forward on particular aspects of the project. The reality on any project is that there's gonna be some public data set or you could generate artificial data, or there's gonna be some some preparatory work that you could do such that when that data comes in, you can be prepared with your own little tool set that you've built for yourself and that you're ready to go on that you aren't. Then starting from scratch the moment that David gets it, I see it is a common mistake and younger, less experienced data scientists that they think the date is the starting point. The data eyes just the transition from it being a problem that you're trying to define and understand to being an implementation that you're trying to develop on DSO we have, ah, big application here, the office that we've done that's around natural language processing. When it was first presented to me, there was potato. And so I spent a while figuring out what the best techniques might be and constructing a sort of schematic design for a feature vector that I thought was gonna be a particularly good encoding of some of the information we might be interested in on. Then, when the data did start to come in and we started to get labels from from users and things like that, I was pleasantly surprised to see you that it only took a couple 1000 of these labels and we were already getting F word scores. Well, about nine and the results were looking very promising and most of the models, So all that preparatory work meant that the date actually got there. It was just a simple matter of hitting run, and most of the work was already done. And so I think that's a really critical challenge that everyone's gonna face on that you you can overcome by simply thinking carefully about your problem. But be prepared for what the data is available.

What are the recent developments in the field? How significant are these improvements over past work? What are their implications for future research & industry applications, if any?

Based on experience at: Lead Machine Learning Engineer/Data Scientist at Walmart Technology, Walmart Technology
Summarized By: Jeff Musk on Tue Jan 28 2020
game A talk on this recently to the office after that Neurex conference. And I think one of the most significance recent developments in a lot of this modeling is attention based models or attention mechanisms within models. So in neural networks, neural attention allows for what you can think of a sort of a dynamic mapping of particular features within different layers, maybe the input layer, maybe somewhere else in your network. It's often used in sequence to sequence models, but I think the realization is occurring. That attention as a mechanism is much more general and much more useful in a wide set of areas. Uh, there are also a lot of really interesting questions when you have an attention mechanism about interpret ability and what you could sort of get out of that attention, uh, components of the model to help explain in some way some of the choices that were being made by the model. I think the combination of techniques around attention based methods and some of the more recent developments in artificial neural networks and deep learning there in a CZ well, is how all those things apply to the explained ability of models are going to be some of the more significant things that we're gonna be looking at. A lot of industry applications. Folks will start off saying they're very comfortable having a black box model, and then the moment you deliver it to them, they'll use it. And then a couple weeks later, they'll come back and start asking you why. It's stupid things it's doing. Um, and I think that's a very natural feeling tohave. When you're working in conjunction with an A I system that is making decisions to try to help you accelerate your job, you want to know how it's doing matter. Wide Street Future research doesn't necessarily need to focus on those industry elements, though. I think future research should really be focusing on how some of these mechanisms can go, uh, towards more advanced styles of computation like composition ality or, When I was in neuroscience, we called It's for Relational, an associative memory because we know that the ability to compose elements on the fly in a dynamic way is really credible to creative thinking and humans. There's a lot of really cool studies and evidence in that space, and so none of our models do this very well. And I think when we're looking at ways to make more general systems or systems which transfer between tasks before easily the ability to do things like composition, ality, perhaps be an attention mechanism or something else is gonna be awesome. Incredible.

What was the hiring process like for your job? What were the roles of people who interviewed you? What kind of questions were asked?

Based on experience at: Lead Machine Learning Engineer/Data Scientist at Walmart Technology, Walmart Technology
Summarized By: Jeff Musk on Tue Jan 28 2020
they interviewed with, I believe 34 separate folks. Three of them were technical energies, which was more than any of the other job interviews that I did. The questions ranged from really basic things. Like how? Okay, you built a binary classifier. How are you gonna evaluate your performance? Um, too much more complicated questions. Like, what are, uh what is the actual role and contribution of skin connections or of of other specific components with the neural networks? It's been a while since I've interviewed, so I don't remember many of the specific questions, but they ranged in level of abstraction and a level of challenge, and some of them were more software based. So let's say someone presents you with this problem. How would you approach it on what software components do you think you're gonna need to build? What tools which you use? Uh, while others were much for theoretical and esoteric. There was one interview which was more what I would scraps behavioral or cultural interview, which was about how I prefer to work and for what my history is and what I like to do. Uh, in terms of particular tasks at work and You know, I'm forced. Mentioned all my non profit work that I've been doing and those sorts of things I think are really great contributions to an interview. Like get to know the person. I think we all work best when we're happy with what we're doing. And so, uh, making clears other folks. What makes you happy is a really nice way to make sure you end up in the job that you actually want. Yeah.

What qualities does your team look for while hiring? How does your team interview candidates?

Based on experience at: Lead Machine Learning Engineer/Data Scientist at Walmart Technology, Walmart Technology
Summarized By: Jeff Musk on Tue Jan 28 2020
there's sort of a baseline set of competencies that we look for, You know, you have to be able to use the tool sets that are common and data science We don't necessarily, but a huge emphasis on your deep understanding of a particular model. So I mentioned earlier that maybe Random Forest is my favorite thing to use as a starting point in some of these classification problems. Uh, and I ask you a bunch of questions about running for us If it becomes clear to me very quickly that you don't know a lot about Random Force beyond what they are, um, that's not necessarily gonna be a deal breaker and interview. What matters is that you know what you know, and then you also know where your gaps are and you're interested in learning those elements that are gonna be relevant to your job. But you don't know. So someone who comes in and is maybe less knowledgeable about a lot of the details of the models but shows a real passion to learn those things is going to be seeing much more favorably than someone who maybe those a good amounts but has no interest in learning more, so that's a huge, huge proponent of it. Another thing that we really look for is this issue of ambiguity, the ability to deal with sort of ill pose problems and questions in a productive way. Some of the questions that will ask will be a intentional sort of vague statements where the goal is to hear how you handle you know of a problem, because that's something that you're gonna really deal with in the real world. So we do have these technical interviews that I'll often do sometimes of the other senior folks. So the team will do. And then we tend to have some sort of once again behavioral or cultural interview to make sure that you're gonna be happy in the position and that the way that we work in the office is gonna be something that you enjoy, which I think personally, most people would do really well in a situation where they've given a lot of freedom. It can be a challenge if you respond better to a very rigid, structured objectives and environments and being told what to do sort of at every step of this environment may not be the right one for you because there's there's a whole lot of freedom. And so some of those things behaviorally around self determination at the ability to make decisions about what is the best use of your time. Uh, those are critical things will look for

What are different entry-level jobs and subsequent job pathways that can lead students to a position such as yours?

Based on experience at: Lead Machine Learning Engineer/Data Scientist at Walmart Technology, Walmart Technology
Summarized By: Jeff Musk on Tue Jan 28 2020
so entry level positions doing data science with another company. You work for a year or two in another company that that experience and that list of projects that you've performed, um, the list of projects is really the critical bit. So if there's if there's two candidates, one that worked for a company for four years and did one project for that company, and that was it. And even if successful, I don't necessarily think that is as good of a reflection as someone. Maybe who hasn't worked in data science before but has a huge list of data science projects. But they've done on the side, maybe for a non profit, like what I like to do, or maybe just for fun things that they enjoy trying to build stuff around. So it's really that that the administration of the skills and capabilities that matters more than the job history or or the particular job path. So I do think folks in data science that have computer science, knowledge and backgrounds tins, you have an easier time coping with some of the challenges because you can think of optimization scale software architectures. You can build components that may be a data science without as much software skills and a special computer science knowledge I wouldn't be able to do. And so it's really it's, ah, more of a thinking in terms of skills, projects, capabilities that have been demonstrated then in the particular job history in question. And that and that would be for you no entry level data science positions here. The more senior you get in terms of being like a senior or staff data scientists more those requirements become a little more challenging.

How did the program prepare you for your career? Think about faculty, resources, alumni, exposure & networking. What were the best parts?

Based on experience at: Doctor of Philosophy (PhD), Neuroscience, University of Illinois at Urbana-Champaign
Summarized By: Jeff Musk on Tue Jan 28 2020
doing research is a very different sort of self determined process than what you're going to deal with an industry industry. The objectives are often extrinsic, that coming from some stakeholder or from your boss. But the abilities that get in doing research at the university and in having Thio ask questions no one has ever asked before and then go answer them. That skill said, is really at the core of everything that you do in any job, and you're confronted with it very directly and doing a PhD, you're confronted with ambiguity. You're confronted with the fact that you are being required to ask questions that are. The answers are not known. I think the best part about it is when you actually get to be the first person to know something like that. It's a rare feeling because you often are so deep in what you're exploring, and whatever your particular topic and research is that you don't even think about it in terms of being something do. It's just some elements of the puzzle you're looking at, but when you finally get to that point of publishing a paper, there's a sort of sense of I've now contributed to the world Now added knowledge where there wasn't some and you get to start becoming part of a conversation where that conversation has been there all along. But you now have all the context you need to really understand what's happening and even being here at Wal Mart. I still keep up with the field and getting to still have a part in that conversation about you know what? How do people do creative thinking? How do people solve problems that involve arbitrary relational binding, which is something none of our A I do well at this point, that conversation is really exciting, and it's really it's really fun to get to imagine the world as it will be as we continue to answer these questions and that ability to resolve the ambiguity that comes from doing research is truly invaluable. And I actually think it goes both ways. I finished my PhD much faster than the program average, and I think it's because I worked before I went to the PhD, so I learned some skills from industry that I I took thio being more successful in academia and then I think this academic skills also make me more successful in industryTexas Sex CS program. I think the most interesting components for the electives elective program elective courses are obviously a thing in every undergrad situation. I took one of neural networks. This was long enough ago that I actually went to the math department at one point and asked about artificial neural networks, and they said, Oh, we've already proven those aren't useful You should look a support vector machines and that was a legitimate opinion at the time. But I took a graduate level elective in neural networks, so learn a lot of the basic map behind it. And, uh, the mentorship that I got through doing that, I think, gave me the confidence that I could understand these topics and that I could actually dig into them. I didn't participate in research for as much as I really wanted to. It should have at that stage in my career, I was much more focused on application. So I picked up projects, uh, working on things like I built a really simple computer vision program for the Vietnam War Archive that was there that just processed binary codes on on images and produced what the decoded version of it was a really simple things like that. One thing that I didn't anticipate as I was doing undergrad that has come up a lot is how much some of the basic computational complexity theory really doesn't go away. It's always there, sitting behind everything you do, bigger notation and the ability to think in terms of the memory in the space, the space and time constraints of what you're working in. If anything becomes more important, the farther along you go in your career. And so some of those things that seem somewhat esoteric as I was taking them in undergrad, I'm glad I paid enough attention that when they came back up, I was able to know howto to go really learned something that I needed to learn.