Womply Director of Analytics and Data Science
Current Time 0:00
/
Duration Time -:-
Progress: NaN%

How did you get to where you are today? What is your story? What incidents and experiences shaped your career path?

Summarized By: Jeff Musk on Thu Nov 28 2019
Happy to share my story. So I didn't get where I am today. It was a different path. So at the David Eccles school business, I did a major in both finance and information systems. And so when I first went to the U, I was pretty sure that I want to go into investment banking. So I joined the student investment club and then there was actually a professor at the U who for my undergrad I took an IS class and it always kind of liked computers of liked technology analysis. And so that really started to pique my interest on. So then that's when I chose to do a major and information systems. But right out of college, I first went to Goldman Sachs, and I joined their team on I spent about four and 1/2 years there. At Goldman, they kind of put people in the role, and you have no idea, Really, like what department specifically will be in, but you try and like, kind of find your place based upon your skillset. And so I quickly found myself in a skillset where started, like, very basic started developing access. The database is because they wouldn't allow us access to production on the technology side. So developed these access databases and actually groom into programs that the company could use to become a lot more efficient on. So this was in their assets servicing division. Fast forward, two years later, they started developing this thing called Strat CE, which later became known as data science. So we did kind of we're the cornerstone of how analytics from an operational perspective, took light of Goldman Sachs s O. I spent about four and 1/2 years there and then quickly realized I wanted to progress my career more on towards hardcore analytics and data science, enjoying the company where I could really move the needle. So then I found myself at least my own company. I called Crest Financial and I got really heavy into data science. And, of course, when I was at the U. David science wasn't a term that was used, and this is no back in 2011 when I graduated. That's forward. A few years that start to become like this term that's being thrown around a lot. Especially his companies start to gather more data. They need to do something with it. And so I became the data scientist at this least own coming across financial. And I was in charge of developing and underwriting the crust, credit risk underwriting model. I had never done data science before. I had never done hardcore statistics on, and so I did a lot of online certifications but used my foundational knowledge that I learned that you in terms of database management, thinking about, like relationships, thinking about how to use some of my strategic management courses. And so it was kind of where today the kind of call you know, a pure data scientist, a unicorn. So it's somebody that has great knowledge and turned in terms of subject matter expertise. You know a lot about statistics, and you also have a good understanding of being able to write code. Call a python are sequel. And so I kind of found myself in that position of where I had a very good grasp on all three of those there was another person of the company that joins that was purely academic, that a PhD in statistics. But they had not spent a lot of time on the business side and they weren't able to connect the dots and really kind of march forward developing this algorithm. There's really supposed to do what the company wanted to do. And so I then found myself in this position. I was like, wow, I have a skill set of where I can really add value. So I spent a couple years of dark company and then said, Okay, I really want to drive like the full analytics architecture for a company and I wanna be able to move the needle because that's kind of been my career path. I want to build things and really change the evolution of a company to succeed. So, then I moved on to a company called Womply. It was headquartered out in San Francisco. I'm currently there, director of Analytics and Data Science. And when I first joined the company, I just joined as analytics Man injured. quickly joined the company. They had done about 50 million and fundraising and their main goal was to service small to medium-sized businesses. So this is a very different space, right? I went from banking to assess, which is a completely different type of business and so that was a little bit of a challenging with itself. But I quickly found myself with my skull so that I found in my other company to be able to shape their foundation, and I came in and I said, You don't have the right talent. You don't have the right structure. I've been here and seen this before at other companies. This is how we should be thinking about solving these problems. And so will team up. I got promoted pretty quickly to senior manager and then pretty quickly after that, got promoted to director of both analytics and data science. So, yeah, that's kind of the story of how it got there. Obviously a different path. One between banking and sass. And then, yeah, What incidents are experiences shaped my career path? I think I kind of talked through a couple of those in terms of just seen, and a clear need for something Where there's not a person or a team being able to fill that need. I think it was about me raising my hand to say. You need this not being afraid to, like, push back with my opinion That has always been something throughout my career. That's it's never comfortable right to dispute what kind of the status quo is and to be disruptive on, disruptive in a way that builds the company and propels it forward in the right direction, not disruptive from the aspect of causing chaos. And so I always had a very firm footing on where I stood and why I thought we needed to do certain things and was backed with research. It was backed with what we were going to do from a company perspective and how this would benefit the company and propelled forward. And so I think through some of those things it just started to operate and that, once I started to see that work, and it actually did help the company, and there was enough trust builds up that's ultimately like how there became a pretty quick progression to where I am today.

What are the responsibilities and decisions that you handle at work? Discuss weekly hours you spend in the office, for work travel, and working from home.

Based on experience at: Director of Analytics and Data Science, Womply
Summarized By: Jeff Musk on Thu Nov 28 2019
Great question. So I'm gonna start with the second out of the question that we'll move into the responsibilities. Womply has a headquarters in San Francisco, and then they also have an office in Lehigh. Our office in the eye is the largest one. A lot of our sales team Customer support, success finance, analytics, corporate marketing, HR people ops, Those teams set in early high office. And then, minutely, our engineers sit in the same risk office. So there's a lot of cross work collaboration between the offices. So a lot of what we do is via just Google hangouts with Crawford, conference room, set up cameras and video monitors. So that's a good way to have crossed collaboration with that said I typically at my level, and even when I was senior manager, typically travel out to Sam was just go about once every couple of months just to check in with the team. I follow up on projects. I find that it's a lot easier to a whiteboard and get like big projects off the ground when you're kind of there in person. In terms of running the business, moving small things along. You can kind of do that remotely. It is challenging, but it can't be done. In terms of working from home, I definitely take full advantage of that. Even though I am at the level I am, I do still typically spend probably 2 to 3 mornings from home and every other week at least a full day from home and so our company culture is kind of about I. It's about getting to results. It's not about face time. It's not about how hard you work, how long you work, it's about getting to results. And so our team, funny enough, is really the team that kind of built that trust for our company culture to allow people that flexibility. When I first joined, it was completely taboo to work from home, but I definitely pushed the envelope because of my prior company. I work from home a lot because I was definitely down in the weeds to bust out of the project. I needed no distractions. But that said what that said, there is kind of a fine line to play because when you do become a manager, you need to make sure that you are there for people to unblock them. You're there to make sure that they feel supported. And you're there to lead by example on. When you do have people in the office, you do have to do that. So a typical work week for me is probably about calling 80 hours. That's not by demand. That is by choice. And that's about being empowered and one into at our M E. Net. A startup is about really moving the needle forward. And it's hard work's wear multiple hats. And so a typical day for me, it looks like I wake up at eight. I hop on my computer. I'm working until about noon. awesome. Quick, quick lunch. Drive into the office. Stick there until about 6 30 Come home. I logged back on my computer around 9 30 and sometimes I'm working. It's all 12 sometimes till two. Interestingly enough, none of this feels like a job, which is obviously a really cool place to be. There are definitely days where you're like, Oh, man, this is you know, it could be rough, right? They're gonna be disagreements there could be projects that are stalling that are hard to get through. But at the end of the day, the things that I do work on, I find very intriguing, very interesting. And I like seeing things push forward. So that is definitely a good thing. That does not always happen at every company and in every role. But when you do, Phil, when you do find the thing that you're most passionate about that definitely I can definitely test does not feel like work. In terms of my responsibilities at the company right now and the type of things that I handle. Eso I manage both our analytics arm and our data science arm and our companies that interesting points where we're starting to expand very heavily into the data space on. And so, from an analytics perspective, it's really full scale. We are a central analytics team. and so we have an analytics manager and then we have analysts. The report to him, their main responsibilities are for basically being They have AI developers, so they will actually develop the data models on top of our data warehouse to report on the business that will develop visualizations through demo or tableau that could be used throughout the company. They will then run occasional statistical analysis tests where we help, like, call it our marketing team, do an AP test on a particular like maybe a direct mailer. Or maybe it's some type of advertisement that we're gonna put on to Facebook or Google. We'll help them with those types of tests. Additionally, we'll look it like, How is our product working us? We'll do various tests to see if our product actually delivering on the value that we are promising our customers on. Luckily for us, we have hundreds of thousands of merchants that we work with. They were able to kind of have a control group by defaults on, and then our test is really our customers, which is kind of nice place to be S O. That's kind of the scope of the analytics team. with that said, because we are a startup, they do wear multiple hats on that side. So we do things. They're more along the lines of, like, data operations, so call it will build some of the pipelines that exist to call it. Somebody comes onto your website, you want to create a lead, you on yourselves, team to call them. We build a pipeline to capture that data, put it into a CRM tool and then work the mechanics to make sure that somebody is actually going to call them. Another aspect to is we use a tool called Segments, which is essentially a data bridge on within that we own the entire architectures. That's capturing all the metadata about people that land on your website and then how they actually then transition to using your products so you can understand, like feature importance of product usage, the stickiness factor of like what things driving the user back into your product. Eso would use some of that Web and Product Analytics is Well,then on the data science side, that's kind of, new fund space that we're starting to build-outs, what we're doing there is on one of the responsibilities are essentially developing these new ML models that will do fuzzy matching, within our product. We have a lot of credit card readers, a lot of merchant data, so we're building consumer profiles, business profiles, understanding consumer behavior spend across various merchants, and so there's a lot of analysis that goes into that. There's also a good chunk of ML that goes into matching Call it hundreds of thousands to millions of businesses or transactions, and you can't have somebody sitting there going like, Yep, I'm gonna do this quick match. Okay. This address generally looks like this looks good, right? You can start out that way, but in order to scale A, you have to have a model that starts to be able to recognize that pattern. And so that's been mainly with that team's gonna be focused on, in terms of, like, general responsibilities. You know, the typical type of things so weakly. Wanna ones with all my direct reports. I handle all of our team meetings across my organization. I work across collaboratively with other leaders of the company, mainly at the VC level, about how can analytics help them either answer questions or push the business forward into new endeavors of where we've identified the nerd, like at an opportunity, Or how can we increase our overall revenue? How can we become more efficient? Which therefore increases our revenue by reducing our costs, and then I handle all of our budgeting and forecasting for our team. It's not like that.

What tools (software programs, frameworks, models, algorithms, languages) do you use at work? Do you prefer certain tools more than the others? Why?

Based on experience at: Director of Analytics and Data Science, Womply
Summarized By: Jeff Musk on Thu Nov 28 2019
I was a Microsoft sql server guy for a really long time. I'm in. So I knew t-sql. Really? Well, it wasn't until I joined Womply that they started using a tool called Snowflake, which is essentially a cloud data warehouse. It's built on AWS. Snowflake has blown up over the past couple of years, and so now, by far, that is my favorite tool to use. In fact, I've spoken for them at a couple of their events in Salt Lake on I'm definitely a power user of theirs. The nice thing about it is, you're able to run a query and you can dedicate these warehouses or chair essentially cluster compute cores to process on your query so you could have called it like an etl warehouse where you're putting all of your retail pipelines and work through that. Then you can have something that life your analytics team is using. You're able to spend up another warehouse that has its own computer cores to run the jobs. Neither of these compete with each other. So when you talk about like bandwidth, you start to reduce it that way. Additionally, on the fly, you can scale those warehouses up in, up and down, based upon how much work you wanna put through him. So call like you have a query that, you're looking to go through like trillions of rows of data. And it's a very complex query. You can just easily like, click and drag, scale it up to, like, 24 compute cores. Your results back super quickly. You have to change out any hardware where is, like on Prem. Forget about it. That's not a possibility. So there. That's one of the reasons they have blown up. But the entirety of our analytics architecture is built on snowflake. So that's when it comes to data in storage and retrieval. One of the best fools, when it comes to machine learning, I've played with a couple of tools that are more driven, some more driven towards like data science. Have the user's some towards like business analysts? One of the tools that have used historically is called data row by, and it was very quick to use in terms of like, I don't want the best model out there, but I also don't want to spend a lot of time like developing this. They also had an a p I Which of my prior company was super useful, right? You didn't have to write any code other than give a Jason payload to this. A p I you get the probability returned back. So we talk about, like, credit underwriting. Super valuable, very low cost that does not prove to be super useful, depending upon your level of accuracy and house clean your data. Is eso like, with any type of auto ml platform? It's like garbage in garbage out. Almost 95% of the work you do is about cleaning up that data set and making sure that yeah, you've removed a lot of the bias in your data. There's likely always going to be some bias, but at the point, we're at WAM play. To be honest with you use right now is a combination of both python and are and just running through the standard algorithms that exist because a lot of our time was spent on that pretty processing. We don't need anything fancy, or quick through an AP I front. So, R is definitely my preferred tool personally, that's also used on our team alive. because a lot of what we do is statistics-driven all the packages and are geared towards statistics when it comes to, how efficiently does a script run? R is really not that great, python is a heck of a lot better if you're trying to run something quickly. what That said there are a lot more packages to install. It can be a little bit more complex to get to, but we definitely do use python for ah lot of r E T l processes in terms of like getting data out of our core production databases, which are already s pushing it into snowflake. That's all driven through python. A lot of our data manipulations that we can't do in Snowflake through a typical sqll Query. Well, then, who's Python? Push that into an M R. Instance in AWS using spark and then pull the results back through time? so there's like, definitely necessary things. But I'd say of like the if I could pick through tools that are the cornerstone of what we do. It's our snowflake and python

What things do you like about your job? Were there any pleasant surprises?

Based on experience at: Director of Analytics and Data Science, Womply
Summarized By: Jeff Musk on Thu Nov 28 2019
So say the thing I like most about my job is the ability to build something from close to nothing. You won't find that in. At least in the exact experience, I have right now, you won't always run into that opportunity when you do, you run with it and you don't look of it. But there are instances of where, like, you can find your niche within a company of this area of space that you fit well into. And you're also passionate about, and so I think it's about identifying those types of things and finding a way for to plug yourself in by working with, you know, call it the VP, the CEO, You know, depending upon will level, you're at of how to play into that. So, what I like about my job was not simply handed to me is kind of the point I'm trying to make here. I had to clearly that find what I wanted to do. And then I was allowed to run with it. So the reason I like my job today is because I fought for that. Right the just working with data is innate Lee. Interesting, I definitely have the personality trait of where I'm a little bit of a skeptic, which is always necessary. When you're thinking about data, you constantly have to question it. You can't just look at it as a result and go Oh, yeah. The data said that You know, we made a $200,000 in revenue last month. You have to kind of see if that passes the sniff test in the after question, it's You have to go. Well, where did that come from? why? Why does that make sense? nd so there's that innate questioning nature that exists that I definitely like And then the ability to, like build up others right and kind of the data space. It's kind of cool to be at this in this space where you're kind of at the you're leading the cutting edge of what's being worked on in the industry. And so when we think about, like, data science were right there on like what a couple of years ago is called, the sexiest job out there, right? I forget the exact article, but, it's cool to kind of be in that space and be working with companies like Google and Facebook and being in the space of just, like, targeted marketing. there are tools that we use today that if I was to just, like, drop a couple of names to people, that have not kept up the speed and the text face and go. What are any of those, like, what is segment? What is next panel? What's amplitude? There's so many tools out there now, but it's kind of cool to be at the forefront of this to just kind of be in the know, in terms of, like, pleasant surprises. I would say that. Call it five years ago. I never thought that I would be in the role I am today. I always kind of thought about career trajectory in a larger company that it would take a very long time to get to the space I wanted to be at. there's also a very fine line about, jumping between companies. And so it wasn't until after I left Goldman, which is a very large company, that I started to see that things move very differently at smaller companies. The ability to move up quicker does exist. When you do move to different jobs, you can move up quicker in terms of both compensation and also title. That said, it can also be You can't get ahead of yourself, right? You can jump into a position that you're absolutely not ready for. That didn't happen. In my instance, I'm sure I was not the number one expert and everything, but that was kind of known ahead of time. But I say the pleasant surprises leaving kind of the banking industry and joining the text software startup industry was a very pleasant surprise to see just kind of what I could do. And it really opened my eyes off where my career could actually go. So I think any advice is never to stay at one company in your entire life. You definitely have to go get a feel of just kind of like what exists out there and what works best for you.

What are the job titles of people you routinely work with inside and outside of your organization? What approaches do you find to be effective in working with them?

Based on experience at: Director of Analytics and Data Science, Womply
Summarized By: Jeff Musk on Thu Nov 28 2019
So I have worked with the CEO, CFO CEO, anybody in the sea, sweets all the way down to, like, the very junior type of people. So it's literally all levels of the organization. So, like specific titles like Lower Down, marketing manager, customer success representative sales development representative like those air the most junior levels at our company. And then you start getting up to, like, VP of growth, director of operations, director of people ops, vice president, brands marketing. So, really, from an analytics perspective, I in my teamwork almost with every single person at the company. At least from an entitled perspective. Not literally every person, but, certainly from, like, director up. We're working with every single person. Outside of the organization, there's limited scope, but for our business, it's about working with investors. It could be people that put money into our company could be working indirectly with board members. It also involves a little bit of working with third counterparties, and it's traditional people that are VP to CEOs. So if we think about like, working with another company from a partnership perspective, it's most certainly working with their CEO. When it comes to like getting tools and demoing tools for our company, then you're typically working with either, like a more senior director, executive director of, like their accounts down to, like, something of an account executive. In terms of approaches of working specifically people inside of the company, communication is number one. Without communication, you're set up the bill. You can almost never communicate enough. And I say that, but that doesn't mean like, every one minute you provide somebody in updates, but it's more of do people know what they are? What they should expect from you. So call it under HVT of gross If I put myself in his shoes, Do I know specifically what the analytics team is going to deliver and when they're going to deliver it? If he can answer that question than I have felt in my job, you always want to make sure you're kind of answering questions ahead of time. So if you think the like, a project's going to drop and you have it in your head, you should immediately either bubble that up to the manager of your team or directly to the person that's a stakeholder and the key person that's going to, be involved with that projects. In terms of the deliveries, I think communication over communication is fundamental. That goes hand in hand with setting of expectations. So when you say that, let's say you know, the CEO of the company comes to me and he says, I need these 10 things. If I have two people on my team and they're already fully booked, I can't say yes, there is the There's the yes, there's the maybe And then there's the yes. there's also definitely a lot of no, and not being able to say no can set you up for failure. So you have to be very comfortable saying no, but there has to be a reason why you're saying no, in some instances, it devout. No, because we're working on these other five things that are way more important than these other two things you just asked for. So we're gonna work on these five things. There's a lot about kind of that accountability level. there's the Yes, If, yes, if I can complete these other projects, and so that's kind of the level of, like expectations setting that is also very critical. And that's, frankly, every level of the organization. If I'm working with, like an I C, I want to be just as respectful of their time as I do of the VP. You should treat everybody is if like their thing is important. But with that said, if an icy athlete and when I use the word I see that's an individual contributor, if an icy ask you specifically for something and you like the manager for team and then of the P dies, they might not have the same weight in terms of overall importance of the company. And so you kinda have to, like, balance that as well. yeah, what else? Let's say that those were probably the most important things. There are other things that you should do. but at least that if you do nothing else, communication and expectations setting are critical.

What major challenges do you face in your job and how do you handle them? Can you discuss a few accomplishments?

Based on experience at: Director of Analytics and Data Science, Womply
Summarized By: Jeff Musk on Thu Nov 28 2019
So being in a start-up, there's a lot about things. Move very quickly on that. Your company trajectory might change a month from now, right? It's a lot about failing fast. Some of the challenges I face is specifically like building out a road map of what I want to do where I think the company is going to go, How I kind of build the resource is and build my mental headspace around how we're going to get there when they when call it a couple of months later, things completely shift and get turned on their side. It's definitely challenging. To let go of everything and then to say whatever we kind of plan where it is dumping. I'm not married to this. It's gone. Now We're going in this other direction. It's not easy. To consistently pivot like that typically will not happen in a more established larger company. But certainly, in a start-up, you have to be comfortable with, with change. Ambiguity, and consistent, quick change. So those are definitely challenges. But the way that I handled them, one of them is I am comfortable to a degree with change. I'm also comfortable with a little bit of chaos. And for me, it wasn't something that originally just kind of fell in place. And that's just who I was as a person. It was about understanding, like, Why is this happening? Why were we doing these things in particular? Once I was able to establish the knowledge and connect the dots of why we would make some of these changes, it was a lot easier to digest. Yep, that makes sense. That's reasonable. I can get behind this new thing. I understand why we're going this direction. If I didn't know that, then it would just feel like this consistent whiplash of like, what's happening? I don't understand. So that's typically how you get around those big challenges. And I think that's not specific to my job, but specific that startups, a specific challenge to me that is unique is, working with the amount of data that we have. There's no other company in the world that has the data asset that our company has, and it's about doing something that nobody has ever done before. It's definitely challenging to wrap your head around this super nebulous thing. And how you want to build it into a product and how you want to make revenue off of it and how you can build a road map of where you're going to do with it. It's extremely hard work that requires a lot of dedication. but how do I handle this type of challenge? because I'm driven to, like, see it through. I want to see it succeed. I'm also super excited to work on something that no other company has access to. So just from like that degree, it feels like a little bit exclusive, right? It feels like you're doing something that could make a big impact for both your company and for other companies in the future. So those were a couple of challenges. In regard to accomplishments, so if I think back not to this job, if I think back, just like throughout my career, kind of stepping stones. One of the clear accomplishments, when I was at Goldman, was about coming in and quickly being known as the person to go to. And with that, I built a lot of trust with all the divisional leaders in addition to people on the team. And so when I started, like getting access to develop these programs for people to use, it ultimately led to people adopting to use them in addition to literally saving, the company about five people's worth of time. Over the course of the entire year and it required, like I don't know, one or two months of my time, I was like, Wow, this is actually really cool eyes able to automate all this manual stuff people were doing and to give you like a flavor of that literally had people there were right-clicking and then left-clicking each row in a platform and doing that 10,000 times a day. And it's like you cannot do this type of stuff in your sleep on, like just recognizing that was really cool. That was like my first cool accomplishment where I was like, Wow, this is that's cool that I could do this just straight coming out of college. The next thing occurred when I kind of got into the data science space and became this self-taught data scientist. Earned some certifications like I said, learned I got that foundation from my IS degree and finance degree and just went full-on data science full-on data nerd and I came in and the company was using this very, brute force model that was basically a scorecard they appeared would use their best judgment in terms of like how they would wait for each of these variables and come up to liken underwriting decision. Let's just say they were not doing too well. They had a lot of default and so, without having any knowledge of evenly using our I jumped headfirst in, started using it, developed an underwriting algorithm, and I spent probably two months developing. It got trust that we start to do a champion Challenger model. They put it on for 20% of all applications that went through our platform. And that said there was a 75% reduction in defaults while maintaining the same amount of approvals. So basically what that meant was, they were throwing out a lot of good people with bad people, and I was able to quickly to find what was actually good versus bad, using data and using data science. That was really cool. My current company now that I've kind of jumped out of the weeds and moved more into a high-level management position. My biggest accomplishment is really about being able to build up my team and promote it. And so, by being kind of a player-coaches, which is my mentality, I had people that I brought on board a couple of months after I started and being able to really build them up to become like the go-to people and being able to promote them and see them basically take, take the vision that I'm seeing for the company and execute on it. And not only do that but up level themselves of where they can start to fill my shoes and then kind of, like, grow this organization up organically. That's been super rewarding to see.

What are the recent developments in the field? How significant are these improvements over past work? What are their implications for future research & industry applications, if any?

Based on experience at: Director of Analytics and Data Science, Womply
Summarized By: Jeff Musk on Thu Nov 28 2019
I think we talked about data science. It's really interesting in terms of just taking a quick step back of how it's perceived externally on what I mean by that is data science can be somebody that does absolutely no machine learning. It could be somebody that is purely doing statistical testing. Or it could be somebody that call it is working, call it Netflix and is working on their recommendation engine or even Amazon and is working on their recommendation engine. They're going to be spending a lot of time in ML. They're gonna be at the forefront of creating new custom algorithms or using something that's open source. Then you have another flavor of data scientist, which is, somebody that at a company opened up a role where really they were looking for like a data analyst or like a business intelligence analyst, but thought they needed a data scientist. And I have seen this occur a lot. That's when I think about the reason I mentioned that it is just to level set of, like, what does it mean to do data science at one plate today? and so it's definitely not the ladder where we were looking for. Like a data analyst. It's most certainly a lot in terms of the analysis space from a statistical perspective based upon our data, and then the second part is starting to delve more into the machine learning space. So I'd say when we think about machine learning today, some of the things we're working on is applying survival analysis to the concept of our customers. And historically, survival analysis was used for the medical industry in terms of doing testing on call it cancer patients. If you're to administer a drug, how does that increase the survivability of that person with cancer? So apply that type of concept to call it to assess customers. Somebody starts paying for your product. What is the survivability of them? Are they going to survive for two months or they're going to survive for 60 months on? So that kind of goes hand in hand when you talk about lifetime value. So we actually applied that concept on it's Not that it's something new for companies to do, but personally, I have not seen that used, at least in more junior sized companies. and so we used it on and used it on our customers. And ultimately we have, like, a 78 reduction in our churn rates on our testing control group. So it's cool, using something that was, like, originally used in medical industry and now applied to like a sass business. So that's one form of kind of quote-unquote ml. And then you get more into, like, extra boost models. Random forests. Just basic regression. So we use those to a degree when it comes time to scoring like if we call a particular person, what's the likelihood they're going to purchase? None of this is necessarily new, but it's applying. Like I'd say, every time you apply an algorithm using ML to, like solving a problem, it's always going to be a different flavor of what somebody's done before. In some instances, it might be new, or it could just be a different flavor. But with the amount of data that we have another customer we have, which is well over 150,000. We do have a lot of data to play around with into test on And so, But if you have, like, 100 customers, forget about ML. That's so overkilling, right? You could just have all that. When you start getting into the thousands, hundreds of thousands, millions, trillions, billions, then you definitely do need some ML because he can't do it all by eyeball. So it's a when we are making some of these decisions from a customer perspective or we're trying to identify out of, like, five million people in United States. Who should we just, like, call on our outbound team? That's something that has been tried to a degree, but people don't have the data that we d'oh. So we're able to use, like, a random forest and get much better results than, like somebody that maybe has, like, a fraction of our data. Something interesting that's happening in the space today if we talk about just purely data, and science kind of comes in from the aspect of working with large data sets. Snowflake is starting to work with a lot of large companies and their opening something called the Data Exchange, and this is allowing multiple companies to share data directly through their data warehouse. So you don't even have to send something through like an API or through an FTP server. You literally can just grab access to that database at another company and have access to that data. That is something that is changing the landscape about speed and ability to answer questions. And when we talk about just like ML models, they're only as good as what you know about the business and the data. Do you have access to it? If we had all the data in the world, we might be able to answer some of the questions a little bit better. But each additional variable you add that is unique, the better your models do get. And so that is a shift in the space of where companies air gathering data, and they're trying to monetize it and so that you're able to get more access to this information that otherwise would have been impossible or super hard or expensive to get. So definitely seen a very clear change in there. That's it doesn't most significant things that we work on it where I'm seeing these shifts. I'll know in a little bit and probably about 4 to 6 months in terms of this other thing that we're building out at our company purely from this consumer and business profile on kind of how that looks. And I kind of consider that a little bit data science, see, but less unlike the algorithmic side.

What was the hiring process like for your job? What were the roles of people who interviewed you? What questions were asked and how did you answer them?

Based on experience at: Director of Analytics and Data Science, Womply
Summarized By: Jeff Musk on Thu Nov 28 2019
My interview process was really weird for my current role s so like I said, I started like their Data analytics manager. I don't think they really knew what they wanted when they opened up the roll. And so I was interviewed by, VP of Brandon Communications was interviewed by the VP of finance, definitely people that were on the team of whom I would be their manager. when the process was, it was actually only one on sites, but there were through there was one initial recruiter from the screen. There was another phone screen with the hiring manager. There was a follow-up phone screen with that hiring manager. There were about an hour and 1/2 conversation and then I had my eye on site after that. That lasted for about three hours. I mean, not a significant amount of interviews compared to certain companies, but also not super short. My Goldman interviews were closer to eight hours, and we had a super day. So this was more of a walk in the park compared to that, so yet that was generally the interview panel. A lot of the questions they asked were around, was I comfortable with chaos, Right, cause it is a startup. How would I operate in that type of environments? And how could I help them understand that through some of my past experience grand with my past experience, I'd come from much larger companies, so I didn't necessarily have the experience to tell him about that. So instead I said, Here's how I would operate. Here's what I would do. And so I talked about, like, clear communication expectations setting and you could see like when heads were nodding. That's generally a good sign. It's also a really good sign when towards the end of an interview, you're kind of just talking about things that aren't even related to the job. And you're starting to kind of build that relationship with that individual. That's definitely clear. Sign that you're both. They feel that you're able to do the job. They feel confident about that, and then also they're kind of trying to assess like what? I generally like working with this person, like what they fit into our culture. And so we always talk about in his culture added in our company. Now, on the team side, they were asking a lot about, like, the tools, the nitty-gritty like that. I know how to use Snowflake. What was my prior analytics experience? How would I act as a shield to them? Because if they keep getting blown up, they can't work very efficiently. So, like, how would I handle the workload? And then from the VP of finance, just kind of want to know like he knew nothing about analytics, Some CFO's air chief like digital officers, depending upon like, the title of the company and, like analytics or data science might roll up to that division and finance. It's kind of hit or miss of like, how will they know the analytic space? They might love analysis, but not know a lot about, like the technical nature of it s o. His questions were more around the lines of like, I don't know anything about this. Like, how are you going to take this and achieve, like the results that I'm looking for and help walk me through the floor literally the nitty-gritty all the way to, like the high level so it's alike on a scale from 1 to 10 of being an easy interview process versus Hard, and 10 is a hard I'd call it like my experience was around like it's seven. And then there was lots of back and forth between me and the recruiting manager in terms of start dates and all that fun stuff.

What qualities does your team look for while hiring? How does your team interview candidates?

Based on experience at: Director of Analytics and Data Science, Womply
Summarized By: Jeff Musk on Thu Nov 28 2019
it can wear multiple hats that can kind of thrive in an environment of change. Somebody there is full-stack. So you've got to be able to I call, like specifically on my team. You've got to be able to pull Sequel. if you don't know how to write SQL, that's It's a dead end. You need to know had to literally pull the data. I know that what you're pulling is good in terms of like, some companies will have a D. V A. We'll have, like this great data architecture things ever spelled out. You have a table for sales. You have a table for revenue. You have a table for your customers and our company that does not exist in its purest form because we've evolved so much overtime, and so you have a lot of things that are built on each other. You require tech debt. It's hard to build that down. So for us, it's about how do you question the data? So it's a good, healthy dose of skepticism of really questioning what you're looking at, an understanding the shape of the data, taking that knowledge and then being able to apply it to some of the complex problems that we're trying to solve today. I should give you a very, basic example, our products, if you're familiar, like a male chimp, are another type of serum tool that reaches out to your customers Are product does that? And so I take it we get this data at its raw, purest form and it's literally just like someday, some properties that say we sent the seam allowed to this customer. Now we want to understand at the very high-level question Are our customers making more money? So we have to Then, look at these email properties. Look at it in its purest form model that matches it to kind of the revenue of the business. And then blast set out to a higher level and then understands of a controlling a test group like, are you actually making more money? And so that's what taking something from literally the very basic data and then seen all the way through to a final analysis which could be delivered. To call it the CEO of the company s. So that's what I consider like full-stack. So definitely those types of things were necessary from, like, a sock skill perspective. Like clear communication, clear expectations set in, being somebody that is kind of a self-starter and is driven, somebody that knows how to put data to work. Meaning that it's great that lights. You could pull this data, but, like, what does that actually mean? Like, Okay, so you pulled that Sales were 20,000 last month in 10,000 months before. If you can't tell me that, like our revenues growing them like we have a problem so that that that's fundamental. What else? Definitely for people that can work well across functions. So somebody that can work well with somebody that's, like at the VP level, and then somebody that can work well with another icy and they can work in like they can work with engineering. They can work with the product. They can work with the finance team s. So you have to have good, like interpersonal skills. I say that those are critical things. There's other soft skills which are nice, but they're not something that would necessarily deter us from hiring somebody. Um, in terms of the interviews use so kind of how I was interviewed for the role. We have a recruiter from screen hiring manager phone screen. Then we typically do one on-site, in which case, depending upon the schedule, will be broken up into two on sites or just one. And that one would typically lost about three hours. after that one. Then we would typically have a follow-up phone scream, and they would just quickly have a conversation with the VP of my team, who's the VP of all of our data on then, if he felt good that they were a good fit for the team in general, that would probably move through to an offer.

What are the various starting positions and salaries in your domain? What are the typical career paths after these starting positions?

Based on experience at: Director of Analytics and Data Science, Womply
Summarized By: Jeff Musk on Thu Nov 28 2019
Starting positions would be something like a data analyst. Business intelligence analyst, data engineer, data scientists, I say data scientist isn't going to be somebody writes out of college, At least for our company. It's gonna be somebody that has 2 to 3 years of experience doing, like business intelligence. Or it could also be somebody that came out of, like, a PhD degree. If we were looking for that specifics to just statistical skill sets, that could be a data scientist. But certainly with just the bachelors would not fall directly into data science. other rules now, nuts, Basically it way would call like something, maybe like a web or product analyst. That again would be somebody with a couple of years of experience doing that type of thing most likely and like the rule of business intelligence or Data Analytics. Ah, in terms of salaries, it literally could be all over the map. So if you think about San Francisco, starting salaries, they're gonna be significantly more than they are in Utah. That's mainly due to the cost of living. So if we think about like, starting salary ranges for people and call it. You are if you're like a BI analyst coming out of college, you're gonna be somewhere between 65 to 80,000 depending upon the company you join, they're more likely to fall in kind of that lower scale and then quickly, with a couple of years experience you started built yourself up in terms of data engineering. Typically, we paid a little bit more due to the work that they on the skill set required to definitely far more technical and Sturm's of understanding python, different database and pipeline technologies. Scientifically, dead engineer is going to be somewhere between 80 to 100 starting. if we think about data scientist, anywhere between call it low nineties to anywhere up to, like 140. There's a wide range there. It also again depends on the company. Eso these quotes are kind of things I've seen. I have searched, to be very clear in terms of the different starting salaries of what we should pay for a type of role if we were to hire. In addition to that it does fall in line of like something we would pay at our company. then when you talk about, like analytics manager, you're anywhere from, like, all the way down. Surprisingly, I've seen some around 95 all the way up to 140. Then you start to move into a kind of my level. And at a director level, you're starting to get into anywhere between 120 to 200. There's some companies depending upon the size that are up to that on, then you have VPs. Where there is some overlap is well of where it is from 150 to well over 200 again depends on the company. Certainly there's more consensus around kind of what you see at the lower levels than keeping director, but I can attest that generally those ranges are correct. There's also kind of the concept of if you are comfortable with more established companies and you don't like change so much, you're probably gonna fall toward the lower end of the spectrum ZX. The companies aren't gonna pay quite as much for like, a startup. They're probably going to pay more because there's gonna be a lot more required of you and things were gonna be a lot more chaotic, but they want to retain, like, very smart people to build on their vision. With that, there's, like risk that, you know, you could be a company that two years down the road fluster sze they don't fundraise again that run out of capital. Everyone's without a job, right? So there's more risk in that type of environment. So you kind of, like, play that game a little bit, right? So it's not just black and white that every company is the same with each type of salary in position.

How did the program prepare you for your career? Think about faculty, resources, alumni, exposure & networking. What were the best parts?

Based on experience at: Bachelor of Science (BS), Finance, University of Utah - David Eccles School of Business
Summarized By: Jeff Musk on Thu Nov 28 2019
Pretty well. I know that now. There's, like, more of a focus on the analytic side and data science, I think had , you know, I've been in those programs I potentially could of could've avoided some of my original like learnings in my 1st 4 and 1/2 years and potentially propelled myself quicker, but again, it was a little bit about finding what I liked personally, so I can't completely say that it would have been entirely different. But I think some of the key things that helped prepare me, was just the breath of things that I was exposed to if I had just been in the weeds and I was super technical, I wouldn't be where I am today. I needed the courses on strategic management. I needed the courses on operational management. I needed the courses in finance to understand basic principles of like, the time value of money. And while it not might may not be that literally. I use each of those calculations each day. It was the concepts of what did that mean? And how do I apply to the real world? So, like, if you were to ask me to calculate, You know, like the present value right now of oven assets and forecast the future value. Probably flustered. I could remember the calculation. But if you were to tell me, like, can you explain the concept of time, the value of money, or even, like options? I could easily tell you that, and I could help. You kind of translate that into, like, how what that means for our business today. So I think, like the breath of just understanding multiple things about a business, Certainly, if you want to be in a management position is imperative again, with a very solid foundation of like, I think, back to my courses on the ice. IS program, you know, I don't really is too much of my networking, coursework. I do use a gross amount of my database coursework on just thinking about just basic relationships of a database that was fundamental. Nowadays do people like to optimize their tables and like their structure? No, it's kind of sad, like the role deviate has disappeared in the industry for newer companies. Now everything's done in the cloud, and it's much easier to just throw a bunch of resource is from a computational perspective at a problem than it is to have, like this very clean optimized index of it of a query or a database so that's definitely a shift, but nonetheless, like having the basic knowledge of like how things should connect together. It's like the building blocks of how you build things. So that the best parts were, I think some of the times that I laugh the most and like I found like, this enjoyment was through some of Boyle's classes, that there was always a lot of things to go through from, the conceptual perspective of just literally learning so many terms. funny enough, some of the terms still stick in my mind. If the ones that I used today a lot of the ones to like the way he would always call it is like you would you would you memorize it and then you flush it after the next exam, but definitely a lot of stuff. I did flush, and I probably don't need to know, But the things that now I apply, I'm thinking back I might go. Yeah, that definitely makes a lot of sense. he definitely got me excited about wanting to learn on and kind of be in that space because from, like, an outside perspective, I was in the fraternity system of the U, and, like, I was like, Oh, cool. I'm gonna do this finance stuff. I don't know if, like, I really want to do data stuff. But nowadays it's like, Oh, that is actually super cool like that is the place to forget about, like, investment banking data. Science is the thing up. So I think just kind of that energy and that vibe definitely helped pick up the pace. I don't know what the perception is in college these days in terms of, like is data the new frontier. And that's what everybody wants to do. If so, then that that that's awesome. Clearly, you know, the word has gotten out. But if not, I'd say it's definitely the cold sitting out there