In the fifth installment of Carmen’s series, the team collaborates to make the big decisions for the final phases of the AI project.
Part five of a series.
When building a Minimum Viable Product, you exclusively focus on the crucial elements of your idea. Those elements are interesting and exciting and the entire reason behind the project. They are your raison d’être.
But when things get real, you also have to consider the less glamorous parts of the project. Your idea cannot live in the world without structure and methodology. Those elements are what is required, de rigueur.
And Now, For the Artificial Intelligence and Machine Learning Part…
This project hatched because I am obsessed with Artificial Intelligence, and Machine Learning in particular. I was completely in my sweet spot during the MVP phase, as we almost solely focused on the Machine Learning elements. I had no idea what I was doing, but I enjoyed every single moment of it.
Now that the project was funded, we needed to build not just a few Machine Learning algorithms but an entire working system. We needed to figure out where we were going to get the data, how we were going to get the data, and where it would live. Necessary, but unglamorous infrastructure decisions. Definitely not my sweet spot.
Thankfully I am surrounded by a team that is much more excited, and enlightened, about these sort of things.
The first decision we (they) made: “What platform will we use?” While there are a variety of cloud options out there that could certainly do the job, we quickly settled on Microsoft. The deciding factors:
- We had people on the team who wanted to learn Azure.
- We had people on the team who wanted to see if we could spin this into a potential Azure Marketplace offering.
- Our Microsoft rep was committed to helping us.
So, not the most rigorous approach to picking our environment but one I am content with. There are so many decisions to be made when managing an innovation project that sometimes you need to just pick the obvious one, the bien entendu, and go with it. Only get into a decision making mêlée when you really need to.
Next, we all met vis-a-vis for a white-board session.
The entire team got together, plus our Microsoft rep, to flesh out the architecture. Up to this point, we had all been working remotely, so this was our first time to actually sit in a room together. While I generally think remote work is highly efficient, there are times that face to face interaction is invaluable. This was certainly one of them.
After some healthy debate, we settled on several key elements including:
- Data Inputs: Taleo (our Applicant Tracking System), Employee Resumes, HR & Project data from our internal database
- Accessing the Data: Azure Data Factory for most. Old fashioned copy/paste for the employee resumes as they were on an old instance of Sharepoint and we are not gluttons for punishment
- Data Storage: Data Lake for the resumes, SQL Server for everything else
- Machine Learning: Our Python scripts from the MVP as well as AzureML
Machine Learning, j’taime. You are why I am doing this project, my raison d’être. But I have a new appreciation for cloud architecture now. While it may never make my heart swoon, it is a necessary part of life, de rigueur.
La vie est belle.
Behind the Scenes of our AI Project
Now that we have a framework in place, the project is humming along. Things are being talked out, figured out, and built out. Here is a quick snapshot behind the scenes:
- What’s making me feel dumb – The more I learn about Artificial Intelligence, the more I realize I do not know. It does not help that the field is a moving target. It changes daily with new technologies and new algorithms and new applications. I subscribe to this really good tech news digest and have been devouring all the articles remotely related to AI. From self-driving cars to poor paint color predictions, I want to understand it all.
- What’s making me feel smarter – I’m still chugging along at my Microsoft Data Science Certificate. I finished out the T-SQL class and moved onto PowerBI. As mentioned before, a hundred years ago I used to manage a Database team. At the time, data visualization tools were pretty awkward and exasperating to use. In the ensuing years, I had little use for reporting tools and lost track of where the market was. Seeing and using PowerBI felt like I went from driving a horse-and-buggy straight to a Tesla Roadster.
- What’s keeping me up at night – Everyone on this project is all in. They are dedicated to learning the technologies, building the right solution and ultimately seeing Smart Staffing become a success. But it is nobody’s day job. Work is done in fits and spurts as they have incidental windows of availability. Ultimately, this is a voluntary project, and as a manager, I need to strike a balance between driving the project forward and being respectful of everyone’s personal schedules. So when I get asked by outsiders when the project will be done, the most honest answer I can give is “When it’s done.”
My hope is by sharing our project’s small victories and painful bruises, you will be encouraged to tackle a project that scares the sh?! out of you too.
