I’m currently finishing up week 2 of my first time teaching a course on MLOps, and I’m already learning so much about what I can and should be doing better the next time I teach it. It’s amazing how the reality of teaching a topic in a certain way completely misses the expectation you might have going into it.
Tools Versus Concepts
I spent about 6-8 weeks prepping for this course. My original idea was that this would not be a tools course, meaning it would not be a course where we focus on learning the different MLOps tools in-depth, but rather we focus on the concepts during lecture, and then the students would go about doing a project where they build a machine learning-based app, and try to deploy it. Doing the course this way meant that I would not spend a lot of time on a specific tool that the students may not even use when they get a job, since companies use different tech stacks. But, there was a problem. The students already have to take a separate capstone course where they build and deploy a machine learning-based app, meaning the project in my class would be redundant, and so I decided to pivot.
Literally 2 weeks before class started, I changed the focus of the project to emphasize the concepts plus the tooling. For each concept, or stage, of the MLOps pipeline (think experiment tracking, data versioning, orchestration, CI/CD, model serving, model monitoring), the students would explore a couple of tools, dig into how they serve the purpose of that stage, and decide which one they liked better. Essentially, I asked the students to get in groups and pretend they are part of a MLOps team at a company, and that I am their manager, and that I am asking them to design our MLOps pipeline. Now, they have to run off and do some proof-of-concept projects to build that pipeline.
MLOps Project Execution
I like the idea of the project I came up with, but so far I have not executed it well. It’s a confusing project. Most of these students have not yet had the experience of doing POCs at a company, and so they are still trying to figure out what it means. I get it. The first time I had to do this I was asked by my manager to figure out which version control system our data science team should use. I had an obvious answer for him: git + Github. But, he wanted to know why, and this was at a time when Github hadn’t been around very long, so it wasn’t the clear winner it is today. So that meant I had to go out and bring back solid convincing evidence to back up my suggestion. I didn’t know where to start. This was early in my career, I had no idea what would be convincing. So, I was a bit lost and spent time doing research and feeling my way through this POC. I imagine this is how my students are feeling. Eventually I figured out that I should try git, try some other tools like SVN, do some research on the pros and cons of each, and figure out which tool would work best for our specific circumstances.
The obvious solution to clearing up my students’ confusion is that I should show them what one of these POCs might look like. But I didn’t think of that before I started the course. And now I plan on spending the weekend putting together an example for them to learn from, and next time I teach the course I’ll be able to showcase some of the stronger projects the students turn in this year.
Demos
Live demos are hard to get right, but I find myself doing a live demo each time our class meets. And as anybody who has experience doing live coding/technology demos knows, no matter how many times I practice the demo beforehand, and work out all of the bugs, something inevitably goes wrong. I practice each demo at least twice beforehand, in fresh environments, until I’m satisfied the code works. But then, I get in front of the class, and I get errors. Every. Single. Time.
I have mixed feelings about demos in general. With recorded demos, you can pause and rewind, but even that is sub-optimal because you have to search around for the spot that you’re looking for. It is simply much easier to walk through a notebook on my own, run the code, and read the explanations, without the distraction of somebody speaking and running code in front of me at the same time. My demo notebooks are very detailed, detailed enough that anybody should be able to go through it and see what each line of code is doing, but I feel like I’m supposed to be doing more than just giving my students a notebook and asking them to go through it while I stand there and wait for questions. Am I wrong for feeling this way? Should I simply provide a lecture, an overview, of the material and the demo, and then let them loose to work on it? Should I record the demos separately so that they can watch it on their own, pause, rewind, fast forward, etc.?
Rest of the Course
These are just a few of the things I’ve noticed after the first two weeks of teaching this course for the first time. I will likely make some adjustments throughout the coming weeks and write one or two more posts documenting the experience.
My goal, at the end of all of this, is to have something interesting to publish about teaching MLOps. It’s a topic that isn’t taught widely yet, and there aren’t a ton of examples to learn from, and it is still a quickly growing area that hasn’t really settled. Most courses I’ve seen expect students to build an app. Most pick one tool and go with it. Many have guest lecturers come and talk about how their company or their team does MLOps. All of these are good ideas. I’m trying something my way, and am excited to see how it all works out in the end.