Allegro Trains, a project management hub designed to seamlessly integrate with ML workflow
Republished here with the author’s permission – original post on Medium here.
Authored by Henok Yemam.
The resurrection of AI due to the drastic increase in computing power has allowed its loyal enthusiasts, casual spectators, and experts alike to experiment with ideas that were pure fantasies a mere two decades ago. The biggest benefactor of this explosion in computing power and ungodly amounts of datasets (thank you, internet!) is none other than deep learning, the sub-field of machine learning(ML) tasked with extracting underlining features, patterns, and identifying cat images. On a serious note, the advancement led by this technology is ubiquitous with news media outlet darlings such as Tesla’s Autopilot, google’s AlphaGo, and Brain-Machine-Interface technologies. Although they are worthy of every bit of compliment coming their way, it hides the incredibly difficult and complex task beneath the end product. Anyone who has dared to try their hands on a deep learning project knows it’s just not easy! It takes excellent project management skills, several frustrating late nights and non-linear assembly of working parts. You would be considered lucky if you end up with no gray hairs by the end.
The Current ML Workflow Problem…
If you are not one of the lucky ones, you are left with spending a lot of time in the mud taping up your spaghetti codes, pushing your wonky branches to GitHub, begging for pull requests, and merging to your poor origin master. This workflow seems to be the most accepted norm in the industry. However, do you have a good picture of the steps you took to get to your destination? Let’s say you were asked to present how you came to the glorious conclusion you put forth, would it not be nice to pull up a flow chart or a webpage that succinctly shows your daily venture into the unknown? If that seems appealing, let me introduce you to a superb abstraction of an automated project management system specifically designed for ML projects. It requires little to no effort to set up on your computer whether you run on Linux, macOS or Windows. The company behind this product (Allegro AI) calls it Allegro Trains, Welcome On Board! Please show your open-source platform membership at the gate when you come in.
As I mentioned earlier, the ability to extract meaningful insights from large datasets is a critical skill in ML. With this necessity comes the need for countless experiments with a wide range of helpful statistical and plotting libraries, frameworks and algorithms. The wide consensus in this field is to utilize a version control management system like GitHub so that your experiments are being tracked in a timely manner before they get out of hand. However, there are a few limitations to community-version GitHub. Some of them are:
- It doesn’t track your resource allocation to each iteration of your experiments.
- It doesn’t integrate from your choice of IDE whether that is Jupyter Notebook/Lab or PyCharm.
- It has a limitation on how much of your code/graph it will display on the web.
GitHub’s game-theory based engine is as eloquent as it gets but inquiries about its ever-growing syntax make it a Stack Overflow darling.
Two lines of code to track your ML Project
from trains import Task Task = Task.int(project_name="MarchMadness", task_name="ETL")
Trains Out-of-the-box Solution
Trains, on the other hand, integrates with your IDE with just Two Lines of Code (TLC) at the top of your cell as shown above. The all too familiar Python’s builtin function import accesses the trains module which you can install from pip. The Task class comes with a plethora of helpful functions such as init which tracks your experiment, ETL in this case. Similar to GitHub, Trains gives you the version control your soul desires but without the worrying aspects of conflicts, commit issues or search for the right syntax. You can imagine if you start a new notebook with a different name and type in TLC with a new project and task, Trains will automatically track these projects separately. If you decide to abort one of your tasks and restart it 2 hours later (or 8 days in my case, caption below), Trains will keep these two versions separately giving you the dynamics of a version control system. Let me show you what I mean.
As you can see above, MarchMadness project was created in three separate Notebook instances and it has 3 experiments or tasks in it: Testing, ETL and Data Exploration. You may have noticed that there are two versions of Testing. These are your branches which you can compare side by side to get a better idea of your progress with just a single click (similar to git diff ). Trains actually goes further than GitHub by tracking individual versions of your tasks’ processing load distribution such as CPU/GPU usage, network and I/O speed(look down below) so you can identify the bottlenecks of your ML codes and work on faster iterations without the need to optimize every single line of code. You could also use this information as a guide before migrating to cloud computing services such as AWS and Azure. If you want to leave a message for every iteration of your experiment (similar to git commit), Trains got your back there as well.
Trains Additional Features
You may be thinking ‘what is the catch here?’ I share your cynical view of the world but I am not done yet. Trains serves more refreshments for ML practitioners and developers with just TLC which are not even within the realm of community-version GitHub. Some examples of these include:
- Saving snapshots of your ML models that can be shared on cloud storage systems.
- Displaying an apt summary of python packages used in your experiment with their specific version which can save you a lot of time while debugging.
- Provides control via a web UI to remotely stop or monitor your experiments which are running on your local machine.
Last but not least, Trains versions the code itself converting .ipynb files to script .py files so that it’s tracking actual improvements to the code as opposed to which code you are executing. This is important when you are collaborating remotely or on different machines so that your code can be reproduced using the script file along with the correct versions of the packages used in your code (similar to requirement text).
Summary
It has been a pleasure giving you a taste of what Trains has in a store for you relative to the normal GitHub workflow you have grown accustomed to. As part of your open-platform membership, there is a lot more to Trains that I have not explored here, awaiting future exploration and discovery. I encourage you to continue to collaborate and take your ML projects to the next level by automating your projects using Trains. I also encourage you to share your progress among your peers of developers and data professionals. From my personal interaction with the team, I know Allegro AI deeply appreciates your continued participation and insightful feedback and suggestions. I encourage you to open issue(s) on this channel and you are bound to get a quick response. As for me, I have data science projects to monitor offline and I hope to catch you on my next ride with Trains which will focus on machine learning and deep learning explorations of chest X-ray and CT images of COVID-19 patients. Until we meet again, Bon Voyage!
