If you had tried training a machine learning model, you would have done lot of experiments changing hyper-parameters, measuring several metrics , you would have got different values for those metrics etc. Whild doing all these experiments you would get lot of versions of the models which you would like to refer to later. As you do more experiments it becomes increasingly difficult to keep track. That’s were MLflow becomes handy tool for you. MLflow from databricks helps you navigate the jungle with ease.
First you have to install mlflow >
pip install mlflow
I am providing below a simple example of how to use mlflow with a simple example below. In the below example you could easy figure out the python api used . start_run, create_experiment, set_experiment, autolog and end_run creates an experiment run , logs some details about this particular training done on the popular iris dataset and showcases how you can visualize these details in a web interface :
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
import mlflow
import mlflow.sklearn
data = load_iris()
X, y = data.data, data.target
seed = 10 #Specify a seed value.
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state = seed, stratify = y)
from sklearn.ensemble import RandomForestClassifier
current_run = mlflow.start_run()
mlflow.create_experiment("first experiment")
mlflow.set_experiment("first experiment")
#Logging the seed value to passed to the train_test_split function.
mlflow.log_param("seed", seed)
estimators = int(input("Estimator(s): "))
#Model definition.
rclf = RandomForestClassifier(n_estimators = estimators)
mlflow.sklearn.autolog()
rclf.fit(X_train, y_train)
mlflow.end_run()Once you had run the below example you run the mlflow ui as below:
>mlflow ui
This will give you a webserver end point like this below:
If you point the web browser to this URL you would see a dashboard explaining the various aspects of your experiments.
You can see the various experiments you had run with all the parameters used each time. You can add metrics, parameters, artifacts of the run programmatically.
