May 31, 2021
Training and Testing Machine Learning Model inside Docker Container
docker containers provide quick environment setup to deploy any product hence we are using this to train our model and then predict by providing some dummy inputs…
Faster deployment is need of today!! As we want to have testing environment to test any model’s accuracy, in our ML Model use-cases for an intelligent prediction which is required. We use to ensure by testing before sending to production environment for wise prediction this could be possible by docker container as they just launch in a second and provide resources directly from base OS by the help of daemon.
Machine Learning:
Machine Learning models does data analysis and create a pattern known as Formula for either Linear or Classified data. So there are mainly two types of ML models:
1. Regression Model : ML models for regression problems predict a numeric value. For training regression models, ML uses standard learning algorithm known as linear regression.
2. Classification Model : ML models for multiclass classification problems allow you to generate predictions for multiple classes (predict one of more than two outcomes). For training multiclass models, ML uses standard learning algorithm known as multinomial logistic regression.
To know more about ML and it’s Origin you may visit my latest blog on Neural Networks, Al and ML, titled “Neural Network — MADALINE to Self-Driving Cars”.
Docker :
Docker is a Containerization tool that helps user to launch an Operating System from a pre-booted image and then use the image for testing or Quality Assurance of the product. Also for Automation and Orchestration we use these Containers. Docker tool itself has many capabilities that ensure the volume attachments, networking by using Porting and also copying, pushing and pulling image from docker Repository (hub.docker.com) and many more.
To know more about docker tool and read about the webserver deployment over the python containers using a Configuration managment tool you may refer my recent blog titles “Setup Web Server inside Docker Container using Ansible Playbook”.
Our Model Dataset is as follows:
You can observe that we have only two columns one is “featureone” which contain the Employee’s Year of Experience. And second is the “Extimation” which contain the historical data for the amount payed on the experience basis. It’s a Linear Data Set having one Independent Feature.
Now We will import the data from this file in our Python Code and Train our model to get a model\function. The Weight (Slope) and Constant\Bais (y-intercept) are must in the model with Linear Regression Model Formula “y=wX+c”. Here, y is the predictor, X is independent feature help to predict y, w is weight\Experience Pattern, c is Constant\Bias i.e. least estimated Salary.
ML-Salary-Estimator-Python Code:
Following is the Python Code that Estimates the Salary of the Employer having some year of Experience. It Estimates salary according to previous pattern it analyze from the historic dataset that we have given in the form of dataset.csv file.
We are using 3 Python Libraries here named “Pandas”, “os” and “scikit-learn”.
This Product asks User to enter the product key before entering and then the training of model starts from:
- Loading of Data
- Fetching relevant independent features for finding prediction pattern\formula
- Loading Linear Regression function as we have linear data in hand (dataset.csv)
- Fitting the value of predictor (x) and predict (y)
- This whole training we are doing here can be just done once and after that we can save the model output weight and bais to load and use. (Suggestion)
- At the end it would ask user about entering the product or exit with ‘Y’ or ’N’ keys
- Then Prediction function would work as user input the ‘Year of Experience’ of new joining employ to estimate salary
- We have options to continue working or ‘exit’ when work is done.
Pre-requisite :
◾ Linux Machine installed in Base OS or VM.
◾ Docker installed over that OS
Let’s Discuss the Problem Statement:
- We have to pull Docker Container image of Centos from hub.docker.com and further create a container from that image.
- Inside that Launched Container
1. Install Python3 software
2. Install python modules scikit-learn, numpy and pandas using “pip3” - Copy the ML code that we have created for Salary Prediction inside the container and then deploy it. We can do this in 2 ways:
1. by “docker cp” command to copy code directly from base OS.
2. by “git clone” as we pushed the code and data to GitHub repository for further use.
Going step by step I will now show you the Execution and solution of the problem statements:
- You can See I have my docker.service running in Rhel 8 Linux Machine
2. We have to pull the “centos:7” image from docker hub by using command:
# docker pull centos:7Now I will show u the centos images I have pulled in my OS:
3. Now lets launch a container named ‘esti-con’ from the centos:7 image with interactive terminal interface:
4. Lets install python3 software inside this container using command:
# yum install python3 -y
5. Using pip3 command lets install the required libraries for our ML code execution using command:
# pip3 install scikit-learn numpy pandas
6. Lets copy the code files in our docker container from base OS using command :
# docker cp <source_path> esti-con:<destination_path>
7. Lets Now run our python program for training the model and further predicting the salary for employees:
Hence we have successfully achieved our goal for “Training and Testing Machine Learning Model inside Docker Container”.
You may refer the code and dataset over my GitHub repository:
You may connect me on LinkedIN for more such contents and blogs:
