Machine learning model

How to build and train Machine Learning Model?

Google Cloud, Machine Learning / By

It takes a lot of effort and time to develop ML models from conception to production. To train the model, you must manage a substantial amount of data, select the best training algorithm, control the computing resources while training the model, and finally deploy the model into a real-world setting. In this post, you will walkthrough on how to build and train machine learning models.

In this tutorial, you will assume the role of a Machine learning Engineer and you have been asked to develop a machine learning model for predicting the top five jobs.

Time to dig in!

Considerations for training a model

When training a model, the following considerations are taken into account and they are:

Productionizing: One should refrain from using TensorFlow in eager mode. For production tensorflow(extended) or tfx in short is an end-to-end platform for deploying production ml Pipelines.

Static or offline inference: It is used for considering all the possible forecasts in a batch using a map reduce or something very much similar. Then write the predictions to an SS table or big table and then feed these to cash or look up the table. 

Do not need to be worried about much of the costs of inference. As it can likely use the batch quota itself. You can do Post verifications on data before pushing that’s like dynamic or online inference meaning that you predict while required using a server.

You can predict any new items on the go in real-time computation. And intensive latency sensitivity may limit the model complexity and thus make monitoring the requirements more exhaustive. 

Training a model as a job in a different environment

The most reliable way to make sure that you train as you serve is to save the set of features used at Serb serving time and then pipe those features to a log to utilize them at the training time in a static training model.

The Dynamic training model is trained online so let us understand them in detail.

In the static training type model, the model is trained offline, that is, we train the model exactly once and then use the train model for a while so it’s easy to build and test and batch training is comfortable, requires monitoring of inputs and it is easy to grow on a scale. 

Dynamic training type model is trained online and the data is continually entering into the system and we are incorporating that data into the model through continuous updates and it continues to feed in training data over time regularly sync and update the version. 

The Progressive validation needs monitoring model rollback and data quarantine capabilities will adapt to changes. 

Stillness issues can be avoided even if you cannot do this. For example, do it for some small fraction such that you can substantiate the consistency between serving and training teams that have made this measurement at Google were sometimes surprised by the results. 

In the YouTube homepage, switch to the logging features and serving time with significant quality improvements and reduction in code complexity and many teams are also switching their infrastructure as Google has told about it tracking.

Now you can view the status of the job using the Google cloud ML engine jobs described and then the job name command. 

Followed by an inspection of the latest logs using the Google Cloud ML Engine job stream logs job, filter the jobs using the Google Cloud ML Engine jobs list and you can even pass the filter argument maybe with a condition that creates time is greater than a date timestamp. 

Now the second metric is the loss function which is used during the training harder to understand and indirectly connected to the business codes, performance metric types, and confusion matrices.

Then the Precision which means true positive or divided by all positive predictions when models set positive whether the class was right then the recall true positives or all actual positives out of all the possible positives how many did the model correctly identify after training.

It is easier to understand directly connected to business goals and retraining or redeployment evaluation can also be processed. 

Monitoring the training jobs 

Depending upon the magnitude of the data set and the complexity of the model, training can take some longer durations training from Real World data can last many hours. You can monitor several aspects of the job while it runs for overall status. The most facile way to check on the job is in the AI platform training jobs page on the cloud console.

You can get the same information programmatically and with the Google Cloud command line tool,  use Google Cloud AI platform jobs to get the details about the current state of the job on the command line. 

You can get a list of jobs associated with the project that includes the job status and creation time with the Google cloud AI platform jobs list note that this command in its simplest form lists all the jobs ever created for the project. 

Also Read: Free Questions on Google Cloud Certified Professional Machine Learning Engineer

You should scope the request to limit the number of jobs reported. The following examples can be considered such as the usage of limit argument to restrict the number of jobs.

Now let’s take an example to list the five most recent jobs so you can say the Google Cloud AI platform jobs list is equal to 5. We can use the iPhone filter or filter argument to restrict the list of jobs to those with the given attribute value you can filter on one or more attributes of the job object as well as the core job attributes. 

You can filter on objects within the job such as training input object jobs can fail if there is a problem with the training application or with the AI platform training infrastructure.  so you can use cloud logging to start the debugging process. 

Did you try our Practice tests on Google Cloud Certified Professional Data Engineer ?

You can find the following resource utilization charts for the training jobs on the job Details page.  the jobs aggregate CPU or GPU utilization and memory utilization and these are broken down by a master worker and parameter server the jobs Network usage is measured in bytes per second. There are separate charts for bytes sent and bytes received.

Go to the AI platform training jobs page in the cloud console and find the job in the list. Now click the job name in the list to open the job Details page and select the tabs labeled as CPU,GPU, or network so that you can view the associated resources in the utilization charts. 

The machine learning model was built successfully and classification was made for the chosen example. 

Summary

Hope this blog helps you to learn how to build and train Machine Learning with illustrative examples. As a Google Machine Learning Engineer, you should have mastered the skills in building ML models. 

To learn more about machine learning and how to make machine learning models, it is recommended to take Google Cloud Certified Professional Machine Learning Engineer certification at Whizlabs. We afford practice tests, video courses and Google sandboxes to assist you in clearing this certification.

If you have any questions or doubts, please comment, and we’ll have our experts answer them for you at the earliest.

About Pavan Gumaste

Pavan Rao is a programmer / Developer by Profession and Cloud Computing Professional by choice with in-depth knowledge in AWS, Azure, Google Cloud Platform. He helps the organisation figure out what to build, ensure successful delivery, and incorporate user learning to improve the strategy and product further.

Related Posts

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top