NORDICS20 - Assets

Deep Learning on AWS

Amazon Web Services Resources EMEA

Issue link:

Contents of this Issue


Page 12 of 50

Amazon Web Services Deep Learning on AWS Page 8 training process. Searching for the optimal hyperparameters is an iterative process, and because of the high dimensionality and complexity of the search space in deep learning implementations, this endeavor can be labor and cost intensive. A variety of strategies have been developed to find the optimal hyperparameter settings via techniques such as grid search, random search, and Bayesian optimization. Hyperparameter tuning is available as a turn key feature in Amazon SageMaker. Step 5. Deploy Models in Production Deploying a machine learning model into production often poses the most challenging part of an end-to-end machine learning pipeline. That is because deploying machine learning workloads differ from traditional software deployments. First, we must consider the type of inference that the model provides: batch inference (offline) versus real-time inference (online). As the name implies, batch inference generates predictions asynchronously on a batch of observations. The batch jobs are generated on a recurring schedule (e.g., hourly, daily, weekly). These predictions are then stored in a data repository or a flat file and made available to end users. Real-time inference is the process of generating predictions in real time and synchronous upon request, most often on a single observation of data at runtime. Second, we must consider how the model is retrained. For a model to predict accurately, the data that is provided to the model to make predictions on must have a distribution similar to the data on which the model was trained. However, in most machine learning deployments, data distributions are expected to drift over time, and because of that, deploying a model is not a one-time exercise but rather a continuous process. It is a good practice to monitor the incoming data continuously and retrain your model on newer data if you find that the data distribution has deviated significantly from the original training data distribution. Based on your use case, an automatic instead of an on-demand approach to retrain your model may be more appropriate. For example, if monitoring data to detect a change in the data distribution has a high overhead, then a simpler strategy such as training the model periodically may be more appropriate. Third, implementing model version control and having a scaling infrastructure to meet demand is not specific to deep learning, but requires additional consideration. Version control is essential because it allows for traceability between the model and its training files in addition to allowing for verifiability, letting you tie the output generated by a model to a specific version of that model. Dynamically adjusting the amount of compute capacity for an inference endpoint in addition to having the capability to add fractions of

Articles in this issue

Links on this page

view archives of NORDICS20 - Assets - Deep Learning on AWS