NORDICS20 - Assets

Deep Learning on AWS

Amazon Web Services Resources EMEA

Issue link:

Contents of this Issue


Page 41 of 50

Amazon Web Services Deep Learning on AWS Page 37 this section, we review some advanced use cases using Amazon SageMaker and other AWS services. Orchestrate Your End-to-End Machine Learning Pipeline using AWS Step Functions AWS Step Functions allow you to build resilient serverless workflows. In AWS Step Functions, a workflow is implemented as a finite state machine. The states can be a task, a choice, a branch of logic, a set of parallel tasks, an error handler, and so on. The workflow is implemented as a Directed Acyclic Graph (DAG) and uses GoTo logic. AWS Step Functions also allows you to throw an exception and do error handling to make the flow more robust. In AWS Step Functions, the task states do most of the heavy lifting. There are two types of task states: Activity task and Lambda task. In Activity tasks, worker requests work from AWS Step Functions, then takes the work and returns the results. The Lambda task is a synchronous call to an AWS Lambda function from AWS Step Functions. The Lambda task has a maximum timeout of 15 minutes as defined by the max execution duration of the Lambda function. AWS Step Functions also allows you to insert human actions such as approval and rejection into the state machine. The actions can be used in the workflow to approve or deny the model push into the production environment. Using all of the capabilities of AWS Step Functions, you can build a complex end-to-end deep learning workflow. You can trigger the workflow when the new data arrives in Amazon S3, start the training job, and deploy the newly trained model. You can make the workflow more robust and transparent by adding notifications and error handling to it. The following workflow diagram is a sample representation of an end-to-end deep learning workflow implemented using AWS Step Functions for retraining and redeployment.

Articles in this issue

Links on this page

view archives of NORDICS20 - Assets - Deep Learning on AWS