NORDICS20 - Assets

Deep Learning on AWS

Amazon Web Services Resources EMEA

Issue link:

Contents of this Issue


Page 24 of 50

Amazon Web Services Deep Learning on AWS Page 20 AWS DL Containers are available in AWS Marketplace or from within the Amazon ECS console. AWS DL Containers version 2.0 for TensorFlow Docker images have been tested with Amazon SageMaker, Amazon EC2, Amazon ECS, and Amazon EKS. One Docker image can be used across multiple platforms on AWS. Networking Enhanced Networking Enhanced networking uses single root I/O virtualization (SR-IOV) to provide high- performance networking capabilities on supported instance types. SR-IOV is a method of device virtualization that provides higher I/O performance and lower CPU utilization when compared to traditional virtualized network interfaces. Enhanced networking provides higher bandwidth, higher packet per second (PPS) performance, and consistently lower inter-instance latencies. Most of the instance types that are used in deep learning support an Elastic Network Adapter (ENA) for enhanced networking. The ENA was designed to work well with modern processors, such as those found on C5, M5, P3, and X1 instances. Because these processors feature a large number of virtual CPUs (128 for X1), efficient use of shared resources like the network adapter is important. While delivering high throughput and great packet per second (PPS) performance, ENA minimizes the load on the host processor in several ways and also does a better job of distributing the packet processing workload across multiple vCPUs. Here are some of the features that enable this improved performance: • Checksum Generation – ENA handles IPv4 header checksum generation and TCP/UDP partial checksum generation in hardware. • Multi-Queue Device Interface – ENA uses multiple transmit and receive queues to reduce internal overhead and to improve scalability. The presence of multiple queues simplifies and accelerates the process of mapping incoming and outgoing packets to a particular vCPU. • Receive-Side Steering – ENA can direct incoming packets to the proper vCPU for processing. This technique reduces bottlenecks and increases cache efficacy. All of these features are designed to keep as much of the workload off of the processor as possible and to create a short, efficient path between the network packets and the vCPU that is generating or processing them.

Articles in this issue

Links on this page

view archives of NORDICS20 - Assets - Deep Learning on AWS