AWS Backend Storage Selection Guide
SoftNAS Cloud NAS supports the broadest range of backend storage options on Amazon Web Services,™ including Simple Storage Service (S3) and Elastic Block Store (EBS). It’s important to select the right backend storage to align with the performance requirements of your application. General guidance is provided below.
Step 1: Understanding AWS Storage Types
In general, Amazon offers two types of native storage services that are used with SoftNAS products: Simple Storage Service (S3), and Elastic Block Store (EBS) . To determine the most appropriate storage type, you will need to know the IOPS requirements of your application. EBS magnetic or SSD backed storage along with S3 object storage have benefits and limitations that work with or against certain workloads.
What is Amazon EBS?
Amazon EBS is is a block level storage type that runs on Amazon EC2. There are three types of EBS volumes: General Purpose, Provisioned IOPS and Magnetic Volumes. There are differences in total throughput capabilities between Provisioned IOPS (io1) and General Purpose SSD (gp2) volumes. Io1 volumes are designed to provide up to 320 MB/second of throughput while gp2 volumes are designed to provide up to 160 MB/second.
What is Amazon S3?
Amazon S3 is a service that provides object storage, which is a type of storage where data is stored in a flat environment without a hierarchy. S3 is highly scalable with high durability and low cost, but S3 performance is generally slower than that of Amazon EBS. Using Amazon S3, SoftNAS Cloud NAS can scale cloud storage to practically unlimited capacity. You can provision each cloud disk to hold up to four petabytes (PB) of data. If a larger data store is required, you can use RAID to aggregate multiple cloud disks.
Step 2: Choose Between EBS and S3 Storage
S3 is usually chosen for applications that do not require heavy read/write operations. However, with SoftNAS® ObjectBacker™ Backend Storage Acceleration with patent-pending ObjFast™ adaptive flow technology, SoftNAS Cloud NAS can write data up to 400% faster than before, making cost-effective S3 storage a viable option for write intensive use cases that want to take advantage of lower cost S3 storage. S3 storage with SoftNAS ObjectBacker is appropriate for write sensitive workloads, including:
- Archival and Backup
- Lift and Shift Data Migrations
- Large-scale Content Publishing
When you use a cloud disk, we suggest using an ephemeral or block device local to the SoftNAS Cloud NAS virtual appliance as a read cache to reduce Amazon S3 I/O charges and improve IOPS and performance for read-intensive workloads. S3 cloud disks are suitable for archive or general access applications where low cost is important. They are not suitable for applications that need high IOPS performance and very high throughput on random read/write workloads where EBS cloud disks are more appropriate. The charts below show IOPS and Throughput performance increases for NFS, CIFS/SMB and iSCSI workloads using SoftNAS ObjectBacker.
AWS CIFS Performance
AWS NFS Performance
AWS iSCSI Performance
Step 3: Choose an EBS Volume Type
If EBS is appropriate for your performance requirements, SoftNAS recommends these general guidelines for selecting an EBS volume type.
General Purpose (SSD backed) Volumes (gp2)
General Purpose Volumes are backed with Solid State Drive (SSD) and are designed for a broad range of purposes. The baseline performance of 3 IOPS/GB and a possibility to burst up to 10,000 IOPS makes them a good fit for applications that need a lot of read and write operations, like Microsoft SQL. General Purpose (gp2) and provisioned IOPS (io1) SSD volumes can achieve faster IOPS performance and very high throughput on random read/write workloads when compared to magnetic disks, but at a higher price point. However, gp2 and io1 volume types are limited to a throughput of 320 MB/s (160 MB/s for gp2, 320 MB/s for io1). General Purpose (gp2) volumes provide a fixed 1:3 ratio between gigabytes and IOPS provisioned, so a 100 GB General Purpose volume provides a baseline of 300 IOPS. Gp2 volumes less than 1TB in size can also burst for short periods, up to 3,000 IOPS. You can provision General Purpose volumes up to 16 TB and 10,000 IOPS.
Provisioned IOPS (SSD) Volumes (io1)
Provisioned IOPS are backed with the same SSD as General Purpose Volumes, but Provisioned IOPS are designed for heavy workloads from 30 IOPS/GB up to 20,000 IOPS. For even more demanding workloads, multiple provisioned IOPS volumes can be striped, ensuring up to 48,000 IOPS or 800 MBps of throughput. Provisioned IOPS (io1) volumes are intended for workloads that demand consistent performance, such as relational (SQL) databases. You can create provisioned IOPS volumes up to 16 TB and 20,000 IOPS. Over a year, Amazon EBS Provisioned IOPS volumes are designed to deliver within 10 percent of the provisioned IOPS performance 99.9 percent of the time.
Magnetic Volumes (st1 and sc1)
Magnetic Volumes are a low cost volume type that can be used with testing and development environments on Amazon EC2. They can also be used with applications that don’t require a lot of read/write operations. Magnetic-backed volume types support higher block sizes up to 1,024 KB. Throughput Optimized HDD (st1) and Cold HDD (sc1) Amazon EBS volume types are based on magnetic storage technology.
Throughput Optimized HDD Volumes (st1)
The Throughput Optimized HDD (st1) volume type is designed for sequential read/write workloads (e.g., Big Data). It can achieve very high throughput (500 MB/s) for sequential read/write workloads (compared to 160 MB/s and 320 MB/s for SSD-backed gp2 and io1, respectively). Generally, Big Data workloads operate on very large sequential datasets and generate data for storage in a similar way. The st1 volume type has a baseline performance of 40 MB/s per terabyte (TB) of allocated storage and, like gp2, can burst beyond the baseline performance for a short period of time.
Cold HDD Volumes (sc1)
The Cold HDD (sc1) volume type is designed for high density and infrequent access workloads. This volume type is suitable for cold storage (infrequent access) applications where low cost is important. Unlike st1, the baseline performance of an sc1 volume is 12 MB/s per TB of allocated storage. It’s important to note that Amazon S3 achieves high availability (HA) by default within a single region, whereas sc1 volumes have to be mirrored across Availability Zones to achieve parity with Amazon S3 in durability and availability of the data. (This doubles and triples the cost of sc1 when compared to Amazon S3.) Nevertheless, depending on certain access patterns (e.g., cold versus warm) of the data, the cost of sc1 volumes can be cheaper for certain workloads.
Introducing the new SoftNAS Platinum Edition.
Accelerate your cloud journey with SoftNAS.
Control your data with SoftNAS and Microsoft Azure.
SoftNAS, Inc. Offices: