How It Works
SoftNAS – How It Works
SoftNAS is deployed as a CentOS, Linux-based virtual machine. It includes a number of major technology building blocks, which are summarized below along with links to more detailed information.
Cloud Data Management Environment (FlexFiles™)
- FlexFiles™ Architect developer environment for data integration and movement
- Robust processor library for filtering and use
- Pre-built and custom templates enable high degree of automation and integration
- Leverages Apache NiFi
High Speed Data Transfer
(UltraFast™ and ObjFast™)
- Patented UltraFast™ technology for high speed global data transfers
- 20x to 100x faster than TCP/IP, overcomes Internet latency/congestion issues
- Patent-pending ObjFast™ technology, high-performance object storage for file systems
- Patent-pending SmartTiers™ technology automates storage tiering to move older data from high-performance block/SSD to low-cost object storage
- Optimization of data resources provides lower TCO and boosted performance
Lift and Shift Migration
(Powered by Apache NiFi)
- Point-and-click wizard to live migrate data to the Cloud, even while still in production
- Continuous sync keeps data up-to-date when migrating to the Cloud or distributing to remote locations
Enterprise-Grade Cloud NAS (SoftNAS Enterprise)
- Enterprise-grade Cloud NAS filer enables applications to connect to Cloud storage services using standard file protocols
- Features cross-zone HA, deduplication, compression, replication, cloning, and encryption and supports block and object storage
- Leverages OpenZFS
ZFS on Linux
- AWS – an AMI is deployed via the AWS Marketplace or AWS Console
- Azure – a VM is deployed via the Azure Marketplace or Azure Resource Manager
- VMware – an OVA file is downloaded and deployed as a VMware VM
Free trials are offered on all three platforms.
Free Developer editions are offered on all three platforms.
The Linux VM has been optimized and security hardened, making it a “virtual appliance” that’s ready to deploy. The software update function ensures the latest SoftNAS and Linux fixes and updates are always available.
The familiar Linux operating system provides an open, extensible and industry-standard platform that’s familiar, proven, mature and very stable – all critically-important attributes for mission-critical deployments. CentOS is the free version of the enterprise-grade RHEL operating system, which was chosen due to its proven reliability and stability. Linux provides standard NFS, CIFS/SMB, iSCSI, AFP, FTP and many other protocols. SoftNAS ships an enhanced version of Samba for CIFS/SMB with Active Direction integration for full ACL support, as well. TCP/IP has been tuned, along with the rest of the virtual appliance, to deliver maximum performance, IOPS and throughput.
ZFS was chosen due to its maturity, stability, data integrity features, high-performance and multi-layer caching, storage snapshots and writable clones for data protection, checksums for additional data integrity assurance, storage efficiency and data reduction features like compression and deduplication and its proven POSIX-compliant filesystem that scales up to Zettabytes.
An administration interface called StorageCenter makes administration of ZFS, Linux and the other major SoftNAS features easy and predictable. StorageCenter is an HTML5 web-based UI that uses REST API calls to a back-end admin server that makes actual administration changes to the appliance. StorageCenter supports strong authentication and has been security audited and approved by large enterprises.
REST API and CLI
Block-Based Replication is provided through patented SnapReplicate™ functionality, which performs block-based replicate-by-exception and synchronization between SoftNAS Cloud NAS nodes. This means that as files change on the ZFS filesystem, changes are captured in storage snapshots as changed data blocks, and only the changed data blocks are replicated. This approach is superior to typical file-based replication, which doesn’t pick up locked files and open files that don’t have up to date file pointer information available. It’s also extremely fast and efficient for constant synchronization.
High Availability Clustering with Automatic Failover
SoftNAS includes two forms of high availability, SNAP HA™ for Object and Block storage, and Dual Controller HA for object storage only. SNAP HA uses SnapReplicate™ to replicate data from a primary node to the secondary node in an HA pair. Dual Controller HA (Object storage only), uses 2 controllers but only one copy of the data as there is no need to replicate the data across nodes because Object storage is already highly-replicated and durable. A floating virtual IP address is used to ensure NAS clients are always connected to the data. Nodes are typically configured across availability zones in the public cloud, or in a VMware fault-tolerant configuration. Automatic failover monitors the health of underlying storage, the ZFS filesystem and network conditions, carefully managing the cluster to ensure applications and workloads remain highly-available, even in the face of underlying cloud platform host, storage or network failures. SoftNAS’ unique, patented high-availability technology, enabling only SoftNAS to provide a No Downtime Guarantee SLA for its customer’s peace of mind in the cloud. Learn more about SoftNAS HA.
SmartTiers Auto-Tiered Storage
SmartTiers Auto-Tiered Storage provides unique, patent-pending block-based space optimization for up to 4 tiers of storage. As shown below in Figure 6, Tier 1 is typically SSD-backed, high-performance block storage. Tier 2 can be other block-based (e.g., magnetic) storage, and Tiers 3 and 4 can be low-cost, highly-durable object storage.
SmartTiers aggregates the tiers transparently into a single ZFS storage pool, then applies data aging and migration rules based upon how frequently and recently a given data block has been accessed. Note that SmartTiers is block-based, not file-based, so portions of a large file that are frequently accessed will be stored on SSD tier 1, while less frequently accessed data blocks of that same file can reside on durable, low-cost object storage. This is how SmartTiers provides the ultimate in price/performance and cost savings.
When we consider the additional cost savings and storage efficiencies by combining SmartTiers with ZFS compression and deduplication, storage space efficiencies of more than 10:1 are readily possible with typical file-based workloads.
Calculate Potential Savings with
SmartTiers Storage Savings Calculator
Lift and Shift Migration and Global File Replication
Point and click file synchronization and file migration using a fill-in-the-blanks wizard makes moving gigabytes to hundreds of terabytes globally between datacenters, into the cloud and between clouds easy, fast and reliable. FlexFiles replication technology ensures reliable delivery of data from NFS and CIFS sources (e.g., existing NAS or file server) into SoftNAS running in the public cloud (e.g., AWS or Azure) or another datacenter (e.g., VSAN on VMware or hyperconverged systems). Lift and Shift performs an initial sync of the data, then repeatedly syncs any changes, making live production migration projects faster and easier, without the issues associated with offline swing gear and other manual export/import methods. Lift and Shift also supports suspend/resume with automatic restart recovery, making very large migration jobs resilient to network issues, taking the frustration of restarting replication or copy jobs out of the migration process.
Lift and Shift can also be used to create permanent replication relationships in point-to-point, one-to-many and many-to-one topologies. And because it leverages the built-in UltraFast data accelerator, it’s now possible to replicate bulk data globally across any kind of network conditions, regardless of latency, congestion and packet loss on dirty or troublesome WAN, Internet, packet radio, cellular, satellite or even DSL connections.
UltraFast™ Global Data Accelerator
UltraFast is patented network and data acceleration technology that overcomes the limitations of TCP/IP when it comes to latency, congestion and packet loss. As shown in Figure 7 below, UltraFast ensures that bulk data transfers and steady streams of data flow globally at the maximum possible throughput, overcoming the challenges associated with latency, congestion and packet loss, without the need to deploy expensive and proprietary hardware-based WAN accelerators or other network optimizers everywhere.
UltraFast intercepts certain TCP/IP traffic as a SOCKS proxy, then encapsulates that traffic in a reliable UDP wrapper using SoftNAS’ patented gambler algorithm, which ensures reliable, fast end-to-end data transfers occur, despite challenging network conditions that render TCP/IP slow and virtually useless.
UltraFast includes the ability to throttle network bandwidth and usage based on time of day to control how much WAN and LAN bandwidth are consumed. It also includes a graphical “speed test” that shows exactly how much traffic acceleration you will get on your specific network and conditions.
Cloud Fabric Architecture with FlexFiles and Apache NiFi
In the figure below, we see an overall block diagram of the major SoftNAS Fabric architecture, along with use cases and out-of-the-box data integrations. Key components of our cloud data fabric are FlexFiles and Apache NiFi. FlexFiles provides rich file-based data integration and data replication capabilities that leverage NiFi as a data orchestration engine.
NiFi additionally provides the “any kind of data” integration layer, along with data transformation, routing, system mediation, provenance capabilities that deliver a unique “chain of custody” for data transfers, ingest and flows. NiFi provides a “universal data translator” that’s fast, reliable and transactional. The drag and drop FlexFiles Architect and NiFi UI makes creation of custom data flows faster and easier than developing everything with code from scratch.
To Learn More about SoftNAS
OR CALL US AT 1-888-801-7524
Introducing the new SoftNAS Platinum Edition.
Accelerate your cloud journey with SoftNAS.
Control your data with SoftNAS and Microsoft Azure.
SoftNAS, Inc. Offices: