Posts Tagged 'Block Storage'

October 29, 2015

How to measure the performance of striped block storage volumes

To piggyback on the performance specifications of block and file storage offerings, SoftLayer provides a high degree of volume size and performance combinations for your storage needs. But what if your storage performance or size requirements are much more specific than what is currently offered?

In this post, I’ll show you to configure and validate a sample RAID 0 configuration with:

  1. The use of LVM on CentOS to create a RAID 0 array with 3 volumes
  2. The use of FIO to apply IO load to the array
  3. The ability to measure throughput of the array

Without going into potential drawbacks of RAID 0, we should be able to observe the benefits of up to three times the throughput and size of any single volume. For example, if we needed a volume with 60GB and 240IOPS, we should be able to stripe three 20GB volumes each at 4 IOPS/GB. You can also extrapolate the benefits from this example to fit a range of performance and reliability requirements.

To start, we will provision 3x 20GB Endurance volumes at 4 IOPS/GB and make it accessible to our CentOS VM but stop short of creating a file system; e.g., you should stop once you are able to list three volumes with:

# fdisk -l | grep /dev/mapper
Disk /dev/mapper/3600a09803830344f785d46426c37364a: 21.5 GB, 21474836480 bytes, 41943040 sectors
Disk /dev/mapper/3600a09803830344f785d46426c373648: 21.5 GB, 21474836480 bytes, 41943040 sectors
Disk /dev/mapper/3600a09803830344f785d46426c373649: 21.5 GB, 21474836480 bytes, 41943040 sectors

Then proceed to create the three-stripe volume with the following commands:

# pvcreate /dev/mapper/3600a09803830344f785d46426c37364a /dev/mapper/3600a09803830344f785d46426c373648 /dev/mapper/3600a09803830344f785d46426c373649
 
# vgcreate new_vol_group /dev/mapper/3600a09803830344f785d46426c37364a /dev/mapper/3600a09803830344f785d46426c373648 /dev/mapper/3600a09803830344f785d46426c373649
 
# lvcreate -i3 -I16 -l100%FREE -nstriped_logical_volume new_vol_group

This creates a logical volume with three stripes (-i) and stripe size (-I) of 16KB with a volume size (-l) of 60GB or 100 percent of the free space.

You can now create the file system on the new logical volume, create a mount point, and mount the volume:

# mkfs.ext3 /dev/new_vol_group/striped_logical_volume
# mkdir /mnt
# mount /dev/mapper/new_vol_group-striped_logical_volume /mnt

Now download, build, and run FIO:

# yum install -y gcc libaio-devel
# cd /tmp
# wget http://freecode.com/urls/3aa21b8c106cab742bf1f20d60629e3f
# tar -xvf 3aa21b8c106cab742bf1f20d60629e3f
# cd fio-2.1.10/
# make
# make install
# cd /mnt
# fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=16k --iodepth=64 --size=1G --readwrite=randrw --rwmixread=50

This will execute the benchmark test at 16KB blocks (--bs), random sequence (--readwrite=randrw), at 50 percent read, and 50 percent write (rwmixread=50). This will run 64 threads (--iodepth=64) until the test file of 1GB (--size=1G) is size is completed.

Here is a snippet of output once completed:

read : io=51712KB, bw=1955.8KB/s, iops=122, runt= 26441msec
write: io=50688KB, bw=1917.3KB/s, iops=119, runt= 26441msec

This shows that throughput is rated at 122r + 119w = ~240 IOPS. To validate that it is what we expect, we provisioned 3x 20 GB x 4 IOPS/GB = 3 x 80 IOPS = 240 IOPS.

Here is a table showing how results would differ if we tuned the load with varying block sizes (--bs) :

As you can see from the results, you may not observe the expected 3x throughput (IOPS) in every case, so please be mindful of your logical volume configuration (stripe size) versus your load profile (--bs). Please refer to our FAQ for further details on other possible limits.

-Nam

April 1, 2015

The SLayer Standard Vol. 1 No. 8

The week in review. All the IBM Cloud and SoftLayer headlines in one place.

Sunny Skies for IBM Cloud and The Weather Company
IBM made big headlines on Tuesday when it announced they would team up with The Weather Company boasting “100 percent chance of smarter business forecasts.”

Bloomberg sits down with Bob Picciano, IBM Analytics Senior VP, and David Kenny, The Weather Company CEO to discuss what makes this different than other companies that have analyzed the weather in the past. Using Watson Analytics and the Internet of Things, the partnership will transform business decision-making based on weather behavior. Read how IBM’s $3 billion investment in the Internet of Things will collect weather data from 100,000 weather stations around the world and turn it into meaningful data for business owners.

Indian Startups Choose SoftLayer
According to the National Association of Software and Services Companies (NASSCOM), India has the world’s third largest and the fastest-growing startup ecosystem. Like many SoftLayer startup customers, Goldstar Healthcare, Vtiger, Clematix, Ecoziee Marketing utilize the SoftLayer cloud infrastructure platform to “begin on a small scale and then expand rapidly to meet workload demands without having to worry about large investments in infrastructure development.”

New SoftLayer Storage Offerings
Last week, SoftLayer announced the launch of block storage and file storage complete with Endurance- and Performance-class tiers. The media was fast to report the new offerings that provide customers more choice, flexibility, and control for their storage needs and workloads.

“ … SoftLayer’s focus on tailored capacity and performance needs coincides with the trend in the cloud market of customizing technology based on different application requirements.”– IBM Splits SoftLayer Cloud Storage Into Endurance, Performance Tiers

“In the age of the cloud, the relationship between cloud storage capacity and I/O performance has officially become divorced.” – IBM Falls Into Cloud Storage Pricing Line

Pick your favorite online tech media and read all about it: SiliconANGLE, Computer Weekly, Data Center Knowledge, CRN, V3, Cloud Computing Intelligence, Storage Networking Solutions UK, and DCS Europe.

#IBMandTwitter
There are more than half a billion tweets posted to Twitter every day. IBM is teaming up with Twitter to turn those “tweets into insights for more than 100 organizations around the world.” Leon Sun of The Motley Fool takes a closer look at what the deal means to IBM and Twitter.

“Twitter provides a powerful new lens through which to look at the world. This partnership, drawing on IBM’s leading cloud-based analytics platform, will help clients enrich business decisions with an entirely new class of data. This is the latest example of how IBM is reimaging work.” – Ginni Romety, IBM Chairman, President and CEO

-JRL

Categories: 
March 25, 2015

Introducing New Block Storage and File Storage

Everyone knows data growth is exploding. The chart below illustrates data growth—in zettabytes—over the last 11 years.

Storing all that data can get complicated. The rise of cloud computing and virtualization has led to myriad options for data storage. Kevin Trachier did a great job of defining and highlighting the differences in various cloud storage options in his blog post, Which storage solution is best for your project?

Today, I’m excited to announce that we’ve expanded SoftLayer’s cloud storage portfolio to include two new storage products: block storage and file storage, both featuring Performance and Endurance options. These storage offerings allow you to create storage volumes or shares and connect them to your bare metal or virtual servers using either NFS or iSCSI connectivity.

The Endurance and Performance classes of both block storage and file storage feature:

  • Storage sizes to fit any application—from 20GB to 12TB
  • Highly available connectivity—redundant networking connections reduce risk and mitigate against unplanned events to provide business continuity
  • Allocated IOPS—meet any workload requirement through customizable levels of IOPS that are there when you need them
  • Durable and Resilient —infrastructure provides safety of mind against data loss without managing system-level RAID arrays
  • Concurrent Access—multiple hosts can simultaneously access both block and file volumes in support of advanced use cases such as clustered databases

The Endurance class of both block storage and file storage is available in three tiers, allowing you can choose the right balance of performance and cost for your needs:

  • 0.25 IOPS per GB is designed for workloads with low I/O intensity. Example applications include storing mailboxes or departmental level file shares.
  • 2 IOPS per GB is designed for most general purpose use. Example applications include hosting small databases backing Web applications or virtual machine disk images for a hypervisor.
  • 4 IOPS per GB is designed for higher intensity workloads. Example applications include transactional and other performance-sensitive databases.

All Endurance tiers support snapshots and replication to remote data centers.

We designed the Performance class of both block storage and file storage to support high I/O applications like relational databases that require consistent levels of performance. Block volumes and file shares can be provisioned with up to 6,000 IOPS and 96MB/s of throughput.

Available sizes and IOPS combinations:

Block storage and file storage are available in SoftLayer data centers worldwide. SoftLayer customers can log in to the customer portal and start using them today.

-Michael

November 11, 2014

Which storage solution is best for your project?

Before building applications around our network storage, here’s a refresher on what network storage is, how it is used, the different types available, and the best uses for each.

What is network storage? Why would you use it?

Appropriately named, network storage is storage attached to a server over our network; not to be confused with directly attached storage (DAS), which is a hard drive located in the server (or connected with a device like a SCSI or USB cable). Although DAS transfers data to a server faster than network storage due to network latency and system caching, there is still a strong place for network storage.

Many different servers can access network storage, and with some network storage solutions, more than one server can get data from the same shared storage volume simultaneously. This comes in handy if one server dies, because another can pick up a storage device and start where the first left off.

With DAS, planned downtime for server upgrades, potential data loss, and provisioning larger or more servers can slow down productivity. The physical constraints of internal drives and costs associated with servers do not affect network storage.

Because SoftLayer manages the disk space of our network storage products, there’s no need to worry about rebuilding a redundant array of inexpensive disks (RAIDs) or failed disks. If a disk fails, SoftLayer automatically replaces it and rebuilds the RAID—in most cases you would be unaware that the changes occurred.

Select network storage solutions are available with tools for your important data. Schedule snapshots of your data, promote snapshots to full volumes, or reset your data to the snapshot point.

And with network storage, downtime is minimal. Disaster recovery tools available on select storage solutions let you send a command to quickly fail over to a different data center so you can access your data if our network is ever down in a data center.

Types of Network Storage And How They Are Different

Storage Area Network (SAN) or Block Storage

Block storage works like DAS, just remotely—only a single server can access a block storage volume at a time. Using an Internet small computer system interface (iSCSI) protocol over a secure transmission control protocol/Internet protocol (TCP/IP) connection, SoftLayer's block storage has excellent features for backup and disaster recovery, and adding snapshot schedules and failover redundancy make it a powerful enterprise solution.

Network Attached Storage (NAS) or File Storage

File storage acts like a remote file system. It has a slim operating system that allows servers to treat it like a remote directory structure. Multiple servers can share files on the same storage simultaneously. Our new consistent performance storage lets you share files quickly and easily using a network file system (NFS) with your choice of performance level and secure connections.

We also have a common Internet file system (CIFS) (Windows), which requires a credential that grants access to any server on our private network. File storage can only be accessed by SoftLayer servers.

Object Storage

Object storage is a standalone storage entity with its own representational state transfer (REST) API that grants applications (not operating systems) access to the files stored there. Located on a public network, servers in any of our data centers can directly access files stored there. Object storage is different in the way those files are stored as well. In object storage there is not a directory structure, but instead metadata tags are used to categorize and search for files. In conjunction with a content delivery network (CDN), you can quickly serve files to your users or to a mobile device in close proximity.

With pay-as-you-go pricing, you don’t have to worry about running out of space. We only charge based on the greatest usage in any given day. That means you can get started right now for free!

Which storage solution is best for your project?

If you are still confused about which network storage option you should build your applications around, take this eight-question quiz to find out if object, file or block storage will work best for you:

-Kevin

Subscribe to block-storage