Thursday, September 5, 2013

NetApp Usable Space Calculator - new version 2.1

March 15, 2014

-Updated the current version 2.1 with few changes in right sized disks.
Thanks to Cor van der Hulst - a blog visitor who gave some tips on right sized disk values.
- Added some new disks released by NetApp

Same link holds good for the above changes

Download NetApp Usable Space Calculator ver 2.1 (Current Version)

September 5, 2013

NetApp Usable Space Calculator 2.1 - New Version Released (Minor Updates)

Updates in this version are
- Added 2 new disk sizes ( 1.2 TB SAS & 4 TB SATA )

Download NetApp Usable Space Calculator ver 2.1 (Current Version)

Thursday, August 8, 2013

NetApp Usable Space Calculator - new version 2.0

August 8, 2013
NetApp Usable Space Calculator 2.0 - New Version Released

Updates in this version are

- A new feature called Raid Group Size Estimator where you key in disks values, raid type and disk type - the software will attempt to provide the best RG size values either based on NetApp recommendations or Optimal Capacity.
(Please note: The Raid group size input in the calculator is only for disk space calculations and this is ignored for the raid group estimator)
- Fixed a few disk calculation bugs
- A few UI changes
- Misc bug fixes on informational error messages


Download NetApp Usable Space Calculator ver 2.0 (Current Version)

Download NetApp Usable Space Calculator ver 1.2

Download NetApp Usable Space Calculator ver 1.1


Monday, July 15, 2013

NetApp Usable Space Calculator - new version 1.2

NetApp Usable Space Calculator - new version 1.2

July 15, 2013
NetApp Usable Space Calculator 1.2 - New Version Released

Updates in this version are

- Choice of display the output in TB/GB
- Included more different capacity disks and disk types
- Informational message for recommended values of Raid Group Sizes

Download NetApp Usable Space Calculator ver 2.0 (Current Version)

Download NetApp Usable Space Calculator ver 1.2 

Download NetApp Usable Space Calculator ver 1.1


Future Updates: Working on right now is a Raid Group Size Calculator where the software will recommend the raid group size values for a given disk configuration.


As usual please provide feedback and comments

Wednesday, February 13, 2013

NetApp Cluster, HA Pair Difference and Confusion

Personally I always get confused over these terms and with the Cluster Mode of NetApp it becomes even more confusing. This post is just to clarify the basics on the terminology used.

What is a NetApp HA Pair

 An HA pair consists of 2 identical controllers; each controller actively provides data services and has redundant cabled paths to the other controller’s disk storage. If either controller is down for any planned or unplanned reason, its HA partner can take over its storage and maintain access to the data. When the downed system rejoins the cluster, the partner will give back the storage resources.



Olden day meaning of Cluster: The term cluster has been used historically to refer to an HA pair running Data ONTAP 7G or 7-Mode. But not anymore.

What is a NetApp Cluster then:

The term cluster now refers only to a configuration of one or more HA pairs running clustered Data ONTAP.


7- Mode, Cluster Mode and Clustered Data ONTAP:

There are 2 variants of NetApp ONTAP operating systems.

1) 7-mode
2) Cluster Mode or C-Mode or the new name Clustered Data ONTAP.

Thursday, December 13, 2012

Access NetApp filer from Linux host

FILER SIDE STEPS

Enable SSH
First enable SSH on the filer. Enable SSH (preferably version 2 - as it's more secure)
NETAPPFILER> secureadmin enable ssh2
NETAPPFILER> secureadmin setup ssh

Exportfs
Then we need to export the /vol/vol0 on the filer. Edit the /etc/exportfs file to add an entry like this
/vol/vol0       -sec=sys,rw=fqdn of the admin linux host or IP address

If you cant edit the file add an entry via exportfs command
exportfs -p rw=linux hostname /vol/vol0

ON THE LINUX HOST

Listing the exports
LinuxHost:#showmount -e filername
this will list the exports and you should see an entry for the vol0

Create a mount point
LinuxHost:# mkdir -p /mnt/mount1

Mounting the /vol/vol0
LinuxHost:# mount -t nfs NETAPPFILER:/vol/vol0 /mnt/mount1

List the contents
Navigate to /mnt/mount1 and do ls -l and you will see the filer's file system
LinuxHost:/mnt/mount1# ls -la

Monday, August 6, 2012

NetApp Usable Space Calculator ver 1.1

NetApp Usable Space Calculator ver 1.1

I have a new version today for the usable space calculator. Please download from here

NetApp Usable Space Calculator ver 1.1

Updates in this version are
- WAFL Reserve included as a field
- Error Message Indicator when not entered any value
- Minor bug fixes


As usual please provide feedback and comments and refer to my earlier post on the formula stuff

Wednesday, August 1, 2012

NetApp Usable Space Calculator

NetApp Usable Space Calculator ver 1.0

For people who are looking at a quick way to calculate the usable space of a NetApp Filer, I have developed this tool. It is a stand alone executable (Approx 2 MB) which will run on Windows platforms. No Installation needed. I understand there may be bugs in the tool. I am ready for feedback and would like to improve this tool. For now please download and use and let me know. Please use this tool as a estimate and this tool is supplied without any warranty. It may not report accurate sizes.





Aug 8 2013 - NEW VERSION is 2.0 Refer to my post here
NEW VERSION is 1.2. Refer to my post here
Download NetApp Usable Space Calculator ver 1.0 (2.5MB)


Many people have blogged about the usable calculation formula. I am not going to repeat the steps here. Instead I will give some pointers to those blogs. Below info was used in developing this calculator. Feedback recommended.

STEP1: Calculate the disk size first

Basically if you go to bestbuy and pick a 1TB drive, you actually think that you store 1TB of pictures and music. Thats not the truth. Storage hardware is using the base 10 system and software is using the base 2 system. So no storage is actually lost, it is just a question of how the information is represented.
Hard disk manufacturers assume / Base 10 System -   Kilo = 10 power 3
File system assumes / Base 2 System:   Kilo = 2 power 10 = 1024

KB2101,024 bytes
MB2201,048,576 bytes
GB2301,073,741,824 bytes
TB2401,099,511,627,776 bytes

So a 1 TB hard disk is actually
1000,000,000 bytes/ 1,073,741,824  = .9313 x 1000 = 931.3 GB
In a similar way
750 GB hard disk is actually
750,000,000/1,073,741,824  = .6986 x 1000 = 698.6 GB


Base 10 Capacity
Base 2 Capacity
Right Sized Capacity
72GB FC
69 GB
67.9 GB
144GB FC
137.9GB
135.8 GB
300GB SAS
279.4 GB
272 GB
450GB SAS
419.1 GB
415 GB
600GB SAS
558.8 GB
553.26 GB
500GB SATA
465.7 GB
413.9 GB
750 GB SATA
698.6 GB
620.9 GB
1TB SATA
931.3 GB
827.8 GB
2TB SATA
1,862.6 GB
1,655.7 GB
3TB SATA
2,794 GB
2,483.5 GB

Now from the above table you are clear that how a 1TB hard drive has come finally to 931.3 GB. But how did we get the number Right sized capacity 827.8..This is called the checksum calculation which we will see it next.

STEP 2 : Check Sum Calculation

From 931.3 multiply x 8/9 and you will get 827.8 GB. Simple..Why do we multiply the value by 8/9. For this Taylor Riggan @ Waflhouse.com gives a nice explanation. See here - http://waflhouse.com/?p=47

Reproducing from WAFL House website

The second part of this can begin to get a little confusing.  It has to do with error-correction codes and their use on enterprise-class disk arrays such as NetApp.  NetApp uses two types of disk drives – SAS disks and SATA disks.  The manufacturers of these disks typically create SAS disks for business use and SATA disks for both business and home user (consumer) use.  Given that fact, they include some additional functionality on SAS disks to allow enterprise-class disk subsystems to consistently validate the consistency of data on disks.  They do this through something called a checksum (nothing more than a hash value that they can reference to validate data consistency).  That checksum, on SAS disks, is stored in the same sector as data (over simplifying here, but this is close).  The sector size on SAS disks is large enough to incorporate this (520 bytes per sector) and, as such, you really don’t see a reduction in capacity on the drives after doing the base-10 to base-2 sizing conversion.  SATA disks is where it gets a bit fuzzy.  Home users don’t necessarily need checksums.  So, disk manufacturers don’t include that extra space within the sectors to accomodate checksums (512 bytes per sector).  Well, what do enterprise-class storage vendors do in order to use checksums on SATA disks?  They use up an extra 9th sector for every 8 sectors used!  This eats away SATA usable capacity much faster than SAS capacity.  When using SATA disks, you have to take the size you get from the  base-10 to base-2 sizing conversion and multiply that by 8/9. Back to that 1TB SATA disk that we were using as an example, after base-10 to base-2 conversion we were left with 931.32GB.  After checksum allocation, we’re left with  827.8GB of usable capacity

So a ITB hard disk leaves us with 827.8 GB of usable capacity so far.

STEP3: RAID GROUP DETERMINATION

Again I am not going to dwell on this much and going to give reference to cfheoh article @ http://storagegaga.wordpress.com/2011/10/20/playing-with-netapp-final-usable-capacity/

Reproducing From Storagegaga.com

RAID group is important because it is used to balance a few considerations
 •Performance in recovery if there is a disk reconstruction or resilvering
 •Combined RAID performance and availability through a Mean Time Between Data Loss (MTBDL) formula

Different ONTAP versions (and also different disk types) have different number of HDDs to constitute a RAID group. For ONTAP 8.0.1, the table below are its recommendation.
 

So, given a large pool of HDDs, the NetApp storage administrator has to figure out the best layout and the optimal number of HDDs to get to the capacity he/she wants. And there is also a best practice to set aside 2 HDDs for a RAID-DP configuration with every 30 or so HDDs. Also, it is best practice to take the default recommended RAID group size most of the time.

I would presume that this is all getting very confusing, so let me show that with an example. Let’s use the common 2TB SATA HDD and let’s assume the customer has just bought a 100 HDDs FAS6000. From the table above, the default (and recommended) RAID group size is 14. The customer wants to have maximum usable capacity as well. In a step-by-step guide,
 1.Consider the hot sparing best practice. The customer wants to ensure that there will always be enough spares, so using the rule-of-thumb of 2 HDDs per 30 HDDs, 6 disks are set aside as hot spares. That leaves 94 HDDs from the initial 100 HDDs.
 2.There is a root volume, rootvol, and it is recommended to put this into an aggregate of its own so that it gets maximum performance and availability. To standardize, the storage administrator configures 3 HDDs as 1 RAID group to create the rootvol aggregate, aggr0. Even though the total capacity used by the rootvol is just a few hundred GBs, it is not recommended to place data into rootvol. Of course, this situation cannot be avoided in most of the FAS2000 series, where a smaller HDDs count are sold and implemented. With 3 HDDs used up as rootvol, the customer now has 91 HDDs.
 3.With 91 HDDs, and using the default RAID group size of 14, for the next aggregate of aggr1, the storage administrator can configure 6 x full RAID group of 14 HDDs (6 x 14 = 84) and 1 x partial RAID group of 7. (91/14 = 6 remainder 7). And 84 + 7 = 91 HDDs.
 4.RAID-DP requires 2 disks per RAID group to be used as parity disks. Since there are a total of 7 RAID groups from the 91 HDDs, 14 HDDs are parity disks, leaving 77 HDDs as data disks.

This is where the rightsized capacity comes back into play again. 77 x 2TB HDDs is really 77 x 1.69TB = 130.13TB from an initial of 100 x 2TB = 200TB.

If you intend to create more aggregates (in our example here, we have only 2 aggregates – aggr0 and aggr1), there will be more consideration for RAID group sizing and parity disks, further reducing the usable capacity.


STEP4: SPARE DISKS

Deduct how many spares you plan to allocate.

Below is recommended from Netapp

On NetApp storage, disk failures automatically trigger parity reconstructions of affected data onto a hot standby (spare) disk, assuming that a spare disk is available. If no spare disks are available, self-healing operations are not possible. The system will run in degraded mode (requests for data on the failed disk are satisfied by reconstructing the data using parity information) until a spare is provided or the failed disk is replaced. During this time, your data is at greater risk should an additional failure occur. (With NetApp RAID-DP™, a RAID group operating in degraded mode can undergo one additional disk failure without suffering data loss.)
The number of spares you need varies based on the number of disk drives attached to your storage system. For a lower-end FAS200 or FAS2000 with a single shelf, one spare disk may suffice (configure two if you want to use Maintenance Center). On the FAS6080, with a maximum spindle count of 1,176 disks, more spare disks are needed to ensure maximum storage resiliency, especially with larger SATA disks that have longer reconstruction times.
NetApp recommends using two spares per disk type for up to 100 disk drives, where disk type is determined by a unique interface type (FC, SATA, or SAS), capacity, and rotational speed. For instance, if you have a system with 28 300GB 15K FC disks and 28 144GB 15K FC disks, you should provide four spares: two of the 300GB capacity and two of the 144GB capacity.

STEP5: Root Volume Consideration

Root volume contains all configurations files (ONTAP Files). It is always preferred to separate the root volume from the data volumes on the filer. For optimal management, it is better to restrict the root volume to just 2 disks (minimum required for a RAID-4 parity group). This enables faster reconstruction times in case of a disk failure on the root volume. Deduct how many disks you have allocated for the root volume alone.

STEP6: Volume Snapshot Reserve

Default snapshot space reserve is 20% of the volume size.
This variable is set at the volume level and is set as a percentage of the volume. Data ONTAP removes the defined percentage of volume from being available for configuring LUNs or for file usage with CIFS or NFS. As Snapshot copies need space, they consume space in the snap reserve area. By default after the snap reserve area is filled, the Snapshot copies start to take space from the general volume.

STEP7: Aggregate Reserve
Aggregates are created by default with 5% reserve space. Some people leave this to 0 and set the reserve on the volume level.

STEP8: WAFL Overhead
WAFL reserves approximately 10% of space for performance reasons, and for some part of the metadata. This is available in the new version 1.1


(A ton of thanks to my friend Guru for his help and support in the development of this tool)
Murugappan