Saturday, June 18, 2005

Restoring NT Shares

Well, it inevitably happens that a large fileserver is being replaced, goes down, or goes bad and you have to restore it. Well if you are restoring to the same type of hardware then life is beautiful and NT/2000/2003 has no problems and acts like the good little boy it should be, but what happens when you have to restore to new hardware, or are refreshing the server to a newer more powerful server? This is when NT can be worse than that bratty little kid I wanted to strangle in the movie Problem Child. Personally John Ritter should have shot him and buried him in the back yard, but I digress. So how can you restore Shares or any other piece of the registry with TSM when NT doesn't like having the registry of another machine of different hardware restored? Well that's were a little ingenuity and some preemptive strategy comes in handy. The easiest way is to setup a script using the command line registry tool REG.EXE that runs a REG EXPORT KEYNAME each day and stores it as a file. Here is the string needed to backup the Shares key. (The following is all one line)

REG EXPORT HKLM\SYSTEM\CurrentControlSet\Services\LanmanServer\Shares C:\BACKUP\SHARESEXP.TXT

The command backs up the Shares key to a folder I created on the C: drive called Backup, but you can put it wherever you like. A simple batch file will suffice to execute the export and you can either schedule it through Windows daily or through TSM. Yes, you can do that and TSM will gladly back it up. This allows you to then restore the export file and reimport it into the server of choice. Just make sure that with any schedule where you are executing a command/batch file you put the full path of the file or else TSM can possibly fail to execute it, even if its in the TSM BACLIENT directory.

Ok, not everyone has that process in place and needs to restore the Share key now! Well have no fear TSM can help you accomplish it. Let's look at the process for this situation and I'll provide some links. First off you can restore the registry without activating it. The command to do so is

RESTORE REGISTRY -ACTIVATE=NO (for Win2K/Win2003)

RESTORE REGISTRY ENTIRE -ACTIVATE=NO (for WinNT)

Now for each process you can restore specific keys but for ease of use the entire restore will work. TSM will restore the registery to the c:\adsm.sys\computername\ directory. From this point you can load the hive in RegEdit containing the share key and export it to a file. Once exported you should be able to import it into the current registry thereby restoring the shares. Here is a Microsoft document that covers loading the hive and processes to use. There is also a good document on backing up and restoring the registry here. If you have any questions feal free to comment and I'll respond as soon as possible.

Tuesday, June 14, 2005

TSMManager

Well, we have been using TSMManager for some time now and I have to say I rely on it a lot for day to day monitoring of our servers. Our environment is pretty large so we need something that will allow us to see problems quickly and easily. TSMManager has a slew of tools and the interface is fairly easy to use. We have a number of Jr. level admins who are able to handle multiple TSM servers with TSMManager's help. The key piece of TSMManager is the administrators console. It is a tabbed interface that allows me to see things at a glance without having to query the activity log in search of the problem. The first tab is basically the admin interface running in console mode. The second tab is the greatest gift to admin in that it is an error tab. It parses out the errors and shows all warnings, errors, and severe errors.


This is the error tab. Click on it and you'll see how it consolidates all errors into a simple interface. Great for those times you're having server issues. Posted by Hello

TSMManager (Cont.)

When not checking for errors the majority of your time will be spent using the Admin tab. The admin tab allows for entering in TSM commands and you'll notice down the upper left side of the window there are preset queries you can issue to the TSM server you have selected. Below the preset commands you'll see the list of the servers you have defined the to TSMManager collector and a simple click on the server you wish to monitor and enter commands for switches the ENTIRE console to that machine. It's instantaneous and you can immediately select the other tabs to see what is happening on your server or enter commands to query for yourself. Another nice feature you can see in the picture below is the ability to save commonly used commands and queries and select them from the drop down list whenever you wish to use them.


Here we have the admin command line and stored and preset commands. Posted by Hello

TSMManager is not DB based so it has a few limitations. We have noticed with it monitoring 30 large TSM servers that the collector will have issues with the dsmadmc executables failing (TSMManager opens a dsmadmc on the collector server for each TSM server it is monitoring up to 30 servers per collector). This could be the host server it is running on, but other than that the product has been a godsend to our workload and daily monitoring. I would recommend it for shops with limited TSM skills and those that require the end users ability to monitor backup. It has e-mailing and alert capabilties, individual TSM server reports, consolidated server reports (combines the info for all TSM servers monitored by TSMManager), and even has a DRM/tape management feature for those without DRM. With TSMManagers admin web interface you can monitor you systems and it also allows the end users to log in to a secure client website that will allow them READONLY access to reports and results of server backups. I have had to use it a number of times to make our DBA's and application owners feel more secure with their system's backups and it works very well. There are numerous other features within TSMManager, but just the ones I've mentioned so far make it worth the price of purchase. If you feel like TSM is causing you the onset of early stage dementia check out TSMManager you'll be happy you did.

Monday, June 13, 2005

Just Google It!

Ok, if you have every had a problem with TSM then you probably have wished Tivoli support was a little more responsive. I'll be honest with you, the large majority of my problems have been resolved with two web based resources. The first is search.ADSM.org which has always been a great resource for finding the answers to most problems. The second and probably the least used is Google. When I Google search my problems I am amazed at how often I can find my answer. Sometimes the problems are OS related and not a true TSM issue and Google helps me identify the correct resolution. I will say this, if ADSM.org and Google don't have the solution check the TSM client README for any OS dependent patches since that has bit me in the rear a number of times. Hey, Google is great and its free and its faster at responding than Tivoli support. You might still have to get with support on the REALLY tough stuff but before you go down that road try the alternatives and you'll be greatly surprised.

Saturday, June 11, 2005

The Case For Raw Volumes!

If you are serious about TSM server rebuild times and want the quickest way to get up and running then I suggest you look into raw logical volumes for all your TSM DB, Log, and storage needs. Of course if you are running on NT I can't say I know of any way TSM can use raw, but in our AIX shop we live by raw volumes. The creation time is quick and with a little script I can have my volumes created and ready for the DB restore in no time. I have been down the road of DSMFMT and know how long large volumes can take to create and since TSM does not like more than 16 volumes some older Unix servers can take time to format. The other nice thing about raw volumes is if the server crashes its rare, except for disk failure, for volume corruption to occur. I have had too many dirty super blocks to deal with in my time, and I don't miss them. Remember, all TSM is really doing with DSMFMT is creating a file and in a way converting it back into raw. So why do the extra steps, save yourself some time if you ever are in a true DR situation.

Wednesday, June 08, 2005

TSM 5.3 Server And The Return Of The Web Interface

There has been such an uproar over TSM 5.3 dumping the web interface that IBM/Tivoli quietly released documentation on how to turn it back on for 5.3 servers. The sad thing is that Tivoli in their push to integrate websphere did not look at the problems faced when you take a critical process like backup and turn the tables on system and storage administrators overnight. The truth is that the new interface is clunky and adds time to the whole rebuild process if your people are not capable of using the command line. I have used the ISC but not enough to have a solid verdict. It looks like it has some nice features but by not making it somewhat similar to the old web interface they have only made a lot of people angry. Here is the documentation for turning back on the web interface on 5.3 servers. For now it works, we'll have to see how long they allow it.

Crititcal Windows Filesystem Issues

Well this is a copy from my other less TSM centric blog Storage Admin Blues but it needs to be known. Here is the post from February -

So I have been working a major data issue on a server with a certain Redmond operating system for almost a week now. As it turns out the afore mentioned OS has a known issue with volumes that contain over 4 million files. What happens is that the system will reboot after a patch or update is applied and on the startup begin a check-disk. That wouldn't be so bad but the check-disk strips permissions from almost all files. The response from Redmond was that we would have to restore the data if we want to fix the permissions. OK! Great! Restore 6 million files when the server is used 24/7. The volume in question is over 570GB space used and it has a gig-Ethernet connection. I swear if there isn't a conspiracy against storage administer when it comes to restore SLA's. I got called in to fix the problem and have had almost no sleep for a week, and in conjunction with that I have a virus that is causing me to cough incessantly and make it hard to breathe. Thank goodness for telecommuting or I'd be in the hospital by now. Lets just hope the people in charge listen this time (it has happened two times before this) when we warn about volume size/file management. If it wasn't for Arrested Development and Scrubs I would go nuts.

The resolution for this problem was to restore the directory structures and then have the system admins apply a script that cascaded the permissions to the files within the directory structure since they all inherited their permissions from the parent folder. We also decided to change the environment to backup all directory structures to disk and retain them there as long as possible before migrating to tape. So we had to use the migration delay and migration continue feature on the disk pool. Trust me restoring directories off of tape is no picnic...very slow. If you have run into something like this let me know how you resolved it. Sharing info is how we learn more and although the my e-mail might say TSM Expert, I'm don't know everything.

TSM and Windows Volume Shadow Copy

First off let me just say that the name TSMExpert was more of a joke. I was setting up aliases on Notes Mail and for fun put TSMExpert and viola I got it. Didn't think it would go through but hey as long as I have it I might as well use it. I can be reached at tsmexpert@us.ibm.com and will answer questions as much as possible. Remember TSM is a fickle beast but when setup correctly it is the best enterprise backup/archive tool on the market.

I have been working with many issues on Windows 2003 and the TSM client. We upgraded to 5.2.4 on many of our clients and some to 5.3 due to the shadow copy issues. TSM seems to have serious issues with VSC and gives RC12 on schedules when its implemented. I am not quite sure how VSC is suppose to help when it resides on the same disk as the data its supposed to correct, but I get its more like the snapshot feature on NetApps. I would recommend you upgrade all 2003 servers if you are having this problem to 5.3 since it is supposed to resolve this issue. Also be aware there are problems with older client versions stability and reliability on 2003 servers. We have experienced numerous crashes and System State backup failures with the older clients. This is another reason for upgrading.