Tuesday, January 30, 2007

TSMExpress Review Wanted!

TSMExpert is looking for a TSMExpress review with comments and opinions on how well it works and ease of implementation. If anyone would like to post a review please contact me so I can add you as a contributor if you are not already listed.

Wednesday, January 10, 2007

I Want My 5.4!

So it's the new year and I haven't heard if TSM 5.4 is still on schedule for release this month? Anyone know? I want some of those features like better restore from backup sets and offsite capabilities for NAS. If anyone has inside information I would love to hear it.

That's Funny!

So my friend Ron L. from California sent me an e-mail and asked me if I was on TSMManager's payroll. I said, "What are you talking about?" He stated my name is in the manual for the product and included a snapshot from it. So I downloaded the TSMManager manual to make sure it wasn't some photoshop joke, and to my surprise he is right. Albeit it is at the end but us schills take what we can get. I thought it was quite funny and somewhat of an honor since I have been touting their product for a number of years and even more so since the ISC debacle. To answer Ron's question, "No I am not on the payroll!" Although a lifetime license to use TSMManager would be nice. :-D



Sunday, January 07, 2007

ANS1035S (RC406) Options file '*' could not be found.

Came across this the other day whilst installing TSM for Enterprise Resource Planning on an AIX 5L server:
Data Protection for mySAP.com(R) technology

Interface between SAPDBA Utilities and Tivoli Storage Manager

- Version 3, Release 3, Modification 10 for AIX LF 64-bit -

Build: 219 compiled on Dec 4 2003(c) Copyright IBM Corporation, 1996, 2003, All Rights Reserved.
BKI5014E: Tivoli Storage Manager Error:

ANS1035S (RC406) Options file '*' could not be found.


BKI5014E: Tivoli Storage Manager Error:ANS1035S (RC406) Options file '*' could not be found.

For a while now I have been symlinking the configuration files back to a central repository to make administration easier, so checking the symlinks is the most obvious place to start:
# ls -ltr /usr/tivoli/tsm/tdp_r3/ora64 | egrep "dsm.opt|dsm.sys"

lrwxrwxrwx 1 root system 16 Nov 20 14:34 dsm.sys -> /var/tsm/dsm.sys

lrwxrwxrwx 1 root system 20 Nov 20 14:35 dsm.opt -> /var/tsm/backint.opt

Does backint.opt exist and is it valid?
# cat /var/tsm/backint.opt

SErvername TSM_BACKINT

* TRACEFILE /tmp/trace.out

* TRACEFLAGS api api_detail verbinfo verbdetail timestamp

What about DSM.SYS?
# cat /var/tsm/dsm.sys
SErvername TSMPRD

COMMmethod TCPip

TCPPort 1500

TCPServeraddress

passwordaccess generate

resourceutilization 2

inclexcl /var/tsm/tsm.include

schedlogname /var/tsm/log/dsmsched.log

schedlogretention 14

errorlogname /var/tsm/log/dsmerror.log

errorlogretention 14

queryschedperiod 2

schedmode prompted

ManagedServices Schedule WebClient
SErvername TSM_BACKINT

COMMmethod TCPip

TCPPort 1500

TCPServeraddress

TCPNodelay yes

TCPBuffSize 32

TCPWindowSize 64

LargeCommBuffers yes

TXNByteLimit 25600

passwordaccess prompt

resourceutilization 2

* inclexcl /var/tsm/backint.include

* schedlogname /var/tsm/log/dsmsched.log

schedlogretention 31

errorlogname /oracle/DEV/sapbackup/backint_error.log

errorlogretention 14

queryschedperiod 2

schedmode prompted

So everything was looking to be configured correctly, so I attempted to turn on tracing on the TSM API but as the TDP could not find the option file it would not work.
The next logical step was to trace the client and figure out where it was dieing:
# truss brbackup -u / -c force -t offline_force -m all -p initDEV.sap -a -c force -p initDEV.sap -cds
execve("/usr/tivoli/tsm/tdp_r3/ora64/backint", 0x2FF22CCC, 0x2FF22CE4) argc: 5

kload("/usr/lib/nls/loc/en_US__64", 128, "") = 4026536384

kload("libi18n.a(shr_64.o)", 384, "/usr/ccs/lib") = 4026536536

access("/usr/lib/nls/msg/en_US/backintv3.cat", 0) = 0

open("/usr/lib/nls/msg/en_US/backintv3.cat", O_RDONLY) = 3

open("/usr/tivoli/tsm/client/api/bin64/en_US/dsmclientV3.cat", 04) = 0

access("/usr/tivoli/tsm/client/api/bin64/en_US/dsmclientV3.cat", 04) = 0 access("/usr/tivoli/tsm/client/api/bin64/en_US/dsmclientV3.cat", 04) = 0open("/usr/tivoli/tsm/client/api/bin64/dsm.sys", O_RDONLY|O_LARGEFILE) Err#2 ENOENT

Bingo - as it turns out I had forgotten to also create my symlinks to the configuration files in the "client/api/bin64" directory.

Tivoli Storage Manager 5.2 and Capacity on Demand gotcha.

A customer recently commissioned a new frame for their 3584 library and installed some new tape drives. TSM was left running under the assumption that paths and drives could be taken offline whilst work was performed on the library to ensure availability to backup to the disk storage pools.

Unfortunately this is not the case when you are commissioning additional storage slots (frames or capacity on demand).

The first indication that something was astray was that after the paths and drives online was that the the inability to check in any more tapes as the library was still showing that it was full.

The new frame allowed for over 400 more slots so this also didn’t make sense. After some thought I remembered the ’show slots’ command. After running the command I noticed that I was only being shown 704 slots rather then ther full number of slots I knew that were available.

The diagnostics and administration tool "tapeutil" showed the new and updated element range which confirmed that the configuration from an operating point of view was correct.

So the problem obviously was to do with TSM:

tsm:> show slots 3584atl

PVR slot information for library L32ATL.
Library : L32ATL
Product Id : 03584L32
Support module : 4
Mount
count : 5
Drives : 16
Slots : 704
Changers : 2
Import/Exports :
10
Device : /dev/smc0
Drive 0, element 257
Drive 1, element 258
Drive 2, element 259
Drive 3, element 260
Drive 4, element 261
Drive 5, element 262
Drive 6, element 263
Drive 7, element 264
Drive 8, element 265
Drive 9, element 266
Drive 10, element 267
Drive 11, element 268
Drive 12, element 269
Drive 13, element 270
Drive 14, element 271
Drive 15, element 272

Changer 0, element 1
Changer 1, element 2

ImpExp 0, element number 769
ImpExp 1,
element number 770
ImpExp 2, element number 771
ImpExp 3, element number
772
ImpExp 4, element number 773
ImpExp 5, element number 774
ImpExp
6, element number 775
ImpExp 7, element number 776
ImpExp 8, element
number 777
ImpExp 9, element number 778

slot element range 1025 -
1728

After some research, I discovered that the element count is only queried once upon TSM startup and it is not possible to update the this information without restarting TSM.

tsm:> show slots 3584atl

PVR slot information for library L32ATL.
Library : L32ATL
Product Id : 03584L32
Support module : 4
Mount
count : 8
Drives : 20
Slots : 1130
Changers : 2
Import/Exports :
10
Device : /dev/smc0
Drive 0, element 257
Drive 1, element 258
Drive 2, element 259
Drive 3, element 260
Drive 4, element 261
Drive 5, element 262
Drive 6, element 263
Drive 7, element 264
Drive 8, element 265
Drive 9, element 266
Drive 10, element 267
Drive 11, element 268
Drive 12, element 269
Drive 13, element 270
Drive 14, element 271
Drive 15, element 272
Drive 16, element 273
Drive 17, element 274
Drive 18, element 275
Drive 19, element 276

Changer 0, element 1
Changer 1, element 2

ImpExp 0, element
number 769
ImpExp 1, element number 770
ImpExp 2, element number 771
ImpExp 3, element number 772
ImpExp 4, element number 773
ImpExp 5,
element number 774
ImpExp 6, element number 775
ImpExp 7, element number
776
ImpExp 8, element number 777
ImpExp 9, element number 778

slot element range 1025 - 2154

ANR0380W The database buffer pool recovered from an overcommit of 867 pages. Consider increasing your buffer pool size by 3468 kilobytes

This error occurs when the BUFPOOL is incorrectly sized or SELFTUNE is enabled and the server has been recently restarted - Check the following:
  • Is SELFTUNEBUFPOOLSIZE YES in use in the dsmserv.opt?

    tsm:> query opt
  • What is the default configured BUFPOOLSIZE?

    tsm:> query opt
  • What is the the current BUFPOOLSIZE and is SELFTUNE enabled?

    tsm:> query db f=d

    Multiply the buffer pool pages by 4096 and then divide by 1024.
On AIX if BUFPOOL is currently set to less than 10% increase the value by editing the dsmserv.opt, otherwise increase BUFPOOL by the amount specificied (3468K) and then restart the server.

Friday, January 05, 2007

Bad Friday!

Word to the wise! Whenever defining an immediate action schedule be VERY careful not to do what I accidentally did....

DEFINE CLIENTACT

That's all I typed when I accidentally hit return and it issued an immediate action schedule for EVERY client in EVERY domain! Now I think I was aware of this, but I must have forgotten as I repeatedly yelled "CRAP!" as I saw the define results scroll across the screen. Good thing I have a handy shell script to manage things like deleting those schedules when they were created. Still a number of them kicked off even when I thought I had gotten them all.