Thursday, December 03, 2020

TSMAdmin Update

 I was laid off from work as of July 1st and in the process of looking for new employment I had to make the hard choice and give up the backup/TSM/Spectrum Protect area and branch out to other avenues due to limited opportunities being available. I applied for many TSM positions with companies who either wanted me to move or didn't even end up hiring. I tried getting back on with IBM but was never considered for interview with any of their Spectrum Protect positions. I won't lie, the Spectrum Protect opportunities appear to be drying up as IBM continues to fall in marketshare in the backup world. My previous employer laid me off as they switched to Commvault and there seemed to be more opportunities with Commvault or various cloud solutions.

What I did end up doing is passing the AWS Architect Associate certification and applying with various companies for cloud and related openings. I had a friend with a large online corporation inform me of an opening with his company so I applied and through a series of interviews was able to secure a position. This opportunity is as a Site Reliability Engineer and it has NOTHING to do with backups. So as I start my new phase of work I wish you all success and if anyone would like to take over this site please contact me and we can discuss a possible transition.  Thank you all for 15+ years of backup/TSM conversation.

Wednesday, June 10, 2020

New Version of TSMExplorer

There is a new version of TSMExplorer available and it has added support for Spectrum Protect Plus. See can download a trial version of 5.2.3 here.  TSMExplorer has a load of features and provides a great central management console when you have multiple TSM/Spectrum Protect Servers. With the added support of managing Spectrum Protect Plus it's even easier to manage your environment.

Here is a video of an older version of TSMExplorer so you can get a feel for the product.

Here is a video of the client agent and how it works with the TSMExplorer console to allow access to the client options files and logs for easier management.

Tuesday, May 26, 2020

Finally - A Universal Backup Agent!

I was contacted by a representative of Spictera about their SPFS tool for Spectrum Protect. This product allows you to use a local mount on your Linux server as a place to save your backup or application dump that is a direct connection to the IBM Spectrum Protect backend storage. It uses Spectrum Protect API calls and allows you to implement versioning and retention policies just as you would normal file system backups or archives. This product can solve your issues with proprietary applications that IBM does not, and probably never will, support. Now you can backup your MongoDB or SAP HANA a lot easier by using the native backup process built into your application. Currently it only supports the following OS's.

Supported operating systems are:
  • CentOS 6 for x86_64
  • CentOS 7 for x86_64
  • CentOS 8 for x86_64
  • Debian 8 for x86_64
  • Debian 9 for x86_64
  • Red Hat 6 for x86_64, ppc64 be, s390x (zLinux)
  • Red Hat 7 for x86_64, ppc64 le, s390x (zLinux)
  • Red Hat 8 for x86_64, ppc64 le, s390x (zLinux)
  • SuSE 11 for x86_64, ppc64 be, s390x (zLinux)
  • SuSE 12 for x86_64, ppc64 le, s390x (zLinux)
  • SuSE 15 for x86_64, ppc64 le, s390x (zLinux)
  • Ubuntu 14.04 for x86_64, ppc64 le, s390x (zLinux)
  • Ubuntu 18.04 for x86_64, ppc64 le, s390x (zLinux)
I would assume if it support CentOS x86_64 you should have no trouble with it on Red Hat. I'll let Spictera confirm that though. I've provided a presentation on the product for your review.

Wednesday, May 13, 2020

HBA Info Script

I thought I would pass this script along for anyone needing a way to out HBA info on Linux in a nicely formatted table.

echo "FC_HOST     |   PortID   |    WWN             | State      | Speed    "
echo "------------+------------+--------------------+------------+----------"
for hba in `ls -d /sys/class/fc_host/host*`;do
  FC_HOST=`basename $hba`
  PortID=`cat $hba/port_id`
  wwpn=`cat $hba/port_name`
  state=`cat $hba/port_state`
  speed=`cat $hba/speed`
  hba=`cat $hba/symbolic_name`
  echo "$FC_HOST| $PortID | $wwpn | $state| $speed" | awk 'BEGIN{OFS=FS="|"}{for(i=1;i<=NF;i++){$i=sprintf("%-12s",$i)};print}'
done #|sort -k3n,6
echo "------------+------------+--------------------+------------+----------"

The output looks like this

FC_HOST     |   PortID   |    WWN             | State      | Speed    
host0       | 0xffffffff | 0x20003890a540876c | Linkdown   | unknown  
host11      | 0xffffffff | 0x20003890a540876d | Linkdown   | unknown  
host12      | 0x640000   | 0x100000109b1bbe00 | Online     | 32 Gbit  
host13      | 0xffffffff | 0x20003890a5803461 | Linkdown   | 40 Gbit  
host14      | 0xffffffff | 0x20003890a5803462 | Linkdown   | 40 Gbit  
host15      | 0x640000   | 0x100000109b1bbe01 | Online     | 32 Gbit  
host16      | 0x03eac0   | 0x100000109b1b9a77 | Online     | 16 Gbit  
host17      | 0x04eac0   | 0x100000109b1b9a78 | Online     | 16 Gbit  

If you would like more detail I added a couple more columns to show make, model, firmware, and driver version. This is a fixed width column layout and the minimum column width can be adjusted for your needs.

echo "FC_HOST         |   PortID       |    WWN             | State          | Speed          | Make           | Model          | Firmware       | Driver"
echo "----------------+----------------+--------------------+----------------+----------------+----------------+----------------+----------------+---------------"
for hba in `ls -d /sys/class/fc_host/host*`;do
  FC_HOST=`basename $hba`
  PortID=`cat $hba/port_id`
  wwpn=`cat $hba/port_name`
  state=`cat $hba/port_state`
  speed=`cat $hba/speed`
  hba=`cat $hba/symbolic_name`
  hba_make=`echo $hba | cut -f1 -d ' ' `
  hba_model=`echo $hba | cut -f2 -d ' ' `
  hba_fw=`echo $hba | cut -f3 -d ' ' `
  hba_driver=`echo $hba | cut -f4 -d ' ' `
  echo "$FC_HOST| $PortID | $wwpn | $state| $speed | $hba_make | $hba_model | $hba_fw | $hba_driver " | awk 'BEGIN{OFS=FS="|"}{for(i=1;i<=NF;i++){$i=sprintf("%-16s",$i)};print}'
done #|sort -k3n,6

echo "----------------+----------------+--------------------+----------------+----------------+----------------+----------------+----------------+---------------"

The output looks similar to this:

Tuesday, April 21, 2020

End of the Road?

For those of you who come here periodically and are wondering why content has been lacking, I wanted to let you all know that it is due to my current employment situation. Currently my employer is in the process of replacing Spectrum Protect throughout the company with Commvault. Once the conversion has completed I will be released and so am currently looking for new employment. Since the 8.1.8 upgrade of our servers I have been working steadily on reporting and prepping for the conversion. The initial thought was that I would stay employed and transition to learning Commvault, but that turned out to not be the case. I have enjoyed managing ADSM/TSM/Spectrum Protect environments for the last 22 years but have seen a drop off of its use as IBM has retreated from marketing their products and moved more towards other areas of the IT field. As I have looked for jobs in the Spectrum Protect realm I have noticed that they are rarer than ever, especially with the current situation the world finds itself. I plan to leave the site and up and will post anything you the readers feel will benefit the Spectrum Protect community. Feel free to send me your article for review and if I feel it fits the nature of this website it will be posted. Unfortunately, unless things change I see myself transitioning into a new role with a cloud emphasis and posts here will end unless you all take over the mantle and provide other admins with advice and examples from your own experiences. I have enjoyed posting and helping the community since founding this website and wish you all success going forward.

Wednesday, August 14, 2019

Error Upgrading to

I performed an upgrade today on a server going from to and after the upgrade we encountered the following errors in the actlog as the system was starting.

08/14/2019 11:59:59      ANR9999D_2835836899 InstGetDB2stmtPass1(instr.c:550)                          Thread<404>: Error in fetching DB2 statement stats,                          rc=235908/14/2019 11:59:59      ANR9999D Thread<404> issued message 9999 from:08/14/2019 11:59:59      ANR9999D Thread<404>  0x00000001000365b4 StdPutText08/14/2019 11:59:59      ANR9999D Thread<404>  0x000000010003784c OutDiagToCons08/14/2019 11:59:59      ANR9999D Thread<404>  0x000000010000cf08 outDiagfExt08/14/2019 11:59:59      ANR9999D Thread<404>  0x000000010019d0d0                          IPRA.$InstGetDB2stmtPass108/14/2019 11:59:59      ANR9999D Thread<404>  0x00000001001a15e0 instrumentStart08/14/2019 11:59:59      ANR9999D Thread<404>  0x00000001012d7b98                          AdmInstrumentBegin08/14/2019 11:59:59      ANR9999D Thread<404>  0x00000001008da68c AdmCommandLocal08/14/2019 11:59:59      ANR9999D Thread<404>  0x00000001008d7af0 admCommand08/14/2019 11:59:59      ANR9999D Thread<404>  0x00000001013fbe68 RunCommand08/14/2019 11:59:59      ANR9999D Thread<404>  0x00000001013f7a48                          ServerMonTwentyMinThread08/14/2019 11:59:59      ANR9999D Thread<404>  0x0000000100010648 StartThread

Although the server started, I wanted to make sure this would not cause issues with the instance. I searched the error ANR9999D_2835836899 and found the following IBM APAR notice. It appears that after the upgrade you may have to manually upgrade the DB2 version using the DB2 update tool. While it appears to be a minor issue, I would recommend you check your instances after upgrade to verify whether you need to run the manual steps to resolve the errors. I've included them here for your convenience.

NOTE: I've added a couple commands (in bold) that we have had to add to the process to make it work.

Local fix

Even though the db2-level might be updated to the latest version, run the below set of commands
Stop TSM server and issue the following commands as the instance owner.
  • db2 start
  • db2 force application all
  • db2 terminate
  • ipclean

login as root user and run
  • /opt/tivoli/tsm/db2/instance/db2iupdt -d <instance name>

login as the instance owner ID and issue the command:

  • db2updv111 -d TSMDB1

This issue is supposedly fixed in version and higher.