Finally - A Universal Backup Agent!

I was contacted by a representative of Spictera about their SPFS tool for Spectrum Protect. This product allows you to use a local mount on your Linux server as a place to save your backup or application dump that is a direct connection to the IBM Spectrum Protect backend storage. It uses Spectrum Protect API calls and allows you to implement versioning and retention policies just as you would normal file system backups or archives. This product can solve your issues with proprietary applications that IBM does not, and probably never will, support. Now you can backup your MongoDB or SAP HANA a lot easier by using the native backup process built into your application. Currently it only supports the following OS's.

Supported operating systems are:

• CentOS 6 for x86_64
• CentOS 7 for x86_64
• CentOS 8 for x86_64
• Debian 8 for x86_64
• Debian 9 for x86_64
• Red Hat 6 for x86_64, ppc64 be, s390x (zLinux)
• Red Hat 7 for x86_64, ppc64 le, s390x (zLinux)
• Red Hat 8 for x86_64, ppc64 le, s390x (zLinux)
• SuSE 11 for x86_64, ppc64 be, s390x (zLinux)
• SuSE 12 for x86_64, ppc64 le, s390x (zLinux)
• SuSE 15 for x86_64, ppc64 le, s390x (zLinux)
• Ubuntu 14.04 for x86_64, ppc64 le, s390x (zLinux)
• Ubuntu 18.04 for x86_64, ppc64 le, s390x (zLinux)

I would assume if it support CentOS x86_64 you should have no trouble with it on Red Hat. I'll let Spictera confirm that though. I've provided a presentation on the product for your review.

TSM SQL TDP Better Backup Script

On a call today I had an admin complain that the TSM SQL TDP was issuing failed backup errors when the script encountered a database that was offline. So I asked if he had searched Google for an alternate script and he had not. I did a quick query and found this nice batch file that appears to do exactly what is needed. The reader on the site commented that the script did not work on his SQL cluster. Does anyone have a similar script that would work on a cluster? Or could you modify this script to work in a cluster environment? The link to the script is provided below.

http://www.applepie.se/tivoli-storage-manager-and-sql-backup

Why Are You Not Using Google & YouTube?

I had an OS Admin contact me through LinkedIn and G-Mail asking for help on trying to find his archived data. He didn't have much experience with TSM and was looking for information on how to find the long term backups (i.e. Archives). I asked him if he even tried to search Google? Google and YouTube are great resources for all your needs. For example if you want  to learn more about the new Operations Center you can see a plethora of videos by searching YouTube. You can also use google to find all sorts of related documents and pages when it concerns APARs and errors. If you haven't done your due diligence you make yourself look dumb. Shoot, you can even search Google from my website and get my posts and outside relevant web pages!

So here is a list of YouTube videos you can reference:

Backup and Archive

Server Administration

Setup TSM Deduplication

Tivoli Data Protection Agents

TDP for Virtual Environments

Poor Performance

Currently I work in an environment where we have a specific TSM instance for a large SAP DB (99TB currently). We just upgraded the drives in the tape library (yes we use tape! I know...I know....) from MagStar 3592 TS1130 (E06) drives to TS1140 (E07) drives. The upgrade was pushed in hopes of a jump in write/backup performance, but I was skeptical. TSM adds so much overhead you cannot use the RAW tape read/write numbers from any manufacturer. Typically IBM is somewhat reasonable with their numbers, but in this case I have seen NO performance increase what-so-ever.  Here is a query of the processes for storage pool backup.

UPDATE (04/04/2014):  Let me give you some more specs, we have the 99TB DB split between 4 TSM Storage Agents each having 4 8Gb HBA's. Each storage agent runs 4 sessions (allocates 4 drives) for their backup process. So all 4 storage agents account for 16 simultaneous sessions and it still takes over 24 hours to perform the 99TB backup. The backups are averaging around 70-78MB/sec. Is this a TSM overhead issue or do I have a tuning issue with the TDP and TSM? I'm getting less than 50% of the throughput I should see.

Here's the command that is run to execute the DB backup:

ksh -c export DB2NODE=7 ; db2 "backup db DB8   LOAD /usr/tivoli/tsm/tdp_r3/db264/libtdpdb264.a OPEN 4 SESSIONS OPTIONS /db2/DB8/dbs/tsm_config/vendor.env.7 WITH 14 BUFFERS BUFFER 1024 PARALLELISM 8 WITHOUT PROMPTING" ; echo BACKUP_RC=$?

PROCESS_NUM: 2667
    PROCESS: Backup Storage Pool
 START_TIME: 03-27 23:21:54
   DURATION: 00 23:20:13
      BYTES: 6.0TB
 AVG_THRPUT: 75.87 MB/s

PROCESS_NUM: 2668
    PROCESS: Backup Storage Pool
 START_TIME: 03-27 23:21:55
   DURATION: 00 23:20:12
      BYTES: 6.2TB
 AVG_THRPUT: 78.48 MB/s

PROCESS_NUM: 2669
    PROCESS: Backup Storage Pool
 START_TIME: 03-27 23:21:55
   DURATION: 00 23:20:12
      BYTES: 6.2TB
 AVG_THRPUT: 77.99 MB/s

PROCESS_NUM: 2670
    PROCESS: Backup Storage Pool
 START_TIME: 03-27 23:21:55
   DURATION: 00 23:20:12
      BYTES: 6.4TB
 AVG_THRPUT: 80.13 MB/s

I average anywhere from 75 to 80 MB/sec.  Here is the Magstar performance chart. I am using JB media, not JC so I do take a little hit in performance for that.

So with JB media I could get as high as 200MB/sec but I am not even 50% of that number.  Is there any specific tuning parameter I should look at that could be hindering the performance? 

FYI - The backup of the 99TB DB runs LAN-Free using 16 tape drives over 26 hrs.

Good TSM Oracle Backup Blog

I came across this website and it has a lot of TSM Oracle backup related information. If you perform Oracle TDP backups then check out this site.

Finally! MySQL backups with TSM

Computer Technology Review has an announcement from Zmanda stating they are releasing a TSM backup agent (Zmanda Recovery Manager for MySQL). The ZRM-TSM agent is available immediately. You can read the full article here. Those of you who have been asking for a TDP for MySQL, the wait is over.

TSM Sharepoint TDP

Has anyone used the Sharepoint TDP for TSM yet? I have a number of Sharepoint servers at work and would like to know what users thoughts are on its capabilities. Is it easy to use or a nightmare to implement? Does it backup well or is it slow? Does it adversely affect the systems performance? If anyone can give me some feedback I'd appreciate it.

Oracle TDP and HACMP 5.3 Incompatibility Resolved

Well I was have a heck of a time with this problem. It seems the 5.3 HACMP binaries are incompatible with the Oracle 64-bit API so the TDP would crash when attempting to backup a DB on a cluster. I was expecting a fix and was told it would be available by January. So I waited, and watched and never saw an update to the TDP. Frustration mounted until I was able to get ahold of a TDP developer who pointed out that it was the HACMP side that had to do the patching and now all is well. So if you are looking to use HACMP and backup Oracle DB's with the TDP make sure they have the latest patch levels installed or the 64-bit API will not work.

Got HACMP?

Well our group got stung again this last week. While working a HACMP implementation we were asked to install the Oracle TDP and TSM client. We installed the TSM client and all went well and we then installed the TDP and the DBA’s could not get the password file to generate. Each time they tried the utility would core dump. When trying to run the utility multiple ways it always core dumped. After calling support we were informed that the current 64-bit Oracle TDP (And from what I have found through Google it could be any 64-bit TDP). It seems they rebuilt HACMP version 5.3 and the TDP is not compatible with the new way it was compiled. There is a patch projected to be released as of end of the month, but we cannot wait that long so our options are to use temporary disk and mirror the DB, then break and backup the mirrored copy when we want to backup, or we can roll back to HACMP 5.2. The problem with the latter is that we basically have to rebuild the boxes. So be warned there are issues with the latest and greatest version of HACMP at this time.

Oracle RMAN Catalogue Cleanup

Why do people love Oracle? When I hear mention of Oracle I think of Luke Skywalker when he saw the Millenium Falcon, "What a piece of junk!" Like the Falcon it looks clunky, breaks down easily, and has the most tempermental behavior. When it's running, however, it screams. The problem is that the RMAN catalogue sometimes doesn't do appropriate cleanup. If you want to make sure that a particular node is performing cleanup within TSM run the following select command -

select object_id from backups where node_name= and
backup_date < '2005-07-01 00:00:00'

This can be redirected to a file then used later to delete with an undocumented delete command. I will give the delete command out to those who need it, but remember any deletion from the TSM DB is done AT YOUR OWN RISK! It's unsupported because Tivoli doesn't trust you to not screw stuff up, and although I don't think you will, it's better safe then sorry.