ANE4035W (Session: 79, Node: ODC-WINCS-1-DR) File '\\od-wincs-1\e$\834\c-oh\files-20170823-123000' currently unavailable on server.
The tape was in a readonly state and accessible by the library so it was not a tape unavailable problem. I then querried the file and ran some tests suggested by IBM in the following link
http://www-01.ibm.com/support/docview.wss?uid=swg21249032
The data is present and on tape but TSM will not restore it. (See the queries below) IBM would not help us because the OS is unsupported as is the TSM client. We could not restore the System State, System Services, or any of the Data to the DR replica. What we could do is access the data from a supported client level on a supported OS level (i.e. TSM Client 6.4 from a Windows 2008 server). We were unable to recover the Win2K/Win2003 servers during the DR test and unfortunately I know many companies out there still have some of these servers in their environment. It's time to upgrade! Your data is at risk!
tsm: OMATSM02>select * from backups where node_name='ODC-WINCS-1-DR' and FILESPACE_ID=1 and hl_name=upper('\834\c-oh\') and LL_NAME=upper('files-20170823-123000')
NODE_NAME: ODC-WINCS-1-DR
FILESPACE_NAME: \\od-wincs-1\e$
FILESPACE_ID: 1
STATE: ACTIVE_VERSION
TYPE: DIR
HL_NAME: \834\C-OH\
LL_NAME: FILES-20170823-123000
OBJECT_ID: 11926208445
BACKUP_DATE: 2018-08-10 03:04:37.000000
DEACTIVATE_DATE:
OWNER:
CLASS_NAME: WIN-DR-MC
ACTUAL_SIZE:
tsm: OMATSM02>show bfo 11926208445
Bitfile Object: 11926208445
Active
**Sub-bitfile 11926208445 is stored in the following aggregate(s)
Super-bitfile: 11926199674, Offset: 26157087, Length 735, Deduped: F
tsm: OMATSM02>show bfo 11926199674
Bitfile Object: 11926199674
**Super-bitfile 11926199674 contains following aggregated bitfiles,
Bitfile Id, offset, length, active state or owner, link bfid
11926199674 0 8732 Active
11926199676 8732 1527 Active
.
.
.
11926208946 26573407 5510 Active
11926208947 26578917 49168 Active
**Sub-bitfile 11926199674 is stored in the following aggregate(s)
Super-bitfile: 11926199674, Offset: 0, Length 8732, Deduped: F
**Archival Bitfile Entry
Bitfile Type: PRIMARY Storage Format: 22
Bitfile Size: 26631381 Number of Segments: 1, flags: 0
Storage Pool ID: 33 Volume ID: 22903 Volume Name: L42803
tsm: OMATSM02>q vol L42803 f=d
Volume Name: L42803
Storage Pool Name: WIN-DR-TAPE
Device Class Name: IBM-LTO-4
Estimated Capacity: 4.2 T
Scaled Capacity Applied:
Pct Util: 100.0
Volume Status: Full
Access: Read-Only
Pct. Reclaimable Space: 0.0
Scratch Volume?: Yes
In Error State?: No
Number of Writable Sides: 1
Number of Times Mounted: 227
Write Pass Number: 1
Approx. Date Last Written: 08/12/18 16:44:25
Approx. Date Last Read: 08/25/18 22:06:34
Date Became Pending:
Number of Write Errors: 0
Number of Read Errors: 0
Volume Location:
Volume is MVS Lanfree Capable : No
Last Update by (administrator): CSMALL
Last Update Date/Time: 08/25/18 20:29:37
Begin Reclaim Period:
End Reclaim Period:
Drive Encryption Key Manager: IBM Spectrum Protect
Logical Block Protected: No
tsm: OMATSM02>q libvol dr-libr L42803
Library Name Volume Name Status Owner Last Use Home Device
Element Type
------------ ----------- ---------------- ---------- --------- ------- ------
DR-LIBR L42803 Private Data 1,089
I do not think this is the result of an imposed restriction. Rather, it looks like a known APAR, IT19033. Take a look at https://www.ibm.com/support/docview.wss?uid=swg1IT19033 which describes the issue, workarounds, and fixing server levels.
ReplyDeleteAndy Raibeck, IBM Spectrum Protect Level 3
Andrew,
ReplyDeleteI had found this APAR when trying to diagnose the issue and when TAPEPROMPT and the VERBOSE options did not resolve the issue I was left without any recourse since the client was an unsupported version (5.4 and 5.5) and IBM could (would) not help us.
Chad,
DeleteI did some checking, and found that for IT19033, the workarounds were not always effective. Thus that APAR should not be ruled out. The next step should be to upgrade your server to a fixing level, then try the restore again.
I also found your support case, and it is still open. If the fix for IT19033 does not resolve the issue, we can continue to pursue it through that support case.
Andy Raibeck, IBM Spectrum Protect Level 3
Sooooo..... I kind of don't want it fixed. The customer is using old unsupported OS (Windows 2003) and very old TSM clients. We were performing a "DR" at Sungard with limited time available. I would have considered upgrading the TSM server but at the time it did not appear to be the issue. The good that came from this is that the customer is taking more seriously their need to upgrade to supported OS and TSM versions. I will notify the team about the issue and how to possibly avoid it in the future since we were definitely on a non-fixed level (8.1.0). I really appreciate the input and help. Thank you!
ReplyDelete