The life of an IBM i system administrator is full of surprises. This story is all about such a surprise. Although there are system administrators, who think that the rule “if it ain't broken, don't fix it” is still valid. I always ask myself how do they know, if the ain't broken is true. The last month of 2021 did learn that there was a zero-day vulnerability in log4j. So my point is that once in a while you have to patch for IBM i we do this by applying PTFs. It is after such an apply, this story begins.
In September, we had a switch planned for one of our customers having an environment with an independent auxiliary storage pool (iASP). For managing and running that environment, a cluster is configured, using IBM PowerHA SystemMirror for i (5770HAS). Combined with PowerHA Tools for IBM i – IASP Manager from Lab Services, with which a iASP FlashCopy backup is done.
Before making the switch from the Production system to the CBU system, all PTF groups were brought to the latest level, including the 7.3 group PTF SF99876 High Availability for IBM i. 5770HAS PTF SI76565 was part of this group, the June 2021 PTF. Part of good system management is running the command: “DSPPTFCVR CVRATR(*SPCINST)” to review the special instructions. As you may know, reading those instructions does tell you about new functionality and special instruction you need to execute in order to apply a PTF. When running this command the PowerHA June 2021 PTF will pop up and this PTF does contain a link as you can see below:
The link https://ibm.biz/PowerHA_PTF_info will bring you to documentation about all the latest PTFs for PowerHA, which is now taken care of by HelpSystems. They are the developers of PowerHA. If you scroll down to the section starting with “Version 7.3 HA 3.7.0, PTF SI76565 (plus Language PTFs)” you will find information about new features, enhancements and other fixes.
So after having gone through this list as a good administrator should, we did apply also this PTF before we made the switch. The switch was done within 15 minutes and after all check were done, the next evening a FlashCopy backup of the iASP was initiated. Is there where the problem started. The job log of the task initiating the backup did report the following in it's joblog:
Due to covid this was the first switch in 2021, before that we did switch twice a year. This was something new. The license key for 5770HAS – IBM PowerHA SystemMirror for i was left untouched since this customer made the switch from software replication to his iASP environment.
When running on a CBU system, seeing the message “IBM i usage limit exceeded - operator action required” is normal. As long as you do not exceed the total number of entitled IBM i core licenses of your production system, you are good to go. Why does PowerHA all of a sudden have a problem with that? In order to find out, I did open an IBM case for this issue. I am not going to bother you with the details, but the end result of that case was that HelpSystems did update their Software Licensing & Compliance Information webpage.
https://helpsystemswiki.atlassian.net/wiki/spaces/IWT/pages/162627586/Software+Licensing+Compliance+Information
It also became clear that this June 2021 PowerHA PTF did check for a license for the total number of cores. The good thing is that the IBM documentation about PowerHA licensing always was clear about what was needed.
https://www.ibm.com/support/pages/message-cpf9e72-or-cpf9e71-received-licpgm-5770has
When you did have a good look at my joblog image, you may have noticed that the license error did occur in the beginning of September 2021. Both the links given before, the Software Licensing & Compliance Information from HelpSystems and IBM contain information when they were last updated. The last link in this article from IBM does contain a link to another document of IBM which has a modify date of 10 June 2021. So the information to avoid the issue I did ran into was already available. I do think that it is almost impossible to keep up with all the information available. The IBM case created for this issue, did make IBM aware of the fact that the PowerHA PTF cover letters need to put more emphasis on the fact that you have to use the link in order to find out what the PTF is all about.
The bottom line is that you can fill in the form: “REQUEST FOR DESIGNATION OF PRIMARY AND BACKUP MACHINE PAIRING” provided by IBM's Entitled Systems Support, to use your production system as the donor system for letting them generate the CBU license keys for PowerHA.
If you want to learn more about using an iASP combined with FlashCopy and PowerHA SystemMirror for i. please have a look at this redbook.