Home Patching CVE-2014-6271 and CVE-2014-7169 on AIX via NIM (bash bug aka shellshock)
Post
Cancel

Patching CVE-2014-6271 and CVE-2014-7169 on AIX via NIM (bash bug aka shellshock)

Below I detail how I patched over 800 AIX LPAR’s that were exposed by CVE-2014-6271 and CVE-2014-7169, also known as shellshock, using the NIM server.

From everything that I’ve been reading on IBM’s Knowledge Centre, creating an LPP source containing only RPM’s isn’t possible. To patch my AIX environment, I decided to use the script resource available to the NIM master, along with the pre-existing NFS mounts that I had configured.

  1. Configure NFS share.

    1
    2
    3
    4
    5
    6
    
     NIM:kristian# cat /etc/exports
     /export/nim/images -ro,anon=0
        
     NIM:kristian# showmount -e
     export list for NIM:
     /export/nim/images (everyone)
    
  2. Download patched bash RPM to NIM master.

    1
    2
    3
    
     NIM:kristian# ls -l /export/nim/images/bash_CVE-2014-6271-7169
     total 3448
     -rw-r-----    1 root  system     1765643 Sep 30 08:22 bash-4.2-17.aix5.1.ppc.rpm
    
  3. Script to patch bash /export/nim/patches/install_bash_CVE-2014-6271-7169

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    
     #!/bin/ksh
     #
     # Script to install new version of bash to
     # patch CVE-2014-6271 and CVE-2014-7169
     #
     # Kristian Milos (29/09/14)
        
     # Get NIM master hostname
     NIM_MASTER_HOSTNAME=`grep NIM_MASTER_HOSTNAME /etc/niminfo | awk -F = '{ print $2 }'`
        
     # Create temporary mount location
     mkdir /install_bash_CVE-2014-6271-7169
        
     # NFS mount patch
     mount ${NIM_MASTER_HOSTNAME}:/export/nim/images/bash_CVE-2014-6271-7169 /install_bash_CVE-2014-6271-7169
        
     # Install patch
     rpm -Uvh /install_bash_CVE-2014-6271-7169/bash-4.2-17.aix5.1.ppc.rpm
        
     # Unmount NFS mount
     umount /install_bash_CVE-2014-6271-7169
        
     # Remove temporary mount location
     rm -r /install_bash_CVE-2014-6271-7169
        
     exit
    
  4. Define NIM resource

    Now that we have the location of the RPM on the NIM master, and the script that will be run on the NIM client to patch bash, we can now define a NIM script resource.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    
     NIM:kristian# nim -o define -t script \
     -a server=master \
     -a location=/export/nim/patches/install_bash_CVE-2014-6271-7169 \
     -a comments="bash fix for CVE-2014-6271 and CVE-2014-7169" bash_CVE-2014-6271-7169
        
     NIM:kristian# lsnim -l bash_CVE-2014-6271-7169
     bash_CVE-2014-6271-7169:
     class       = resources
     type        = script
     comments    = bash fix for CVE-2014-6271 and CVE-2014-7169
     Rstate      = ready for use
     prev_state  = unavailable for use
     location    = /export/nim/patches/install_bash_CVE-2014-6271-7169
     alloc_count = 0
     server      = master
    
  5. Define NIM machine group

    We will now create a NIM machine group that will contain all the NIM clients that we will update. I find the easiest way to do this is by listing out all the NIM client definitions in the format required for the group define command. An example is shown below.

    1
    2
    3
    4
    5
    6
    7
    
     NIM:kristian# for i in `lsnim -t standalone | awk '{ print $1 }'`; do echo "-a add_member=$i \\"; done
     -a add_member=aix1 \
     -a add_member=aix2 \
     -a add_member=aix3 \
     -a add_member=aix4 \
     -a add_member=aix5 \
     -a add_member=aix6 \
    

    Define the NIM group

    1
    2
    3
    4
    5
    6
    7
    
     NIM:kristian# nim -o define -t mac_group \
     -a add_member=aix1 \
     -a add_member=aix2 \
     -a add_member=aix3 \
     -a add_member=aix4 \
     -a add_member=aix5 \
     -a add_member=aix6 PROD_LPARS
    
  6. Validate NIM master to client communications

    The next thing I do is validate that the NIM master can actually talk to all the NIM clients in the machine group.

    1
    2
    3
    4
    5
    6
    7
    
     NIM:kristian# for srv in `lsnim -g PROD_LPARS | grep member | grep -v EXCLUDED | awk '{ print $3 }' | sed 's/;//' | sort`; do printf "%-20s" $srv; nim -o lslpp $srv >/dev/null 2>&1; [ "$?" == 0 ] && echo OK || echo "Problem"; done
     aix1            OK
     aix2            OK
     aix3            OK
     aix4            OK
     aix5            Problem
     aix6            OK
    
  7. Exclude uncontactable hosts

    For any NIM client that returns “Problem”, I exclude them from the NIM group operation

    1
    
     NIM:kristian# nim -o select -a exclude=aix5 PROD_LPARS
    
  8. Patch all NIM clients via NIM master

    We’re now in a position to execute the patch across all the NIM clients listed in the group definition.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    
     NIM:kristian# nim -o cust -a script=bash_CVE-2014-6271-7169 -a concurrent=10 PROD_LPARS
     +-----------------------------------------------------------------------------+
                         Concurrency Control
     +-----------------------------------------------------------------------------+
        
     Processing will begin with the first 5 machines from the group...
        
     +-----------------------------------------------------------------------------+
                         Initiating "cust" Operation
     +-----------------------------------------------------------------------------+
     Allocating resources ...
        
     Initiating the cust operation on machine 1 of 5: aix1 ...
        
     Initiating the cust operation on machine 2 of 5: aix2 ...
        
     Initiating the cust operation on machine 3 of 5: aix3 ...
        
     Initiating the cust operation on machine 4 of 5: aix4 ...
        
     Initiating the cust operation on machine 5 of 5: aix6 ...
        
     +-----------------------------------------------------------------------------+
                         "cust" Operation Summary
     +-----------------------------------------------------------------------------+
     Target                  Result
     ------                  ------
     aix1                    INITIATED
     aix2                    INITIATED
     aix3                    INITIATED
     aix4                    INITIATED
     aix6                    INITIATED
        
     Note: Use the lsnim command to monitor progress of "INITIATED"
     targets by viewing their NIM database definition.
        
     +-----------------------------------------------------------------------------+
                         Concurrency Control
     +-----------------------------------------------------------------------------+
     The first 8 machines have been processed.  As machines finish
     installing processing will resume with the remaining members
     of the group, one at a time.
        
     +-----------------------------------------------------------------------------+
                         Concurrency Control: "cust" Operation Summary
     +-----------------------------------------------------------------------------+
     Target                  Result
     ------                  ------
     aix1                    COMPLETE
     aix2                    COMPLETE
     aix3                    COMPLETE
     aix4                    COMPLETE
     aix6                    COMPLETE
    
  9. Validate installed version of bash

    Once the process has completed, you can validate the version of bash installed across all NIM clients by running the following command.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    
     NIM:kristian# for i in `lsnim -g PROD_LPARS | grep member | grep -v EXCLUDED | awk '{ print $3 }' | sed 's/;//' | sort`; do echo $i; nim -o lslpp -a lslpp_flags=-Lc -a filesets=bash $i | grep bash | awk -F : '{ print $2 }'; echo ""; done
     aix1
     bash-4.2-17
        
     aix2
     bash-4.2-17
        
     aix3
     bash-4.2-17
        
     aix4
     bash-4.2-17
        
     aix6
     bash-4.2-17
    
  10. Include previously excluded NIM members

    One final clean up task, is to ensure you include all NIM members back into the group if you excluded them previously.

    1
    
    NIM:kristian# nim -o select -a include_all=yes PROD_LPARS
    
This post is licensed under CC BY 4.0 by the author.

AIX boot hangs with HMC 2700 LED code

PowerHA SystemMirror and NIM mksysb failures