TEXT   45

x86jumpstart

Guest on 16th August 2022 01:48:04 PM

  1. 1) ISC DHCP
  2.    Since SUN DHCP server is quite complex/cumbersome and hard to manage
  3.    we (like many other admins) use the ISC DHCP server as the admins
  4.    first choice dhcp server - some people still love the idea/paradigm
  5.    "keep it small and simple".
  6.  
  7.    So the first hurdle to take was to determine the parameters required
  8.    to get x86 jumpstart working. Since Sun JumpStart/NetworkInstall manuals
  9.    always refer to its own defined option names (e.g. BootFile, BootSrvA)
  10.    and gives no example, how one may do that with probably one of the
  11.    most used dhcp servers aka ISC DHCP server (remember almost every Linux
  12.    as well as *BSD system uses the ISC dhcp server as its default one).
  13.  
  14.    Furthermore the documentation is pretty confusing wrt. what is really
  15.    required for x86 jumpstart. E.g. 817-5504 table 6-5 on page 105 lists
  16.    vendor category options "required to enable a DHCP server to support
  17.    Solaris installation clients". This might be the case for sparc based
  18.    DHCP jumpstart, however for x86 jumpstart it turns out, that actually
  19.    none of these options are required at all. Also the documentation
  20.    explains a lot of ore or less useful stuff, but actually not, how it
  21.    really works. I think, the following is, what one really
  22.    needs to know, to get the picture, to be able to troubleshoot and
  23.    finally to get it work (actually, don't know, whether this is
  24.    really what happens, but this is my best guess adding 1 and 1):
  25.  
  26.    a) PXE tries to lookup via DHCP a tftpserver and the bootloader to
  27.       download, which will be used for booting the [solaris] kernel via
  28.       network.
  29.           Wrt. ISC DHCP the tftp server is described by the parameter
  30.       'next-server' (for SUN DHCP by 'BootSrvA') and the bootloader by the
  31.       parameter 'filename' (for SUN DHCP by 'BootFile').
  32.       For x86 jumpstart the bootloader file is usually pxegrub, which is
  33.       automatically copied to /tftpboot/pxegrub.I86PC.Solaris_$SREL-$N and
  34.       linked to /tftpboot/01$ETHERADDR of the client by add_install_client.
  35.       So the appropriate  section for ISC DHCP could look like this:
  36.       group {
  37.           # x86 jumpstart clients
  38.           next-server 192.168.1.1;
  39.           host dax
  40.           { hardware ethernet 0:14:4f:aa:bb:cc; fixed-address dax;
  41.                 filename 0100144FAABBCC; }
  42.       }
  43.       subnet 192.168.1.0 netmask 255.255.255.0 {
  44.           option domain-name-servers  192.168.1.2;
  45.           option broadcast-address 192.168.1.255;
  46.           option domain-name "deep.space.nine.net";
  47.           option routers 192.168.1.254;
  48.           option time-servers 192.168.1.3;
  49.           option ntp-servers 192.168.1.3;
  50.       }
  51.    b) PXE downloads pxegrub via tftp and starts it.
  52.    c) pxegrub tries to obtain a list of parameters required to boot
  53.       solaris via network (sparc OBP firmware does this by asking a
  54.       bootparamd in the same network). These paremeters are listed
  55.       in a text file named /tftpboot/menu.lst.01$ETHERADDR of the client,
  56.       which gets automatically created by add_install_client using the
  57.       -d and -e options. E.g.: /tftpboot/menu.lst.0100144FAABBCC
  58.    d) pxegrub downloads the solaris kernel and miniroot described in
  59.       the downloaded menu.lst.01* file via tftp.
  60.    e) pxegrub loads and boots the kernel with the bootparameters given
  61.       in the downloaded menu.lst.01* file
  62.    f) The kernel takes over, starts the OS and usually the install process.
  63.  
  64.  
  65.  
  66. 2) The next obstacle is, that something like:
  67.  
  68.    name_service=NIS+ {domain_name=deep.space.nine.net \
  69.    name_server=odo(192.168.1.3)}
  70.    network_interface=PRIMARY {default_route=192.168.1.254 \
  71.    netmask=255.255.255.0 protocol_ipv6=no}
  72.  
  73.    does not work for x86 jumpstart (neither for u3 nor for b57). Used this
  74.    for sparc based jumpstarts for more than 10 years and never had a problem
  75.    with it.
  76.  
  77.    So I needed to manipulate the begin scripts  to be able to troubleshoot,
  78.    why the installer always invokes the network setup dialogs first.
  79.    My guess is, that it doesn't get the hostname right (this one and the
  80.    nis+ name server name are the only fields, which are not preset with an
  81.    appropriate value).
  82.    Since I couldn't find out, how to get it work this way, I switched over
  83.    to 'network_interface=PRIMARY {dhcp protocol_ipv6=no}'.
  84.    This worked (i.e. causes not firing up the network setup dialogs).
  85.  
  86. 3) However 'network_interface=PRIMARY {dhcp protocol_ipv6=no}' causes the
  87.    installer to not set the SI_HOSTNAME variable on a x4500 and to set it
  88.    to 'localhost' on an Ultra 40. However the correct hostname is essential
  89.    for our begin and finish scripts to adjust the common configuration for
  90.    the specific machine. So a further (quick and dirty) modification
  91.    of the scripts was required to fix this problem:
  92.    SI_HOSTNAME=`nismatch -s, addr=$SI_HOSTADDRESS hosts.org_dir | cut -f1 -d,`
  93.  
  94. 4) Furthermore 'network_interface=PRIMARY {dhcp protocol_ipv6=no}' causes
  95.    the installer to setup the machine as a dhcp client, but we wanna have
  96.    static IP-Addresses for all Solaris machines.
  97.    It turned out, that just removing /a/etc/dhcp.* is not sufficient.
  98.    So finish script required additional work to remove /a/etc/dhcp/*.dhc
  99.    as well, to deterin the used NIC and to setup /a/etc/hostname.*,
  100.    /a/etc/inet/hosts, /a/etc/inet/ipnodes and /a/etc/nodename "manually"...
  101.  
  102.    BTW: First time a got an jupstarted x86 and thus dhcp enabled machine,
  103.         Solaris was unable to deterine its hostname as well. Pretty
  104.         strange, since any none-Solaris machine (i.e. Linux, BSD and even
  105.         Windows client) is able to get its hostname via ISC dhcp...
  106.  
  107. 5) The next problem is, that one can't do something like
  108.    chroot /a $script
  109.    in the finish script anymore. E.g. [ -x /usr/bin/gconftool-2 ] returns
  110.    true, but /usr/bin/gconftool-2 ... gives an exec error - strange.
  111.    So I needed to create "invent" a mechanism, which starts the script,
  112.    when the machine comes up the first time after its installation and
  113.    of course at the right time in the boot sequence ...
  114.  
  115. 6) Last but not least: Our install scripts (begin/finish) require
  116.    sed/egrep with full regular regex support. Since SUNWxcu4 is missing in
  117.    the miniroot, up to now (i.e. for sparc) we did a simple:
  118.    pkgadd -R $DEST_PATH/$OS_RELEASE/Tools/Boot \
  119.         -d $DEST_PATH/$OS_RELEASE/Product SUNWxcu4
  120.    and everything was ok.
  121.  
  122.    For x86 it took some addtional time to find out, how to create a new
  123.    miniroot, mainly because of the documentation bug in 817-5504 on
  124.    page 127 point f).
  125.  
  126.    Finally I came up with:
  127.         # on x86
  128. SCRATCH=/export/scratch/$USER/i386
  129. + rm -rf $SCRATCH ; + mkdir -p $SCRATCH
  130. + /boot/solaris/bin/root_archive unpackmedia /net/install/$DEST_PATH $SCRATCH
  131. + pkgadd -R $SCRATCH -d /net/install/$DEST_PATH/$OS_RELEASE/Product SUNWxcu4
  132.  
  133. DST=$HOME/tmp/miniroot/${OS_RELEASE}_$OS_SUBRELEASE
  134. + rm -rf $DST
  135. + mkdir -p $DST/${OS_RELEASE}
  136. + mkdir -p $DST/boot/grub
  137. + /boot/solaris/bin/root_archive packmedia $DST $SCRATCH
  138.  
  139.    # on sparc install server
  140. DST=$HOME/tmp/miniroot/${OS_RELEASE}_$OS_SUBRELEASE
  141. cd $DEST_PATH/$OS_RELEASE/Tools/Boot
  142. + mkdir ../Boot.orig
  143. find .  | + cpio -puvmd ../Boot.orig
  144. cd $DEST_PATH/boot
  145. + mkdir ../boot.orig
  146. find .  | + cpio -puvmd ../boot.orig
  147. cd $DST
  148. find . | + cpio -puvmd $DEST_PATH
  149.  
  150.  
  151. 7) last but not least I got to fix the x86 add_install_client, which prevents
  152.    (if unmodified) the install server to come up on reboot, if the install dir
  153.    resides on a ZFS.
  154.  
  155.    So now I use:
  156.    + patch -b -u -d $DEST_PATH/$OS_RELEASE/Tools \
  157.         -i /pool1/install/jumpstart/misc/add_install_client_x86.patch
  158.  
  159.    whereby the patch is this one:
  160. #################### start patch ##########################
  161. --- add_install_client.orig     Thu Aug 31 10:52:27 2006
  162. +++ add_install_client  Wed Feb 14 04:07:58 2007
  163. @@ -1899,40 +1899,37 @@
  164.  
  165.  if [ "${PGRP}" = "i86pc" ]; then
  166.  
  167. -       # lofs mount /boot directory under /tftpboot
  168. +       # cp /boot directory under /tftpboot
  169.         if [ ! -f ${IMAGE_PATH}/grub/pxegrub ]; then
  170.                 echo "${myname}: ${IMAGE_PATH}/grub/pxegrub does not exist," \
  171.                     "invalid boot image"
  172.                 cleanup_and_exit 1
  173.         fi
  174. +       if [ ! -f /tftpboot/vfstab ]; then
  175. +               rm -rf /tftpboot/vfstab
  176. +               echo '# Need this file to remember, where /tftpboot/I86PC.Solaris_10-* comes from.
  177. +# So do not delete it - maintained by x86 add_install_client itself' \
  178. +                       >/tftpboot/vfstab
  179. +       fi
  180.  
  181. -       # Check if it is already mounted
  182. -       line=`grep "^${IMAGE_PATH}[     ]" /etc/vfstab`
  183. +       # Check if it is already copied
  184. +       line=`grep "^${IMAGE_PATH}[     ]" /tftpboot/vfstab`
  185. +       DIR_EXISTS=""
  186.         if [ $? = 0 ]; then
  187.                 mountpt=`echo $line | cut -d ' ' -f3`
  188.                 BootLofs=`basename "${mountpt}"`
  189.                 BootLofsdir=`dirname "${mountpt}"`
  190. -               if [ ${BootLofsdir} != ${Bootdir} ]; then
  191. -                       printf "${myname}: ${IMAGE_PATH} mounted at"
  192. -                       printf " ${mountpt}\n"
  193. -                       printf "${myname}: retry after unmounting and deleting"
  194. -                       printf " entry form /etc/vfstab\n"
  195. -                       cleanup_and_exit 1
  196. +               if [ ${BootLofsdir} != ${Bootdir} -o ! -f $mountpt/multiboot ]; then
  197. +                       rm -rf "${mountpt}"
  198. +                       tf=`mktemp /tmp/vfstab.XXXXX`
  199. +                       grep -v "^${IMAGE_PATH}[     ]" /tftpboot/vfstab >$tf
  200. +                       cp $tf /tftpboot/vfstab
  201. +                       rm -f $tf
  202. +               else
  203. +                       DIR_EXISTS="y"
  204.                 fi
  205. -
  206. -               # Check to see if the mount is sane, if not, kick it.
  207. -               #
  208. -               # Note: One might think that the case when kicking the
  209. -               #       mounpoint won't work should then be handled, but
  210. -               #       if that were the case, the code path for no existing
  211. -               #       mount would have been taken resulting in a new
  212. -               #       mountpoint being created.
  213. -               #
  214. -               if [ ! -f $mountpt/multiboot  ]; then
  215. -                       umount $mountpt
  216. -                       mount $mountpt
  217. -               fi
  218. -       else
  219. +       fi
  220. +       if [ -z "$DIR_EXISTS" ]; then
  221.                 # get a new directory name and mount IMAGE_PATH
  222.                 max=0
  223.                 for i in ${Bootdir}/I86PC.${VERSION}* ; do
  224. @@ -1945,14 +1942,15 @@
  225.  
  226.                 BootLofs=I86PC.${VERSION}-${max}
  227.                 mkdir -p ${Bootdir}/${BootLofs}
  228. -               mount -F lofs -o ro ${IMAGE_PATH} ${Bootdir}/${BootLofs}
  229. +               cd ${IMAGE_PATH}
  230. +               find . | cpio -puvmd ${Bootdir}/${BootLofs}
  231.                 if [ $? != 0 ]; then
  232. -                       echo "${myname}: failed to mount ${IMAGE_PATH} on" \
  233. +                       echo "${myname}: failed to copy ${IMAGE_PATH} to" \
  234.                            "${Bootdir}/${BootLofs}"
  235.                         cleanup_and_exit 1
  236.                 fi
  237. -               printf "${IMAGE_PATH} - ${Bootdir}/${BootLofs} " >> /etc/vfstab
  238. -               printf "lofs - yes ro\n" >> /etc/vfstab
  239. +               printf "${IMAGE_PATH} - ${Bootdir}/${BootLofs} " >> /tftpboot/vfstab
  240. +               printf "lofs - yes ro\n" >> /tftpboot/vfstab
  241.         fi
  242.         # cleanup of lofs mount is done after Menufile setup
  243. #################### stop patch #######################################
  244.  
  245.  
  246. 8) The last thing I still need to find out is, how to get nv b57 to
  247.    install the correct bzipped boot_archives:
  248.    Right now, it seems to pretty inefficient wrt / FS. I.e. to get usable
  249.    boot_archives at all, I need to manually
  250.         rm -f /a/platform/i86pc/boot_archive
  251.         rm -f /a/platform/i86pc/amd64/boot_archive
  252.         bootadm update-archive -R /a
  253.    in the finish script. Braindeadly bootadm than creates these files
  254.    again with same size AND UNCOMPRESSED!!! So when the system tries to
  255.    boot the first time, one gets several "NOTICE: alloc: /: file system full"
  256.    warnings and the system doesn't come up, but asking for
  257.    Root password for system maintenance:
  258.  
  259.    # df -k
  260. Filesystem            kbytes    used   avail capacity  Mounted on
  261. /dev/dsk/c5t0d0s0     497312  483184       0   100%    /
  262.  
  263.    # ls -al /platform/i86pc/boot_archive /platform/i86pc/amd64/boot_archive
  264. -rw-r--r--   1 root     root     112349184 Feb 22 05:43 /platform/i86pc/amd64/boot_archive
  265. -rw-r--r--   1 root     root     112349184 Feb 22 05:43 /platform/i86pc/boot_archive
  266.  
  267.    Hello, a / partition of 512 MB is not enough? That's rediculous!
  268.    IMHO 256 MB is still pretty large and should be sufficient for x86 Solaris
  269.    as well!
  270.  
  271.    To fix the problem, I need to setup an /a/etc/rcS.d/S80firsttime script,
  272.    which recreates the boot archives early in the boot process:
  273.    
  274.    # rm /platform/i86pc/boot_archive /platform/i86pc/amd64/boot_archive
  275.    # bootadm update-archive
  276.  
  277.    # df -k
  278. Filesystem            kbytes    used   avail capacity  Mounted on
  279. /dev/dsk/c5t0d0s0     497312  302576  145005    68%    /
  280.    # ls -al /platform/i86pc/boot_archive /platform/i86pc/amd64/boot_archive
  281. -rw-r--r--   1 root     root     20664320 Feb 22 05:59 /platform/i86pc/amd64/boot_archive
  282. -rw-r--r--   1 root     root     19175424 Feb 22 05:59 /platform/i86pc/boot_archive
  283.  
  284.    So this thing is pretty "unique" and not wanted. Just remember, on a
  285.    Linux-Box with a / fs of 256 MB one is able to host about 10 kernels
  286.    incl. its full blown driver archive aka /lib/modules/2.6.* ...
  287.  
  288.  
  289. 9) More or less  minor: x86 probe_rootdisk does not find a boot disk
  290.    on a X4500, which actually causes a div by zero awk error.
  291.    So here one needs to do additional work again, to be able to
  292.    do SI_ROOTDISK related tasks in jumpstart...

Raw Paste


Login or Register to edit or fork this paste. It's free.