Notes on setting up the unidata LDM on a LinuxArch system; last updated:20130521




Host computer: heron



Linux Arch Installation

OS: Linux version 3.7.10-1-ARCH

  1. Arch Generalities

    The setup was based on the instructions given here Notes having to do with Arch Linux on Gryphon and friends. Here, I just want to point out some generalities of arch that help to know when following the installation instructions.

    1. Most notably, arch uses the command systemd as the system and service manager, which in turns is controlled by the command systemctl.

    2. The page on systemd also provides an explanation for the concepts of units and targets, which handle configuration files and run levels.

    3. The arch filesystem hierarchy can be found at File system hierarchy

    4. The logging system is handled by journal

    5. The package manager is pacman

  2. Sticky Points

    1. using the default repository did not work: mkinitcpio -p linux reported an error of kernel image not found. Solution: used different repository (Georgia Tech works)

    2. After the first reboot, the internet was not up, with no clues about why. It turns out that a mistake in the file /etc/conf.d/netcfg was the problem. I did not realized this at first because during the installation a temporal network was set up, which gave me the impression that all configuration files were keep for the permanent installation (after all the internet was already working). However,the temporal setup is done by modifying /sys/class/net and /etc/resolv.conf; the permanent is set up with the files /etc/network.d/external, /etc/conf.d/netcfg, and /etc/resolv.conf (which stays the same). So, be sure to check these files if the network is not up after rebooting.

  3. Special instructions for heron

Note: that heron is intended to be an stand alone server ( no resources from gryphon) instead of mounting remotely grypphon:/usr/local/etc, just copy its content over;

  1. heron needs to share its data disk with merlin: modify the /etc/exports to include the line

    /data.w merlin(rw)

    and then add the daemons: systemctl enable rpcbind.service;

    systemctl enable rpc-idmapd.service;

    systemd enable rpc-mountd.service

  2. to enable the daily backups of the home directories we need

    1. to have pass-wordless connections to merlin and peregrine, so just copy over the public key (it was generated during the installation of ssh) to those machines. (this site is a good reference on pas-wordless connections: everyDay_LHT .)

      1. ssh-copy-id -i ~/.ssh/id_rsa.pub root@peregrine

      2. ssh-copy-id -i ~/.ssh/id_rsa.pub root@merlin

    2. The backups are manage by the root crontab --a copy of the crontab.root is located in /home/ldm

      1. become root and cp /home/ldm/crontab.root /etc/rootcrontab

      2. crontab -u /etc/rootcrontab

  3. In order to monitor the system, we need EXIM to work. Just follow the instructions in EXIM notes

  4. We need to create the group ldm and the users: ldm, mcidas

    1. groupadd ldm

    2. useradd -m -g users -G ldm -s /bin/bash ldm ; passwd ldm ; su -l ldm and run arch-setup

    3. mcidas don't have to belong to the ldm group, but its home directory must be readable and serchable by the ldm user (for accessing libraries and headers)

      useradd -m -g users -s /bin/bash mcidas ; passwd mcidas ; su -l mcidas and run arch-setup; chmod -R g+rX /home/mcidas/

LDM installation

The ldm requires a decoder from the package ldm-mcidas, namely (pnga2area). In order to compile this package, we need mcidasx (netcdf comes bunfled with this package), this in turns needs a FORTRAN compiler.

  1. Fortran Compiler

    1. make sure that you have gfortran: pacman -S gcc-fortran

  2. lMCIDASX

    1. login as mcidas and open a browser to download the latest mcidasx package.

    2. follow the instructions at Installing McIDAS-X on Unix or Mac OS X Workstations

  3. LDM-MCIDAS

    1. login as ldm and open a browser to download the latest ldm-mcidas package

    2. unpack according to the istructions at LDM-McIDAS:Building and Installing.

    3. cd to /home/ldm/ldm-mcidas/ldm-mcidas-2012/src

    4. set the "CUSTOMIZE" file. (a copy of the one that worked is here ~/ldm_mcidas_customize.set )

    5. run ./configure

      ## Note: “make all” did not work -- it is trying to build some deprecated libs. instead just build decode_basic

    6. make decode_basic > ldm-mcidas.log 2>&1

    7. make decoders_install > ldm-mcidas-decoders_install.log 2>&1

    8. make decode_test > ldm-mcidas-decode_test.log 2>&1

    9. the required file is at /home/ldm/ldm-mcidas/bin/pnga2area copy or link this file to the directory /home/ldm/decoders/satellite

  4. LDM

    1. login as ldm and open a browser to download the latest ldm package.

    2. Modify the files .bash_profile and .profile with copies from the backup directory ~/ldm

    3. Unpack and install according to Installing, Configuring, and Activating the LDM System (follow the Long Installation Instructions)

    4. activate the ldm crontab – a copy is at ~ldm/crontab.ldm

    5. Wrinkles

      1. ldm not able to write to its own log files.

        There are some tips about why and what to do in this ldm-archived discussion . So far the system is working, but the logs are own and written by root. The current workaround is to add the ldm user to the log group, so it is able to check the logs (usermod -a -G log ldm).

      2. The crontab sent the following message:

        bin/scour: line 171: /usr/local/ldm/etc/scour.conf: No such file or directory

        after I activated the line to purge old data (0 8 * * * bin/scour > /dev/null).

The work around is to make a symbolic link of the ldm dir in /usr/local





APPENDIX

/etc/exports

/home/ldm/crontab.root

/home/ldm/crontab.ldm

/home/ldm/.bash_profile

/home/ldm/.profile