Linux installation guide
Linux General Policy and Installation instructions
This document contains guidelines in setting up/configuring and updating of Linux systems. If you need to operate the system differently then a clear argument must be made and the decisions taken documented. In any case an accurate account of what has been done for each machine (server) MUST be made in the \\electra\docs\System-configs folder. Clients do not need an entry in this folder but it is nice to have a group config file ex. linux-rackables.
For Linux:
* we have standardized on the Fedora distribution of Linux
* our current standard release is Fedora Core 3
** Our new standard is Fedora Core 4 to which we are moving
to slowly. --ank 19:16, 27 Οκτ 2005 (EEST)
* we also operate our own update service for doing updates
* each machine installed MUST operate with these parameters and the following
guideliness unless it is strictly for evaluation purposes.
Installing Linux
Filesystem structure
* Filesystem configuration
We always use LVM version 2 and up for setting up Fedora filesystems
* Physical partitions and volume groups set up
o One physical partition for /boot with 100MB ( MUST be outside of LVM set up)
o One physical partition, large enough, to contain the LVM Volume group for
OS install (VolGroup00).
Suggested size is 15,000MB. This gives enough space for installing and
expanding later if required.
o One LVM Volume group (VolGroup00) for OS. See above.
o One LVM Volume group (VolGroup01 or DataVG01 etc) for DATA (if required).
This is prepared only if a very large volume of data is anticipated or if
we determine that a separete volume group is necessary or desirable.
Otherwise we can create a filesystem within the OS volume group. It can
also be constrained into a physical drive if required.
* Filesystems
o Within the VolGroup00 we have:
(root) / 1024MB (1GB)
/usr 4096MB (3GB) for server machines
5120MB (5GB) for client machines
(can be increased or decreased if requirements demand it)
/var 1024MB (1GB)
/tmp 512MB
/usr/local 512MB
/opt 512MB
/home 512MB (before mounts)
swap space
this depends on what the system will be doing. If physical memory is
large enough >= 1GB then swap=memory.
/sys-data (optional, see above)
Remaining of disk space.
Data file-system (in case VolGroup00 will not be used for data storage).
o Within VolGroup01, if it exists, create a filesystem for data and mount it
to a suitable directory node (prefer /sys-data). This file system can be any
format of ext3, xfs, jfs or reiserfs. If extra speed is required prefer xfs
or reiserfs.
Security
- The IPtables system should be installed and operational on ALL client and server machines
Services and Applications installed
A list of services and applications that the machine is being used for must be maintained.
Client machines
Server machines
- Install only services absolutely required for the server functions.
- Server machines (web, mail, db etc) install also X-Window system, Gnome desktop and all administration utilities. This helps in managing the system. If it is preferred to disable the X-Window interface on the machine then do so in /etc/inittab.
C. Configuration of Linux systems
C.1 Network
We prefer a DHCP assigned network configuration.
Note: The majority of the Linux clients do not send their hostname to the DHCP server when they obtaining IP address. As a result dynamic DNS updates do not work. This can be resolved by adding the following line in the configuration file of dhclient, usually /etc/dhclient (if the file does not exist, i.e FC4, create it).
send host-name "hostname";
C.2 NTP
* the following minimum configuration must be made in /etc/ntp.conf
keys /etc/ntp/keys
server 194.42.16.20
restrict 194.42.16.20 mask 255.255.255.0 nomodify notrap
* delete (or comment out) any other server xx.xx.xx.xx lines
C.3 Automounting with autofs
Autofs controls the operation of the automount daemons. The automount daemons automatically mount filesystems when you use them and unmount them after a period of inactivity. To configure autofs we have to edit the file /etc/auto.master.
# This is an automounter map and it has the following format # key [ -mount-options-separated-by-comma ] location # For details of the format look at autofs(5). /home /etc/auto.home --ghost
This specifies that any attempt to access folders within /home triggers a reference to the /etc/auto.home file. Each of the items in the first field serves as a root for mounting. As such, it should exist in the file system as an empty directory, the automounter will handle the creation of whatever subdirectories are needed. The --ghost option directs the automounter to create empty directories of all the mount points listed in the configuration file, regardless of whether any of the file systems is actually mounted or not. The mount points are then defined in the configuration file, in our case, /etc/auto.home.
courses -rw nas200:/courses faculty -rw nas200:/faculty projects -rw nas200:/projects research -rw nas200:/research students/cs -rw uls:/home/students/cs support -rw nas200:/support
The last step is to configure autofs to run each time the system is started.
chkconfig autofs on
C.4 Authentication
C.4.1 NIS
NIS works best if the client does an autodiscover of available server when needed. If for some reason the auto-discover mode (broadcast) does not work then we prefer to have at least two servers for each client in the yp.conf file and the option to auto-discover (broadcast) if both fail. Minimum example of a /etc/yp.conf file:
# /etc/yp.conf - ypbind configuration file # Valid entries are # # domain NISDOMAIN server HOSTNAME # Use server HOSTNAME for the domain NISDOMAIN. # # domain NISDOMAIN broadcast # Use broadcast on the local net for domain NISDOMAIN # # domain NISDOMAIN slp # Query local SLP server for ypserver supporting NISDOMAIN # # ypserver HOSTNAME # Use server HOSTNAME for the local domain. The # IP-address of server must be listed in /etc/hosts. # # broadcast # If no server for the default domain is specified or # none of them is rechable, try a broadcast call to # find a server. # domain csnis server nis1.cs.ucy.ac.cy domain csnis server nis2.cs.ucy.ac.cy broadcast
Note: If you are using dynamic IP address assignment through a DHCP server then the file /etc/yp.conf is controlled by the dhcp client procedure and will be overwritten by it. Make sure that the DHCP server assigns proper NIS domain and server values.
C.4.2 LDAP
To authenticate against an LDAP server do the following:
Logging and monitoring (/var/log, Nagios, syslogd, snmp)
- Each system should have its own logging facility enabled (writing to /var/log files).
- Attention should be payed so that /var/log/ files are properly rotated so that the /var filesystem does not run the risk of overflowing.
- We are using the Nagios system to monitor our systems. In order to allow proper monitoring of Linux systems the following RPMs must be installed:
- The NRPE (Nagios Remote Process Execution) system RPM
- The Nagios Plug-ins RPM
To install these do the following after you set up the systems for correct updating as described in Section D below:
yum install nrpe yum install nagios-plugins This is applicable to Fedora Core 4. Do not try it on earlier systems. Upgrade!
Configuring the NRPE environment
In order to allow the NRPE to work properly with our monitoring system do the follwoing:
Edit the /etc/nagios/nrpe.cfg file as follows:
allowed_hosts=194.42.16.17
dont_blame_nrpe=1
Check that the port chosen in /etc/nrpe.cfg (default 5666) has access to this machine. The iptables configuration will likely need to be changed.
It must be emphasized that this is a minimal configuration and any special monitoring cases must be coordinated between the Nagios admin the the Linux machine admin.
C.6 Security
- All systems must have remote root access disabled (including ssh)
- All systems must have telnet disabled (ssh is enabled)
On systems that ssh access does not need to be open for public access then disable such access.
Example: For mostly closed systems (servers) then do the following
File /etc/hosts.deny have the following line:
ALL: ALL
File /etc/hosts.access have the following line (or something similar depending on what you want to allow)
ALL: .cs.ucy.ac.cy, .in.cs.ucy.ac.cy
Restart sshd by
service sshd restart
D. Updating Linux
The CS dept operates its own yum server for upgrading all Linux machines. Updates are currently maintained for FC2, FC3 and FC4. This list will gradually change according to the active releases we are using (see above). It is expected that we will maintain at least three releases at all times. Updates are downloaded/refreshed every Sunday at 2:00 am. Each machine should use this system and not external update sources to minimize network traffic.
In order for a Linux Fedora machine to access the YUM server it must be configured to do so. A working configuration is available on system eris (also the yum server machine). There is a problem with the gpg key import.Disabling gpgkey=0 in yum.conf fixes it but...
D.1 For Fedora Core2
in the file /etc/yum.conf replace the following lines: name=Fedora Core $releasever - $basearch - Base baseurl=http://download.fedora.redhat.com/pub/fedora/linux/core/$releasever/$basearch/os/ [updates-released] name=Fedora Core $releasever - $basearch - Released Updates baseurl=http://download.fedora.redhat.com/pub/fedora/linux/core/updates/$releasever/$basearch/ with [base] name=Fedora Core $releasever-$basearch - Base on CS-UCY Yum server baseurl=ftp://yum.cs.ucy.ac.cy/pub/linux/fedora/core/$releasever/$basearch/os [updates-released] name=Fedora Core $releasever-$basearch - Released Updates on CS-UCY Yum server baseurl=ftp://yum.cs.ucy.ac.cy/pub/linux/fedora/core/updates/$releasever/$basearch
D.2 For Fedora Core3 and later
- make sure that the reposdir parameter is NOT defined in /etc/yum.conf. - in the file /etc/yum.conf add the lines [base] name=Fedora Core $releasever-$basearch - Base on CS-UCY Yum server baseurl=ftp://yum.cs.ucy.ac.cy/pub/linux/fedora/core/$releasever/$basearch/os [updates-released] name=Fedora Core $releasever-$basearch - Released Updates on CS-UCY Yum server baseurl=ftp://yum.cs.ucy.ac.cy/pub/linux/fedora/core/updates/$releasever/$basearch [extras] name=Fedora Core $releasever-$basearch - Extra packages on CS-UCY Yum server baseurl=ftp://yum.cs.ucy.ac.cy/pub/linux/fedora/extras/$releasever/$basearch [cs-extras] name=Fedora Core $releasever-$basearch - CS - Extra packages on CS-UCY Yum server baseurl=ftp://yum.cs.ucy.ac.cy/pub/linux/cs-extras/fedora/$releasever
D.3 RedHat's up2date
If we like to be able to use also the up2date tool provided by RedHat and the up2date applet on our desktop then the following configuration should be made in /etc/sysconfig/rhn/sources:
* comment out the default yum configuration
* insert the following lines:
o yum fedora-core ftp://yum.cs.ucy.ac.cy/pub/linux/fedora/core/3/$ARCH/os
o yum updates-released ftp://yum.cs.ucy.ac.cy/pub/linux/fedora/core/updates/3/$ARCH
o yum extras ftp://yum.cs.ucy.ac.cy/pub/linux/fedora/extras/3/$ARCH
o yum cs-extras ftp://yum.cs.ucy.ac.cy/pub/linux/cs-extras/fedora/3
Unfortunately up2date cannot detect the release level installed and therefore it must be configured for the correct release version of Fedora. Above we have it for Fedora Core3.
D.4 Update Policy
The following policy MUST be adhered to in upgrading all Linux machines managed by Support: Downloading of upgades are done every Sunday night.
D.4.1. Client Machines
Client machines should be updated weekly with all the updates available (kernel included). Particular attention is payed to security updates.
* It is advisable that NOT all machines are upgraded the same day so that in case of error in the updates only a small number of clients will be affected. One possible way to do this is to have all machines with system number ending to 1 or 2 (ex CS101, CS102) make an upgrade on Monday, number 3,4 on Tuesday etc.
* It is advisable that critical software not be allowed to update automatically unless we
have determined (by doing a manual update) that updates actually work. Critical software
are:
--- kernel (see also below)
--- applications software that MUST be available
(ex. C compiler in teaching labs)
D.4.2 Server Machines
D.4.3 Things to watch for
When upgrading Linux systems watch for the following:
1. when upgrading kernels the /root partition tends to become full.
This is mainly because the /lib directory grows for ever since upgrading the kernel
does not remove the old ones and /lib/modules and other directories get large.
To remove a kernel do something like:
yum remove kernel-2.6.11-1.14_FC3
or rpm -e kernel-2.6.11-1.14_FC3
The best practice should be to leave around 2-3 generations of kernels in case
we need them.
D.5 Automatic Installation using kickstart
Linux enables administrator to prepare a configuration file,named ks.cfg in order to prepare multiple clients, with minimal effort. Kickstart file, can be placed on a floppy drive, on bootable cd, or on ftp, http server.
Click here to find out how this can be done.
D.6 Bug with the anaconda installer for FC4 (at least)
There is a bug with the anaconda installer for FC4 (it may appear on other releases also) which gives partly the following error immediately before filesystems are ready to be formatted:
The kernel was unable to re-read the partition table on /dev/hda .......... (Device or resource busy).
This error appears only on pre-partitioned systems that need to be reinstalled with a different partiiton scheme. To resolve this issue:
- boot from the install CD
- go into rescue mode (F5)
- when you are at the promp type
dd if=/dev/zero of=/dev/hda bs=512 count=1
- reboot and continue as usual
D.7 Bug with ks.cfg and %packages section (only for FC4 (at least))
If you need to exclude packages in ks.cfg file,then you have to exclude them as -<packagename>.platform i.e -perl-XML-SAX.i386, otherwise you will get excemption errors during installation and it will be failed.