LPI-201: Hướng dẫn Administration Level Intermediate (Cấp độ Trung cấp)

LPI-201 Intermediate Level Administration

Exam: 117-201

Lưu hành nội bộ

SaigonCTT – V1.0

INDEX

Chapter 1

The Linux Kernel

Chapter 2 Hardware Management

Chapter 3

Linux System Startup

Chapter 4

Linux Filesystem

Chapter 5 RAID and LVM

Chapter 6

File Sharing and Services - NFS

Chapter 7

File Sharing and Services - Samba

Chapter 8 Recovering a Linux System

Chapter 9 Part1: Regular Expression

Chapter 9 Part2: Perl Language

Chapter 9 Part3: System Logging Automation

Chapter 10 Troubleshooting

LPI 201 Intermediate Level Administration –– LPI 201 Intermediate Level Administration

Chapter 1 Chapter 1

Linux Kernel Linux Kernel

Last updated: 02 March 2004

1 © Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – The Linux Kernel> - 1

Objectives Objectives

• Kernel Terminology

• Kernel Components

• Compiling a Kernel

• Patching a Kernel

• Customizing a Kernel

Last updated: 06 December 2004

2

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – The Linux Kernel> - 2

Kernel Terminology Kernel Terminology

• Kernel

– A binary file which is created as a result of compiling the

kernel sources.

• Kernel sources

– large collection of text files mostly written in the C

programming language.

• Vendor kernel

– A kernel that has been patched by a vendor (i.e. Red Hat or

Debian).

• Patch

– A set of instructions to modify a file or set of files from an

old version to a new one.

Last updated: 06 December 2004

3

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – The Linux Kernel> - 3

Kernel Components Kernel Components

• Different types of kernel images

– monolithic kernel

• contains all the driver code

• does not need the assistance of modules

– kernel image is compressed to save space

• make zImage

• make bzImage

Last updated: 06 December 2004

4

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – The Linux Kernel> - 4

Kernel Components Kernel Components

• Identifying stable and development kernels

– Kernel version are made by three basic components:

• The major number

• The minor number

• The micro number (patch number)

– There is the fourth number

• Sometimes applied after a dash is generally the patch level , applied

by kernel maintainers

– The minor number is even = stable version

– The minor number is odd = development version.

Ex : 2 . 4 . 18 - 10

Last updated: 06 December 2004

5

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – The Linux Kernel> - 5

Kernel Modules Kernel Modules

• insmod

– Install & binding a module to kernel.You can customize

module loadtime parameters in /etc/modules.conf or /etc/conf.modules

• rmmod

– Unload a module from kernel

• modinfo

– Details about a module’s description

• modprobe

– Load a set of modules either a single module a stack of

dependent modules or all modules that are marked with a specified tag

( Package : modutils )

Last updated: 06 December 2004

6

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – The Linux Kernel> - 6

Special Kernels Special Kernels

• Symmetric Multi Processing kernels - smp

– Support Multi Processors (>1 )

Ex: kernel-smp-2.4.21.0.1.EL.i386.rpm

• Support large memory ( >4Gb)

Ex: kernel-bigmem-2.4.20-31.9.i386.rpm

Last updated: 06 December 2004

7

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – The Linux Kernel> - 7

Compiling a Kernel Compiling a Kernel

• Prepare software utilities

• Getting the kernel sources • Creating a .config file • Compiling the kernel

• Installing the new kernel

• The mkinitrd command

Last updated: 06 December 2004

8

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – The Linux Kernel> - 8

Software Utilities Software Utilities

• make

– Determines which pieces of a large program need to be

recompiled and compile them.

– Executes commands in Makefile (sometime named makefile)

to update programs or module components

• C/gcc :

– C compiler integrated into gcc

• binutils:

– a collection of binary utilities

• gas(assembler), ld(likner),nm, ranlib, objdump, …

Last updated: 06 December 2004

9

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – The Linux Kernel> - 9

Getting the Kernel sources Getting the Kernel sources

• The latest Linux kernel sources at:

http://www.kernel.org

Last updated: 06 December 2004

10

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – The Linux Kernel> - 10

The Source Tree The Source Tree

• Kernel source released as a zipped tar file

Ex: linux-2.4.29.tar.bz2

• NEVER unpack your kernel sources directly into

/usr/src/linux

# tar xjvf linux-2.4.29.tar.bz2 –C /usr/src/

Last updated: 06 December 2004

11

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – The Linux Kernel> - 11

The Source Tree The Source Tree

• Subdirectories in the source tree :

/usr/src/linux/

arch

ipc

crypto

kernel

Documentation

lib

drivers

mm

fs

net

include

scripts

init

• This is the structure that most kernel sources will follow.

Last updated: 06 December 2004

12

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – The Linux Kernel> - 12

.config filefile

Creating a .config Creating a

• Create new .config file

– make config or – make menuconfig or – make xconfig

• Create from exists .config file

– Copy exists .config file to /usr/src/linux/ – make oldconfig or – make menuconfig with load exists config file.

• The results of all of these choices are stored in:

/usr/src/linux/.config

Last updated: 06 December 2004

13

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – The Linux Kernel> - 13

.config filefile

Creating a .config Creating a

• Verify : allow load modules

# cat .config | grep -i module

# Loadable module support CONFIG_MODULES=y

• Verify : support ext3

# cat .config | grep -i ext3

CONFIG_EXT3_FS=m CONFIG_EXT3_INDEX=y CONFIG_EXT3_FS_XATTR=y CONFIG_EXT3_FS_XATTR_SHARING=y CONFIG_EXT3_FS_XATTR_USER=y CONFIG_EXT3_FS_POSIX_ACL=y

Last updated: 06 December 2004

14

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – The Linux Kernel> - 14

Interface make menuconfig Interface make menuconfig

Last updated: 06 December 2004

15

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – The Linux Kernel> - 15

Interface make xconfig Interface make xconfig

Last updated: 06 December 2004

16

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – The Linux Kernel> - 16

Compiling the Kernel Compiling the Kernel

(cid:131) make dep

– Source files are examined for dependencies – The resulting table is stored in .depend file. – The .depend files are automatically included in make.

• make clean

– Removes old output files that may exist from previous

kernel builds.

• make bzImage and make zImage

– Our ultimate goal is a bootable kernel image file.

Last updated: 06 December 2004

17

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – The Linux Kernel> - 17

(cont.) Compiling the Kernel (cont.) Compiling the Kernel

• make modules

– Device drivers and other items that were configured as

modules are compiled.

• make modules_install

– All of the modules compiled are installed under

/lib/modules/kernel-version

Last updated: 06 December 2004

18

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – The Linux Kernel> - 18

Installing the new kernel Installing the new kernel

(cid:131) *Copy new kernel to /boot

cp arch/i386/boot/bzImage

/boot/vmlinuz-2.4.29

(cid:131) Copy Sytem.map to /boot

cp Sytem.map /boot/System.map-2.4.29

(cid:131) Copy .config to /boot

cp .config /boot/config-2.4.29

Last updated: 06 December 2004

19

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – The Linux Kernel> - 19

command mkinitrd command

TheThe mkinitrd

• Adds the capability to load a RAM disk by the boot

loader.

• Mainly designed system startup to occur in two

phases: – the kernel comes up with a minimum set of compiled-in

drivers

– additional modules are loaded from initrd

• mkinitrd /boot/initrd-2.4.29.img 2.4.29

Last updated: 06 December 2004

20

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – The Linux Kernel> - 20

Edit boot loader –– lilolilo Edit boot loader

(cid:131) Add new entry to /etc/lilo.conf

timeout=50

default=linux

boot=/dev/hda ….

image=/boot/vmlinuz-2.4.29

label=linux

initrd=/boot/initrd-2.4.29.img

read-only

append="hdc=ide-scsi root=LABEL=/"

(cid:131) Execute /sbin/lilo -v

Last updated: 06 December 2004

21

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – The Linux Kernel> - 21

Edit boot loader –– grubgrub Edit boot loader

(cid:131) Add new entry to /etc/grub.conf

default=15 timeout=10 splashimage=(hd0,0)/grub/splash.xpm.gz …….

title Red Hat Linux (2.4.9)

root (hd0,0) kernel /vmlinuz-2.4.9 ro root=LABEL=/

hdc=ide-scsi

initrd /initrd-2.4.9.img

Last updated: 06 December 2004

22

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – The Linux Kernel> - 22

Patching a Kernel Patching a Kernel

• A patch file is editing instructions to change one set of

files into new version of the files Ex: patch-2.4.29.bz2

• Steps to patch :

# ln –s /usr/src/linux-2.4.29 /usr/src/linux

# cp patch-2.4.29.bz2 /usr/src/linux

# cd /usr/src/linux

# bzcat patch-2.4.29.bz2 | patch –p1

Last updated: 06 December 2004

23

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – The Linux Kernel> - 23

Patching a Kernel Patching a Kernel

• Testing the Patch only, before applying

patch –p1 -–dry-run < patchfile

• Keep original files

patch –b –p1 < patchfile

• Keep original files to backup-directory

patch –B backup-dir –p1 < patchfile

Last updated: 06 December 2004

24

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – The Linux Kernel> - 24

Remove patch from Kernel Remove patch from Kernel

• A kernel patch can be removed from a kernel: – removing it from the production kernel source tree – compiling a new kernel.

• Try to apply the patch a second time for remove

# bzcat patch-2.4.29.bz2 | patch –p1

• Find any patch files failed:

find . -name '*.rej'

Last updated: 06 December 2004

25

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – The Linux Kernel> - 25

Remove patch from Kernel Remove patch from Kernel

• A kernel patch can be removed from a kernel: – removing it from the production kernel source tree

– compiling a new kernel.

• Try to apply the patch a second time for remove

# bzcat patch-2.4.29.bz2 | patch –p1

Last updated: 06 December 2004

26

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – The Linux Kernel> - 26

Edit Kernel version Edit Kernel version

• Kernel version is defined Makefile file

# cd /usr/src/linux

# vi Makefile

VERSION = 2

PATCHLEVEL = 4

SUBLEVEL = 29

EXTRAVERSION =

KERNELRELEASE=$(VERSION).$(PATCHLEVEL).$(SUBLEVEL)$(EXTRAVERSION)

<.........>

Last updated: 06 December 2004

27

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – The Linux Kernel> - 27

kmod versus kmod

versus kerneld kerneld

• kmod and kerneld make dynamic loading of kernel-modules.

• use modprobe to manage dependencies and dynamic loading of modules

• kerneld is a daemon • kmod is a thread in the kernel itself.

• kerneld is done through System V IPC. • kmod operates directly from the kernel

• kmod replaces kerneld as of Linux kernel 2.2.x.

# cat .config | grep -i kmod CONFIG_KMOD=y

Last updated: 06 December 2004

28

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – The Linux Kernel> - 28

Summary Summary

• Kernel Terminology

• Kernel Components

• Compiling a Kernel

• Patching a Kernel

• Customizing a Kernel

Last updated: 06 December 2004

29

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – The Linux Kernel> - 29

Question & Answer Question & Answer

Last updated: 06 December 2004

30

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – The Linux Kernel> - 30

LPI 201 Intermediate Level Administration –– LPI 201 Intermediate Level Administration

Chapter 2 Chapter 2

Hardware Management Hardware Management

Last updated: 02 March 2004

1 © Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Hardware Management Chapter 2 - 1

Objectives Objectives

• Adding New Hardware

• Software And Kernel Configuration

• Configuring PCMCIA Devices

Last updated: 06 December 2004

2

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Hardware Management Chapter 2 - 2

Adding New Hardware Adding New Hardware

• Bus structures

• USB devices

• Configuring disks

• Serial devices

• Configuring output devices

Last updated: 06 December 2004

3

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Hardware Management Chapter 2 - 3

systems The main PC bus--systems The main PC bus

• ISA

– 16 or 8bit, cheap, slow (usually 8MHz), standard, many cards available,

but not many new motherboards are shipped with ISA anymore;

• EISA

– 32bit, expensive, fast, few cards available, but almost obsolete.

• MCA

– 32 or 16bit ex-IBM-proprietary, fast, obsolete/rare.

• VESA-Local-Bus

– 32bit, based on 486 architecture, cheap, fast, many cards available,

obsolete.

• PCI-Local-Bus

– 32bit (64 bit coming), cheap, fast, many cards available, the de facto

standard

Last updated: 06 December 2004

4

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Hardware Management Chapter 2 - 4

USB devices USB devices

• usbview

– find out which USB devices are currently connected.

– provides a graphical display.

– depends on X and the GTK toolkit.

– not always available on all USB aware distributions.

– http://sourceforge.net/projects/usbview/

Last updated: 06 December 2004

5

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Hardware Management Chapter 2 - 5

USB devices USB devices

• The USB-device filesystem is a dynamically generated

filesystem. – To mount it “by hand”:

# mount -t usbdevfs none /proc/bus/usb # ls -l /proc/bus/usb dr-xr-xr-x 1 root root 0 Mar 10 09:42 001

dr-xr-xr-x 1 root root 0 Mar 10 09:42 002

dr-xr-xr-x 1 root root 0 Mar 10 09:42 003

dr-xr-xr-x 1 root root 0 Mar 10 09:42 004

-r--r--r--

1 root root 0 Mar 10 09:42 devices

-r--r--r--

1 root root 0 Mar 10 09:42 drivers

Last updated: 06 December 2004

6

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Hardware Management Chapter 2 - 6

USB devices USB devices

• USB was supported in Kernel 2.2.7 but was not incorporated

until the 2.4 kernel

Host controler

Kernel Module

OHCI (Compaq)

usb-ohci.o

UHCI (Intel)

usb-uhci.o

EHCI (USB v2.0)

ehci-hdc.o

Last updated: 06 December 2004

7

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Hardware Management Chapter 2 - 7

mount & umount USB disk mount & umount USB disk

• Mount USB disk (with windows format )

# mkdir /usbdisk

# mount –t vfat /dev/sda /usbdisk

– If you have more USB devices on system, replace /dev/sda with

/dev/sda1|2|3...

• Unmount

# umount /usbdisk

Last updated: 06 December 2004

8

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Hardware Management Chapter 2 - 8

Hard Drivers Hard Drivers

Add a hard driver

• 1. Write down the new drive’s specifications

• 2. Configure the new drive appropriately for your bus

– Setting jumper on the driver for Master, Slave.

• 3. Install new driver

– Ensuring that all data and power cables are attached

• 4. Configure your BIOS

• 5. Boot the Operating Sytem

Last updated: 06 December 2004

9

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Hardware Management Chapter 2 - 9

Hard Drivers Hard Drivers

• 6. Create one or more partitionson hard disk.

# fdisk /dev/hdb

• 7. Format filesystems

# mkfs – t ext3 –j /dev/hdb1

• 8. Create mount point

# mkdir /mnt/newdisk

Last updated: 06 December 2004

10

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Hardware Management Chapter 2 - 10

Hard Drivers Hard Drivers

• 9. Mount the driver

# mount –t ext3 /dev/hdb1 /mnt/newdisk

• 10. Test the new driver

• 11. update /etc/fstab

/dev/hdb1 /mnt/newdisk ext3 defaults 1 1

Last updated: 06 December 2004

11

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Hardware Management Chapter 2 - 11

Serial devices Serial devices

• COM ports are also known as serialports.

• COM ports 1-4 are initialized using default I/O ports

and IRQ values

DOSDOS

IRQIRQ

Linux Linux

I/OI/O

COM1

/dev/ttyS0

0x3f8

COM2

/dev/ttyS1

0x2f8

COM3

/dev/ttyS2

0x3e8

COM4

/dev/ttyS3

0x2e8

Last updated: 06 December 2004

12

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Hardware Management Chapter 2 - 12

Printer Ports (LPT) Printer Ports (LPT)

(cid:131) Most PCs have one physical printer port

(cid:131) The second can be set up, if needed.

DOSDOS

IRQIRQ

Linux Linux

I/OI/O

LPT1

/dev/lp0

0x378

LPT2

/dev/lp1

0x278

Last updated: 06 December 2004

13

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Hardware Management Chapter 2 - 13

Serial devices Serial devices

# cat /proc/interrupts

CPU0

0: 973258 XT-PIC timer

1: 48 XT-PIC keyboard

2: 0 XT-PIC cascade

5: 0 XT-PIC usb-uhci

8: 1 XT-PIC rtc

9: 5436 XT-PIC usb-uhci, eth0

10: 0 XT-PIC ehci-hcd

11: 0 XT-PIC usb-uhci

12: 323 XT-PIC PS/2 Mouse

15: 4342 XT-PIC ide1

Last updated: 06 December 2004

14

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Hardware Management Chapter 2 - 14

Serial devices Serial devices

• Configuring serial devices

setserial device [parameters]

where:

– port [portnumber]: Specify the I/O port address, e.g. 0x2f8 – irq [irqnum]: specify which IRQ line the serial device is using – uart [type]: specify the UART type, e.g. 16550, 16450 or none – autoirq: specify that the kernel should try to figure out the IRQ itself – skip_test: specify that the kernel should skip to test the UART type – autoconfig: the kernel automatically determine the UART type.

Last updated: 06 December 2004

15

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Hardware Management Chapter 2 - 15

Teminal connection Teminal connection

• Check system serial ports

# dmesg | grep tty

ttyS0 at 0x03f8 (irq = 4) is a 16550A

# setserial -g /dev/ttyS[0-3]

/dev/ttyS0, UART: 16550A, Port: 0x03f8, IRQ: 4

/dev/ttyS1, UART: unknown, Port: 0x02f8, IRQ: 3

/dev/ttyS2, UART: unknown, Port: 0x03e8, IRQ: 4

/dev/ttyS3, UART: unknown, Port: 0x02e8, IRQ: 3

Last updated: 06 December 2004

16

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Hardware Management Chapter 2 - 16

Teminal connection Teminal connection

• Configure your inittab to support serial console logins

– Modify /etc/inittab, add line

# Run agetty on COM1/ttyS0 and COM2/ttyS1

s0:2345:respawn:/sbin/agetty -L -f /etc/issueserial 9600 ttyS0 vt100

Where:

-L : force line to be local line with no need for detect (do not have modem)

-f : alternative /etc/issue file. This is what a user sees at the login prompt.

-i : do not display any messages at the login prompt.

9600 : serial line rate in bps.

ttyS0 : this is the serial port identifier. vt100 : is the terminal emulation.

Last updated: 06 December 2004

17

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Hardware Management Chapter 2 - 17

Teminal connection Teminal connection

• stty

– Handle the configuration of the serial port. – Replaced by agetty.

where: -a or --all

print all current setting in human-readable

-F device or --file=device

set the line opened by the filename specified in device

-g or --save

print all current setting in form that can be used for another STTY commands.

Last updated: 06 December 2004

18

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Hardware Management Chapter 2 - 18

Kernel Utilities Kernel Utilities

• lsmod

– List current loaded modules by scnning /proc/modules

# lsmod Module Size Used by Not tainted autofs 13348 0 (autoclean) (unused) 8139too 17704 1 mii 2156 0 [8139too] mousedev 5524 0 (unused) keybdev 2976 0 (unused) hid 22244 0 (unused) input 5888 0 [mousedev keybdev hid] ehci-hcd 17480 0 (unused) ….

Last updated: 06 December 2004

19

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Hardware Management Chapter 2 - 19

Kernel Utilities Kernel Utilities

• modinfo : Display information about the kernel module.

options:

-a, --author

-d, --description

-l, --license

-n, --filename

-p, --parameters

# modinfo usbcore filename: /lib/modules/2.4.29/kernel/drivers/usb/usbcore.o

description:

author:

license: "GPL"

Last updated: 06 December 2004

20

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Hardware Management Chapter 2 - 20

Kernel Utilities Kernel Utilities

• insmod : install module to running kernel.

options:

-f, --force

-k, --autoclean -L, --lock : prevent load the same module. -n, --noload : do everything, except load -o modulename : explicitly name the module -p, --probe -s, --syslog : output to syslog instead of the terminal

Last updated: 06 December 2004

21

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Hardware Management Chapter 2 - 21

Kernel Utilities Kernel Utilities

• modprobe : load a modules stack and its dependencies

options:

-a, --all

-c, --showconfig

-d, --debug

-k, --autoclean -l, --list : list matching modules -r, --remove : do autoclean -t moduletype

Last updated: 06 December 2004

22

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Hardware Management Chapter 2 - 22

Kernel Utilities Kernel Utilities

• rmmod : unload module from running kernel

options: -a, --all : autoclean -r, --stacks

-s, --syslog

Last updated: 06 December 2004

23

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Hardware Management Chapter 2 - 23

Kernel Utilities Kernel Utilities

• /etc/modules.conf: controls the modprobe

# cat /etc/modules.conf alias parport_lowlevel parport_pc

alias eth0 8139too

alias usb-controller ehci-hcd

alias sound-slot-0 i810_audio

......

Last updated: 06 December 2004

24

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Hardware Management Chapter 2 - 24

Hardware utilities Hardware utilities

• lsdev

– part of the procinfo tool suite – list infomation on the hardware installed on the machine

– scans from files: /proc/interrupts

/proc/dma

/proc/ioports

Last updated: 06 December 2004

25

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Hardware Management Chapter 2 - 25

Querying your PCI bus Querying your PCI bus

• /sbin/lspci

– reads the /proc/bus/pci interface – PCI ID database (e.g. /usr/share/pci.ids) is used to translate

PCI vendor and device codes to readable strings.

options -n : shows PCI vendor and device codes as numbers -t : shows a tree-like diagram -m : Dump PCI devices in readable form. -v : verbose mode.

Last updated: 06 December 2004

26

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Hardware Management Chapter 2 - 26

Querying your PCI bus Querying your PCI bus

• See the interconnection between the buses, bridges

and connections

• lspci -vt

-[00]-+-00.0 Intel Corp. 82845G/GL [Brookdale-G] Chipset Host Bridge

+-02.0 Intel Corp. 82845G/GL [Brookdale-G] Chipset Integr ...

+-1d.0 Intel Corp. 82801DB USB (Hub #1)

+-1d.1 Intel Corp. 82801DB USB (Hub #2)

+-1d.2 Intel Corp. 82801DB USB (Hub #3)

+-1d.7 Intel Corp. 82801DB USB EHCI Controller

+-1e.0-[01]----01.0 Realtek Semiconductor Co., Ltd. RTL-8139..

+-1f.0 Intel Corp. 82801DB ISA Bridge (LPC)

+-1f.1 Intel Corp. 82801DB ICH4 IDE

+-1f.3 Intel Corp. 82801DB SMBus

\-1f.5 Intel Corp. 82801DB AC'97 Audio

Last updated: 06 December 2004

27

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Hardware Management Chapter 2 - 27

/proc and /proc

and procinfo procinfo

• /proc

– dierectory, carry many information about a system

# ls -l /proc

-r--r--r--

1 root root 0 Mar 10 17:24 apm

dr-xr-xr-x 4 root root 0 Mar 10 15:07 bus

-r--r--r--

1 root root 0 Mar 10 17:24 cmdline

-r--r--r--

1 root root 0 Mar 10 17:24 cpuinfo

-r--r--r--

1 root root 0 Mar 10 17:24 devices

-r--r--r--

1 root root 0 Mar 10 17:24 filesystems

-r--r--r--

1 root root 0 Mar 10 17:24 interrupts

-r--r--r--

1 root root 0 Mar 10 17:24 iomem

-r--r--r--

1 root root 0 Mar 10 17:24 meminfo

......

Last updated: 06 December 2004

28

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Hardware Management Chapter 2 - 28

/proc and /proc

and procinfo procinfo

• procinfo

– report the system status, gather information from /proc

options -f : run procinfo continuously full-screen -nN: pause N second between updates -m : shows info about modules -a : shows all -d or -D : shows memory in total -S : same -d or -D, always show values by secand

Last updated: 06 December 2004

29

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Hardware Management Chapter 2 - 29

PCMCIA Utilities PCMCIA Utilities

• cardmgr

– monitoring PCMCIA sockets

options

-q : quiet mode -d : user modprobe to follow module dependencies -f : foreground -c configpath : orther configuration scripts (default /etc/pcmcia)

• configuration files

– Redhat: /etc/sysconfig/pcmcia – Debian: /etc/pcmcia.conf

Last updated: 06 December 2004

30

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Hardware Management Chapter 2 - 30

PCMCIA Utilities PCMCIA Utilities

• /var/lib/pcmcia/stab

– contains identification and device driver information

# cat /var/lib/pcmcia/stab

Socket 0: ATA/IDE Fixed Disk

0 ide ide-cs 0 hde 33 0

Socket dev.class driver Dev.#s Dev.Name Major Dev.# Major Dev.#

Last updated: 06 December 2004

31

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Hardware Management Chapter 2 - 31

PCMCIA Utilities PCMCIA Utilities

• cardctl

– monitors and controls the state of PCMCIA sockets

options:

status

config

ident

suspend | resume

Last updated: 06 December 2004

32

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Hardware Management Chapter 2 - 32

documentation documentation

• Serial - HOWTO

– http://www.tldp.org/HOWTO/Serial-HOWTO.html

• PCMCIA - HOWTO

– http://www.tldp.org/HOWTO/PCMCIA-HOWTO.html

Last updated: 06 December 2004

33

http://www.vanemery.com/Linux/Serial/serial-console.html

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Hardware Management Chapter 2 - 33

Summary Summary

• Adding New Hardware

• Software And Kernel Configuration

• Configuring PCMCIA Devices

Last updated: 06 December 2004

34

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Hardware Management Chapter 2 - 34

Question & Answer Question & Answer

Last updated: 06 December 2004

35

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Hardware Management Chapter 2 - 35

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Hardware Management Chapter 2 - 36

LPI 201 Intermediate Level Administration –– LPI 201 Intermediate Level Administration

Chapter 3 Chapter 3

Linux System Startup Linux System Startup

Last updated: 02 March 2004

1 © Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Linux System Startup Chapter 3 - 1

Objectives Objectives

• Describe the 4 stage of the boot sequence

• Understand kernel loading

• Understand hardware and deamon initialization

• Determine boot problems based upon LiLO errors

• Understand the significance of /boot/boot.### files

• Use mkinitrd to make custom RAM disk images

• Edit startup scripts and files to customize system

runlevels and boot processes

Last updated: 06 December 2004

2

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Linux System Startup Chapter 3 - 2

Boot stages Boot stages

• 1. BIOS Stage

– BIOS checks the system (POST - Power On Self Test)

• 2. Bootloader Stage

– The bootloader loads the second stage into memory

• 3. Kernel Stage

– The Kernel is loaded in to memory

• 4. Init Stage

– init load all services and user space tools and mounts all

from /etc/fstab

Last updated: 06 December 2004

3

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Linux System Startup Chapter 3 - 3

Stage 1: BIOS Stage 1: BIOS

• POST

– Check the system board

– Check memory

– Check system configuration

– Starts the video operation

– ...

• Check bootloader in MBR (Master Boot Record)

Last updated: 06 December 2004

4

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Linux System Startup Chapter 3 - 4

Stage 2: Bootloader Stage 2: Bootloader

• Lilo

– 1. Firmware loads the LILO bootsector and executes it.

– 2. LILO loads its map file

– 3. The User selects which kernel to boot

– 4. LILO loads the kernel

– 5. LILO executes the kernel

• Grub

– GRUB is more advanced bootloader than LILO

– 1. Firmware loads the GRUB bootsector and executes it.

– 2. The code loaded using BIOS, with knowledge filesystem

– 3. GRUB puts up a menu of defined boot options

Last updated: 06 December 2004

5

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Linux System Startup Chapter 3 - 5

Stage 3: Kernel Stage 3: Kernel

• The RAM disk

– load the initial ramdisk

– load the kernel into memory

• nash

– is a very simple script interpreter - not shell

– dedigned to run simple linuxrc scripts

• Kernel stages

– the kernel takes over for bootloader

– the kernel and the initial RAM disk image are already into

memory.

Last updated: 06 December 2004

6

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Linux System Startup Chapter 3 - 6

Stage 4: Init Stage 4: Init

• The final for the boot process is handled by /sbin/init

• Runlevel Scripts

– init call the rc script with an argument that tells it which is

the target runlevel.

Last updated: 06 December 2004

7

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Linux System Startup Chapter 3 - 7

Runlevels Runlevels

Runlevel Runlevel

rc.d Directory rc.d Directory

Description Description

/etc/rc.d/rc0.d

Shutdown

/etc/rc.d/rc1.d

Single user mode

/etc/rc.d/rc2.d

Multi-user mode (without NFS)

/etc/rc.d/rc3.d

Full multi-user mode

/etc/rc.d/rc4.d

unused

/etc/rc.d/rc5.d

Full multi-user and GUI mode

/etc/rc.d/rc6.d

Reboot

Last updated: 06 December 2004

8

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Linux System Startup Chapter 3 - 8

Troubleshooting LILO Troubleshooting LILO

• LILO Start Message

– : No part of LILO has been loaded – L : The first stage boot loader has been loaded – LI : The first stage boot loader was able to load the second stage – LIL : The second stage boot loader has started, but it can't load

descriptor table.

– LIL? : boot loader has been loaded at an incorect address – LIL- : The descriptor table is corrupt – LILO : All parts of LILO have been successfully loaded

Last updated: 06 December 2004

9

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Linux System Startup Chapter 3 - 9

Troubleshooting LILO Troubleshooting LILO

• Map Installer Errors and Warning

– Fatal Errors

– Warning

• Disk Error Codes

• Hex-Error Codes

Last updated: 06 December 2004

10

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Linux System Startup Chapter 3 - 10

Restoring the MBR with LILO files Restoring the MBR with LILO files

• LILO creates backup copies named:

/boot/boot.

– 0300:/de/hda (primary IDE)

– 0800:/dev/sda

– ....

• Use dd command

dd if=/boot/boot.0300 of=/dev/hda bs=466 count=1

Last updated: 06 December 2004

11

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Linux System Startup Chapter 3 - 11

Removing LILO Removing LILO

• via DOS

FDISK /MBR

Last updated: 06 December 2004

12

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Linux System Startup Chapter 3 - 12

inittab Fields inittab Fields

– Example: 15: 5: wait: /etc/rc.d/rc 5

1 2 3 4

• 1. unique identifier for the line

• 2. which runlevels activate the line

• 3. How to do when it parses the last field

– once: when intering the runlevel – wait: like once, but it wait until the program has finished before

continuing

– respawn: respawn the program when it is no longer running – initdefault: specifies the system default runlevel • 4. The command line to be executed by init.

Last updated: 06 December 2004

13

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Linux System Startup Chapter 3 - 13

Initial RAM Disks Initial RAM Disks

• Need to load driver and mount filesystem before load

kernel

• Making an initrd Image

$ mkinitrd /boot/initrd-2.4.29-edu.img 2.4.29-edu

Last updated: 06 December 2004

14

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Linux System Startup Chapter 3 - 14

Cutomizing Startup Cutomizing Startup

• /etc/inittab

– The init process uses /etc/inittab as its control file :

• This is a text file, can be edited by the system admin • init must be told about changes to /etc/inittab by: # init q • One line per process to run at define run levels • Lines starting with ‘#’ are comments

– Processes run from /etc/inittab are daemons – init read /etc/inittab every time the run level is

changed

Last updated: 06 December 2004

15

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Linux System Startup Chapter 3 - 15

Cutomizing Startup Cutomizing Startup

• /etc/inittab

id:3:initdefault:

# System initialization.

si::sysinit:/etc/rc.d/rc.sysinit

l0:0:wait:/etc/rc.d/rc 0

l1:1:wait:/etc/rc.d/rc 1

l2:2:wait:/etc/rc.d/rc 2

l3:3:wait:/etc/rc.d/rc 3

l4:4:wait:/etc/rc.d/rc 4

l5:5:wait:/etc/rc.d/rc 5

l6:6:wait:/etc/rc.d/rc 6

......

Last updated: 06 December 2004

16

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Linux System Startup Chapter 3 - 16

Run Command Scripts Run Command Scripts

(cid:131) These startup programs are symbolic links to the actual scripts

in the /etc/init.d/

(cid:131) The startup link names are formatted: firstcharacter is S

(started) or K (killed or stopped), the next two digitsidentify the orderthat scripts are executed by rc program

Last updated: 06 December 2004

17

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Linux System Startup Chapter 3 - 17

Changing Run Levels Changing Run Levels

(cid:131) Use init command to change between run levels. Syntax :

init

telinit

(cid:131) Examples :

# init 0

Shutdown system

# init 6

Reboot system

# init 3

Change to runlevel 3, multi-user text mode

# init 5

Change to runlevel 5, multi-user X11 mode

Last updated: 06 December 2004

18

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Linux System Startup Chapter 3 - 18

Changing Run Levels Changing Run Levels

(cid:131) Use init command to change between run levels. Syntax :

init

telinit

(cid:131) Examples :

# init 0

Shutdown system

# init 6

Reboot system

# init 3

Change to runlevel 3, multi-user text mode

# init 5

Change to runlevel 5, multi-user X11 mode

Last updated: 06 December 2004

19

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Linux System Startup Chapter 3 - 19

Customs Boot Script Customs Boot Script

case "$1" in

start)

;;

stop)

;;

restart)

;;

*)

echo $"Usage: $0 {start|stop|restart}"

exit 1

esac

exit $?

Last updated: 06 December 2004

20

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Linux System Startup Chapter 3 - 20

Deploys Boot Script Deploys Boot Script

• 1. Copy script to /etc/init.d/

– # cp myscript /etc/init.d/

• 2. Make soft link from rc#.d

– by hand

# cd rc3.d

# ln -s /etc/init.d/myscript S99myscript

– by command

# chkconfig --add myscript

Last updated: 06 December 2004

21

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Linux System Startup Chapter 3 - 21

Deploys Boot Script Deploys Boot Script

• chkconfig [option] [name]

--add : add soft link to rc#.d

--del : delete soft link from rc#.d

--list : list all deamon

• List all Deamon

# chkconfig --list

Last updated: 06 December 2004

22

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Linux System Startup Chapter 3 - 22

Summary Summary

• Describe the 4 stage of the boot sequence

• Understand kernel loading

• Understand hardware and deamon initialization

• Determine boot problems based upon LiLO errors

• Understand the significance of /boot/boot.### files

• Use mkinitrd to make custom RAM disk images

• Edit startup scripts and files to customize system

runlevels and boot processes

Last updated: 06 December 2004

23

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Linux System Startup Chapter 3 - 23

Question & Answer Question & Answer

Last updated: 06 December 2004

24

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Linux System Startup Chapter 3 - 24

LPI 201 Intermediate Level Administration –– LPI 201 Intermediate Level Administration

Chapter 4 Chapter 4

Maintaining and Configuring a Linux Maintaining and Configuring a Linux file system file system

Last updated: 02 March 2004

1 © Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Maintaining and Configuring a Linux file system Chapter 4 - 1

Objectives Objectives

• Understand different types of filesystems

• Addnew swap to system.

• Use file system related utilities such as fsck, mke2fs,

tune2fs, and dumpe2fs

• Configuring mount on demand

Last updated: 06 December 2004

2

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Maintaining and Configuring a Linux file system Chapter 4 - 2

Overview Overview

• Linux supports a number of local file systems,

including: – minix

– ext2

– ext3

– ReiserFS

– XFS

– JFS

– ....

• Each file system type has their own strengths and

weaknesses

• Linux also supports exported file systems

Last updated: 06 December 2004

3

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Maintaining and Configuring a Linux file system Chapter 4 - 3

MinixMinix

• First file system supported by Linux

• Used in floppies and initrd

• Not for use on modern Linux hard drives because of

its small capacity, short filename lengths, etc.

Last updated: 06 December 2004

4

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Maintaining and Configuring a Linux file system Chapter 4 - 4

ext2ext2

• One of the oldest and probably the most widespread

Linux file system

• Simple, reasonably fast, reasonably robust

• No journaling capability

Last updated: 06 December 2004

5

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Maintaining and Configuring a Linux file system Chapter 4 - 5

ext3ext3

• Same as ext2, but has journaling

• Journalling filesystem

– reduced time spent recovering a filesystem after a crash

– and is therefore in high demand in environments

– allow a crashed machine's filesystem to be recovered on

another machine, on a cluster of nodes with a shared disk.

Last updated: 06 December 2004

6

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Maintaining and Configuring a Linux file system Chapter 4 - 6

ReiserFS ReiserFS

• First journaling file system supported by Linux

• Uses b-trees and other indexing algorithms for for fast

operations

Last updated: 06 December 2004

7

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Maintaining and Configuring a Linux file system Chapter 4 - 7

XFSXFS

• Ported by SGI (Silicon graphics, Inc.) from their IRIX

version of Unix

• Supports journaling, multi-processor systems, and

massive scaling

Last updated: 06 December 2004

8

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Maintaining and Configuring a Linux file system Chapter 4 - 8

JFSJFS

• Journaling file system ported by IBM from OS/2 and

AIX

• Supports online resizing

Last updated: 06 December 2004

9

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Maintaining and Configuring a Linux file system Chapter 4 - 9

Compare Compare

Last updated: 06 December 2004

10

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Maintaining and Configuring a Linux file system Chapter 4 - 10

Swap file and Swap partition Swap file and Swap partition

• Swap file

– Temporary swap space for system.

– Use files on filesystem.

– Can mount on /etc/fstab

• Swap partition

– Permanent swap space for system

– Use dedicate partition

Last updated: 06 December 2004

11

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Maintaining and Configuring a Linux file system Chapter 4 - 11

Create swap file Create swap file

• Create new raw file for swap

# dd if=/dev/zero of=/tmp/SWAPFILE bs=1k count=10240

• Format raw file for swap filesystem type

# mkswap /tmp/SWAPFILE

• Register swap file to system

# swapon tmp/SWAPFILE

Last updated: 06 December 2004

12

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Maintaining and Configuring a Linux file system Chapter 4 - 12

Create swap file Create swap file

• Check it available

# cat /proc/swaps

• Unregister swap file

# swapoff tmp/SWAPFILE

• Add to /etc/fstab

/tmp/SWAPFILE swap swap defaults 0 0

Last updated: 06 December 2004

13

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Maintaining and Configuring a Linux file system Chapter 4 - 13

Create swap partition Create swap partition

• Create new partition ex: /dev/hda16

• Format partition for swap filesystem type

# mkswap /dev/hda16

• Add to /etc/fstab

/dev/hda6 swap swap defaults 0 0

• Make all swap available

# swapon -a

Last updated: 06 December 2004

14

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Maintaining and Configuring a Linux file system Chapter 4 - 14

File System Utilities File System Utilities

• fsck - check and repair a Linux file system

options:

-b use alternative superblck

-c check for bad blocks

-f force checking even when partition is marked clean

-p automatic repair

-y answer yes to all question

Last updated: 06 December 2004

15

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Maintaining and Configuring a Linux file system Chapter 4 - 15

File System Utilities File System Utilities

• sync - flush filesystem buffers

– updates modified superblocks and inodes

– executes delayed writes

Last updated: 06 December 2004

16

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Maintaining and Configuring a Linux file system Chapter 4 - 16

File System Utilities File System Utilities

• badblocks - search a device for bad blocks

– options:

• -b block size

• -c number of blocks tested at a time

• -i file with a list of known bad blocks, these blocks will be skipped

• -o output file, passed to mkfs

Last updated: 06 December 2004

17

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Maintaining and Configuring a Linux file system Chapter 4 - 17

File System Utilities File System Utilities

• mke2fs - create an ext2/3 filesyste

– options:

• -b blocksize

• -i number of bytes between consecutive inodes 'bytes-per-inode'

• -N number of inodes

• -m Percentage of blocks reserved for user root

• -c Check for bad blocks

• -l Read bad blocks from file

• -L Set a volume LABEL

• -j/-J Create journal (ext3)

• -T Optimise filesystem “type”, values are:

– news : one inode per 4kb block

– largefile : one inode per megabyte

– largefile4 : one inode per 4 megabytes

Last updated: 06 December 2004

18

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Maintaining and Configuring a Linux file system Chapter 4 - 18

File System Utilities File System Utilities

• dumpe2fs - dump filesystem information

– dumpe2fs prints the super block and blocks group information for the filesystem present on a device

• debugfs - ext2 file system debugger

– options:

• -w open the filesystem as writeable

• -b blocksize

Last updated: 06 December 2004

19

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Maintaining and Configuring a Linux file system Chapter 4 - 19

File System Utilities File System Utilities

• tune2fs - adjust tunable filesystem parameters on

second extended filesystems – options:

• -l read the superblock

• -L set the device's volume LABEL

• -m change the filesystems reserved blocks for user root

• -j or -J set a journal

Last updated: 06 December 2004

20

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Maintaining and Configuring a Linux file system Chapter 4 - 20

Configuring automount Configuring automount

• Mounting can be automated using a automount or

autofs. – /usr/sbin/automount

– /etc/init.d/autofs

• Reads the mapfile for configuration.

– /etc/auto.master

Last updated: 06 December 2004

21

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Maintaining and Configuring a Linux file system Chapter 4 - 21

Configuring automount Configuring automount

• Mounting can be automated using a automount or

autofs. – /usr/sbin/automount

– /etc/init.d/autofs

• Reads the mapfile for configuration.

# cat /etc/auto.master

/extra

/etc/auto.extra

/home

/etc/auto.home

Last updated: 06 December 2004

22

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Maintaining and Configuring a Linux file system Chapter 4 - 22

Configuring automount Configuring automount

/home

/extra

automount

• mount point directory is created by autofs

/extra ├ nfs ├ cdrom

Last updated: 06 December 2004

23

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Maintaining and Configuring a Linux file system Chapter 4 - 23

Summary Summary

• Understand different types of filesystems

• Addnew swap to system.

• Use file system related utilities such as fsck, mke2fs,

tune2fs, and dumpe2fs

• Configuring mount on demand

Last updated: 06 December 2004

24

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Maintaining and Configuring a Linux file system Chapter 4 - 24

Question & Answer Question & Answer

Last updated: 06 December 2004

25

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Maintaining and Configuring a Linux file system Chapter 4 - 25

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Maintaining and Configuring a Linux file system Chapter 4 - 26

LPI 201 Intermediate Level Administration –– LPI 201 Intermediate Level Administration

Chapter 5 Chapter 5

RAID and LVM RAID and LVM

Last updated: 02 March 2004

1 © Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –RAID and LVM Chapter 5 - 1

Objectives Objectives

• Understand the purposes of RAID and LVM • Describe the RAID 0, 1, 4, and 5 • Hardware and Software RAID • Requisite packages and kernel patches/modules for

RAID and LVM

• Configure software RAID on Linux • Understand LVM terminology • Concepts of resizing and “online resizing”

Last updated: 06 December 2004

2

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –RAID and LVM Chapter 5 - 2

RAID Levels RAID Levels

• Two types of RAID:

– Tripping: slits data to multi physical device

– Mirroring: An extra copy of data to a different physical

device

– Parity: Uses XOR (exclusive OR) operation on two bits

generates a third such that, given the loss of any one bit the other two can be used to reconstruct the missing bit

Last updated: 06 December 2004

3

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –RAID and LVM Chapter 5 - 3

RAID 0 : striping

• Provides no redundancy.

Last updated: 06 December 2004

4

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –RAID and LVM Chapter 5 - 4

RAID 1 : mirroring

• Provides redundancy only.

Last updated: 06 December 2004

5

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –RAID and LVM Chapter 5 - 5

RAID 1+0 : striping then mirroring

• Hybrid of RAID 1 and RAID 0

Last updated: 06 December 2004

6

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –RAID and LVM Chapter 5 - 6

RAID 2, 3, and 4

• RAID 2 and 3 seldom used.

• RAID 4 uses dedicated XOR parity.

Last updated: 06 December 2004

7

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –RAID and LVM Chapter 5 - 7

distributed parity RAID 5 : distributed parity

• Most popular form of RAID.

• Provides aggregation and redundancy.

Last updated: 06 December 2004

8

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –RAID and LVM Chapter 5 - 8

(3 disks) XOR parity (3 disks) XOR parity

Last updated: 06 December 2004

9

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –RAID and LVM Chapter 5 - 9

(4 disks) XOR parity (4 disks) XOR parity

Last updated: 06 December 2004

10

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –RAID and LVM Chapter 5 - 10

Software vs. Hardware RAID Software vs. Hardware RAID

• Hardware RAID

– The hardware-based system manages

– Independently from the host, presents to the host only a

single disk per RAID array.

– Moves all RAID handling "intelligence" into a controller

– RAID controllers also come in the form of cards that act like

a SCSI controller

– Can hot-swap support

Last updated: 06 December 2004

11

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –RAID and LVM Chapter 5 - 11

Software vs. Hardware RAID Software vs. Hardware RAID

• Software RAID

– Implements the various RAID levels in the kernel disk

– It offers the cheapest possible solution

– Also works with cheaper IDE disks as well as SCSI disks

– The performance of a software-based array is dependent on

the server CPU performance and load.

Last updated: 06 December 2004

12

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –RAID and LVM Chapter 5 - 12

Designing RAID Sets

• The four major factors in designing a RAID set:

– partitioning

– performance

– redundancy/fault tolerance

– cost

Last updated: 06 December 2004

13

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –RAID and LVM Chapter 5 - 13

Limitations of RAID Limitations of RAID

• RAID protects data from hardware failures, but not:

– Accidental removal or other user or administrative error

– Software bugs causing corruption/removal of data

– Security incidents: deliberate malicious removal or

corruption

Last updated: 06 December 2004

14

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –RAID and LVM Chapter 5 - 14

Configure software RAID Configure software RAID

• Kernel modules

RAID0

raid0.o

RAID1

raid0.o

RAID4 or RAID5

raid5.o

• raidtools package

/sbin/lsraid : query raid devices /sbin/mkraid : create md devices from instructions given in

/etc/raidtab

/sbin/raidstart and raidstop: start and stop md device

Last updated: 06 December 2004

15

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –RAID and LVM Chapter 5 - 15

Configure software RAID Configure software RAID

• /etc/raidtab structure

raiddev

Meta disk devices name

raid-level

level of RAID

nr-raid-disks

number of raid devices in array

nr-spare-disks

number of spare devices in array

persistent-superblock 0/1

chunk-size

set the stripe size to sizes bytes

device

devices path, follow by raiddisk, sparedisk, paritydisk

raid-disk

disk in raid order

spare-disk

disk in spare order

parity-disk

disk in parity order

failed-disk

disk in fail order

Last updated: 06 December 2004

16

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –RAID and LVM Chapter 5 - 16

Configure RAID 1 Configure RAID 1

• 1./ Make two new partitions of same size

• 2./ Configure software RAID 1 on these partitions

– Example: /etc/raidtab

raiddev /dev/md0

raidlevel 1

nr-raid-disks 2

nr-spare-disks 0

chunk-size 4

persistent-superblock 1

device /dev/hda14

raid-disk 0

device /dev/hda15

raid-disk 1

Last updated: 06 December 2004

17

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –RAID and LVM Chapter 5 - 17

Configure RAID 1 Configure RAID 1

• 3./ Use raidtools

– create meta device

mkraid /dev/md0

– start meta device

raidstart /dev/md0

• 4./ Format device

mke2fs /dev/md0

• 5./ Mount device mkdir /mddev

mount /dev/md0 /mddev

Last updated: 06 December 2004

18

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –RAID and LVM Chapter 5 - 18

Logical Volume Management Logical Volume Management

• LVM provides a higher-level view of the disk storage

• Gives the system administrator much more flexibility

in allocating storage

• Can be resized and moved around almost at will.

• Allows management of storage volumes in user-

defined groups

• Use sensibly named volume groups – such as "development" and "sales"

– rather than physical disk names such as "sda" and "sdb"

Last updated: 06 December 2004

19

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –RAID and LVM Chapter 5 - 19

LVM Terminology LVM Terminology

• volume group (VG):

– This is a container of “physical volumes” (PVs) and of “logical volume”

(LVs).

• physical volume (PV):

– This is a drive, a partition or, under Linux, a RAID set, represented as a

/dev/md* device.

• logical volume (LV):

– This is a “partition” on which we can create a file system.

• physical extent (PE):

– The units in which space can be allocated from a volume group.

• logical extent (LE):

– The units in which space is allocated to a logical volume.

Last updated: 06 December 2004

20

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –RAID and LVM Chapter 5 - 20

Logical Volume Management Logical Volume Management

Volume Group (VG)

Logical Volume

Physical Extent (PE)

/dev/hda1

/dev/hdd1

/dev/hdb1

Physical Volume (PV)

Last updated: 06 December 2004

21

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –RAID and LVM Chapter 5 - 21

LVM Kernel Module and Tools LVM Kernel Module and Tools

• Kernel module – lvm-mod.o • lvm package

– pv tools: pvcreate, pvmove, pvdisplay, ...

– VG tools: vgscan, vgcreate, vgremove, vgdisplay, ...

– LV tools: lvcreate, lvextent, ...

• Config files (created by vgscan)

– /etc/lvmtab

– /etc/lvmtab.d

Last updated: 06 December 2004

22

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –RAID and LVM Chapter 5 - 22

Demo: Create LVM Demo: Create LVM

Create LVM on three partition:

1. Run vgscan to create the /etc/lvmtab 2. Partition three volumes, mark the LVM partitions (PVs) as

type 8e (Linux LVM) 3. Create physical volume – pvcreate /dev/hda1 – pvcreate /dev/hda2 – pvcreate /dev/hda3

4. Create volume group, name volumeA

– vgcreate volumeA /dev/hda1 /dev/hda2 /dev/hda3 This command will create directory /dev/volumeA/ The Default PE size is 4Mb

Last updated: 06 December 2004

23

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –RAID and LVM Chapter 5 - 23

Demo: Create LVM Demo: Create LVM

Create LVM on three partition:

5. Create logical volume

– lvcreate -L 150M -n lv0 volumeA This command create block device /dev/volumeA/lv0

6. Make a filesystem on /dev/volumeA/lv0

– mkfs -t ext3 /dev/volumeA/lv0

– mkdir /lvm

– mount /dev/volumeA/lv0 /lvm

Last updated: 06 December 2004

24

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –RAID and LVM Chapter 5 - 24

Demo: Create LVM Demo: Create LVM

• Change LV size

e2fsadm -L 25 /dev/volumeA/lv0

-L : size in megabytes

-l : size in logical extents (lv), default 1lv=4Mb

Last updated: 06 December 2004

25

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –RAID and LVM Chapter 5 - 25

Summary Summary

• Understand the purposes of RAID and LVM

• Describe the RAID 0, 1, 4, and 5

• Hardware and Software RAID

• Requisite packages and kernel patches/modules for

RAID and LVM

• Configure software RAID on Linux

• Understand LVM terminology

• Concepts of resizing and “online resizing”

Last updated: 06 December 2004

26

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –RAID and LVM Chapter 5 - 26

Question & Answer Question & Answer

Last updated: 06 December 2004

27

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –RAID and LVM Chapter 5 - 27

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –RAID and LVM Chapter 5 - 28

LPI 201 Intermediate Level Administration –– LPI 201 Intermediate Level Administration

Chapter 66 Chapter

File Sharing and Services File Sharing and Services NFSNFS

Last updated: 06 December 2004

1 © Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – File Sharing and Services - NFS Chapter 6 - 1

Objectives Objectives

• Introduction

• Setting up an NFS Server

• Setting up an NFS Client

• Security and NFS

• Troubleshooting

Last updated: 06 December 2004

2

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – File Sharing and Services - NFS Chapter 6 - 2

Introduction Introduction

Last updated: 06 December 2004

3

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – File Sharing and Services - NFS Chapter 6 - 3

Setting up an NFS Server Setting up an NFS Server

• Introduction to the server setup

• Setting up the Configuration Files

• Getting the services started

• Verifying that NFS is running

• Making changes to /etc/exports later on

Last updated: 06 December 2004

4

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – File Sharing and Services - NFS Chapter 6 - 4

Introduction to the server setup Introduction to the server setup

• It is assumed that you will be setting up both a server

and a client.

• Setting up the server will be done in two steps:

– Setting up the configuration files for NFS,

– and then starting the NFS services.

Last updated: 06 December 2004

5

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – File Sharing and Services - NFS Chapter 6 - 5

Setting up the Configuration Files Setting up the Configuration Files

• There are three main configuration files you will need

to edit to set up an NFS server: – /etc/exports

– /etc/hosts.allow

– /etc/hosts.deny

• Edit /etc/exports to get NFS to work

Last updated: 06 December 2004

6

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – File Sharing and Services - NFS Chapter 6 - 6

Setting up the Configuration Files Setting up the Configuration Files

/etc/exports

• This file contains a list of entries; • each entry indicates a volume that is shared and how it is

shared.

directory machine1(option11,option12) machine2(option21,option22)

Where: directory •

the directory that you want to share.

machine1 and machine2

•

client machines that will have access to the directory.

Last updated: 06 December 2004

7

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – File Sharing and Services - NFS Chapter 6 - 7

Setting up the Configuration Files Setting up the Configuration Files

/etc/exports

• optionxx

Default, the directory is shared read only; the client machine will not be able to write to it.

The client machine will have read and write access to the directory.

no_root_squash

By default, any file request made by user root on the client machine is treated as if it is made by user nobody on the server. If no_root_squash is selected, then root on the client machine will have the same level of access to the files on the system as root on the server.

no_subtree_check

If only part of a volume is exported, a routine called subtree checking verifies that a file that is requested from the client is in the appropriate part of the volume. If the entire volume is exported, disabling this check will speed up transfers.

sync

By default, the exportfs command will use async behavior, telling a client machine that a file write is complete when NFS has finished handing the write over to the filesystem. This behavior may cause data corruption if the server reboots, and the sync option prevents this.

Last updated: 06 December 2004

8

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – File Sharing and Services - NFS Chapter 6 - 8

Setting up the Configuration Files Setting up the Configuration Files

/etc/hosts.allow /etc/hosts.deny

service: host [or network/netmask] , host [or network/netmask]

• These two files specify which computers on the network can use services on

your machine.

• Each line of the file contains a single entry listing a service and a set of

machines.

– It first checks hosts.allow

if the machine matches a description listed in there; then the machine is allowed access.

• •

– If the machine does not match an entry in hosts.allow:

•

the server then checks hosts.deny; If it does then the machine is denied access.

•

– If the client matches no listings in either file, then it is allowed access.

Last updated: 06 December 2004

9

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – File Sharing and Services - NFS Chapter 6 - 9

Setting up the Configuration Files Setting up the Configuration Files

• Sample:

– /etc/exports

/usr/local 192.168.0.1(ro) 192.168.0.2(ro) /home 192.168.0.1(rw) 192.168.0.2(rw)

– /etc/hosts.deny

/etc/hosts.allow

portmap: 192.168.0.1 , 192.168.0.2 lockd: 192.168.0.1 , 192.168.0.2 rquotad: 192.168.0.1 , 192.168.0.2 mountd: 192.168.0.1 , 192.168.0.2 statd: 192.168.0.1 , 192.168.0.2

portmap:ALL lockd:ALL mountd:ALL rquotad:ALL statd:ALL

Last updated: 06 December 2004

10

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – File Sharing and Services - NFS Chapter 6 - 10

Getting the services started Getting the services started

• Starting the Portmapper

– NFS depends on the portmapper daemon, either called

portmap or rpc.portmap. – It will need to be started first.

# /etc/rc.d/init.d/portmap {start|stop|status|restart|reload} # /etc/rc.d/init.d/portmap {start|stop|status|restart|reload}

Last updated: 06 December 2004

11

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – File Sharing and Services - NFS Chapter 6 - 11

Getting the services started Getting the services started

• The Daemons

– NFS serving is taken care of by five daemons:

• rpc.nfsd, which does most of the work; • rpc.lockd and rpc.statd, which handle file locking; Starting

with 2.2.18, lockd is called by nfsd upon demand • rpc.mountd, which handles the initial mount requests • rpc.rquotad, which handles user file quotas on exported

volumes.

# /etc/rc.d/init.d/nfs {start|stop|status|restart|reload} # /etc/rc.d/init.d/nfs {start|stop|status|restart|reload}

Last updated: 06 December 2004

12

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – File Sharing and Services - NFS Chapter 6 - 12

Verifying that NFS is running Verifying that NFS is running

• rpcinfo –p: Query the portmapper.

program vers proto port

111 portmapper 111 portmapper 749 rquotad 749 rquotad 759 mountd 761 mountd 764 mountd 766 mountd 769 mountd 771 mountd

100000 2 tcp 100000 2 udp 100011 1 udp 100011 2 udp 100005 1 udp 100005 1 tcp 100005 2 udp 100005 2 tcp 100005 3 udp 100005 3 tcp 100003 2 udp 100003 3 udp

2049 nfs 2049 nfs

......

Last updated: 06 December 2004

13

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – File Sharing and Services - NFS Chapter 6 - 13

later on /etc/exports later on

Making changes to /etc/exports Making changes to

• run the command exportfs -ra to force nfsd to re-

read the /etc/exports file.

• can use kill -HUP nfsd for same purpose.

Last updated: 06 December 2004

14

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – File Sharing and Services - NFS Chapter 6 - 14

Setting up an NFS Client Setting up an NFS Client

• Mounting remote directories

• Getting NFS File Systems to Be Mounted at Boot Time

Last updated: 06 December 2004

15

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – File Sharing and Services - NFS Chapter 6 - 15

Mounting remote directories Mounting remote directories

• /proc/filesystems and making sure there is a line

containing nfs.

• Typing insmod nfs if NFS has been compiled as a

module;

• Otherwise, you will need to re-build a kernel.

• Example: mount nfs

# mount -t nfs 192.168.90.100:/home /mnt/home # mount -t nfs 192.168.90.100:/home /mnt/home

• Example: umount

# umount /mnt/home # umount /mnt/home

Last updated: 06 December 2004

16

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – File Sharing and Services - NFS Chapter 6 - 16

Getting NFS to Be Mounted at Boot Time Getting NFS to Be Mounted at Boot Time

• NFS file systems can be added to your /etc/fstab

• Sample: /etc/fstab

0 0 0 0

nfs nfs

rw rw

# device mountpoint fs-type options dump fsckorder # device mountpoint fs-type options dump fsckorder ... ... 192.168.90.100:/home /mnt 192.168.90.100:/home /mnt ... ...

Last updated: 06 December 2004

17

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – File Sharing and Services - NFS Chapter 6 - 17

Security and NFS Security and NFS

• The portmapper

• Server security: nfsd and mountd

• Summary

Last updated: 06 December 2004

18

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – File Sharing and Services - NFS Chapter 6 - 18

portmapper The portmapper The

• The portmapper keeps a list of what services are

running on what ports.

• The easy way to check if your portmapper is good or

not is to run strings

# strings /sbin/portmap | grep hosts. # strings /sbin/portmap | grep hosts.

/etc/hosts.allow /etc/hosts.allow /etc/hosts.deny /etc/hosts.deny @(#) hosts_ctl.c 1.4 94/12/28 17:42:27 @(#) hosts_ctl.c 1.4 94/12/28 17:42:27 @(#) hosts_access.c 1.21 97/02/12 02:13:22 @(#) hosts_access.c 1.21 97/02/12 02:13:22

Last updated: 06 December 2004

19

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – File Sharing and Services - NFS Chapter 6 - 19

portmapper The portmapper The

• /etc/hosts.deny should contain the line

portmap: ALL portmap: ALL

• which will deny access to everyone.

rpcinfo -p rpcinfo -p

• /etc/hosts.allow

portmap: 192.168.0.0/255.255.255.0 portmap: 192.168.0.0/255.255.255.0

Last updated: 06 December 2004

20

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – File Sharing and Services - NFS Chapter 6 - 20

Server security: nfsdnfsd and Server security:

mountd and mountd

• we can decide that we don't want to trust any

requests made as root on the client.

• /etc/exports

/home slave1(rw,root_squash) /home slave1(rw,root_squash)

• On client, when user (IP=0) attempts to access nfs, the server assigns server's 'nobody' account to him.

• But, the root user on the client can still use su to

become any other user and access files!

Last updated: 06 December 2004

21

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – File Sharing and Services - NFS Chapter 6 - 21

Troubleshooting Troubleshooting

• 1. File requests hang or timeout waiting for access to

the file.

• This usually means that the client is unable to communicate with

the server.

• 2. reason given by server: Permission denied

• Check your /etc/exports file and make sure that the volume is exported and that your client has the right kind of access to it.

• Be sure to type exportfs -ra to be extra certain that the exports

are being re-read.

• Check the file /proc/fs/nfs/exports and make sure the

volume and client are listed correctly.

Last updated: 06 December 2004

22

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – File Sharing and Services - NFS Chapter 6 - 22

Troubleshooting Troubleshooting

• 3. RPC: Program Not Registered (or another

"RPC" error)

• This means that the client does not detect NFS running on the

server

• typing rpcinfo -p on the server

• type rpcinfo -p server where server is the DNS name or IP address

of your server.

Last updated: 06 December 2004

23

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – File Sharing and Services - NFS Chapter 6 - 23

Troubleshooting Troubleshooting

• 4. Do not have permission to access files on the

mounted volume.

• check the export options on the server by looking at

/proc/fs/nfs/exports and make sure the filesystem is not exported read-only

• check /proc/mounts and make sure the volume is mounted

read/write

• If you are not root, Type id [user] on both the client and the

server and make sure they give the same UID number

• If you are root, then you are probably not exporting with the

no_root_squash option

Last updated: 06 December 2004

24

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – File Sharing and Services - NFS Chapter 6 - 24

Summary Summary

• Introduction

• Setting up an NFS Server

• Setting up an NFS Client

• Security and NFS

• Troubleshooting

Last updated: 06 December 2004

25

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – File Sharing and Services - NFS Chapter 6 - 25

Question & Answer Question & Answer

Last updated: 06 December 2004

26

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – File Sharing and Services - NFS Chapter 6 - 26

LPI 201 Intermediate Level Administration –– LPI 201 Intermediate Level Administration

Chapter 7 Chapter 7

File Sharing and Services File Sharing and Services SAMBA SAMBA

Last updated: 06 December 2004

1 © Spring 2004, The Saigon Center of Techniques and Technology – – - 1

Objectives Objectives

• Mount and unmount SMB file systems

• Control smbd’s file locking behavior

• Define how smb.conf predefined [homes] share can be used to provide home directory service to users

• Set up a nmbd WINS server

• Change the workgroup in which a server participates

• Define a shared directory in smb.conf

Last updated: 06 December 2004

2

_________________________________________________________________________

© Spring 2004, The Saigon Center of Techniques and Technology – – - 2

Objectives Objectives

• Configure basic print services under Samba

• Provide client support for manipulating print queues

• Create a login script for Samba clients

• Use nmblookup to test WINS server functionality

• Use the smbmount command to mount an SMB share

on a Linux client

Last updated: 06 December 2004

3

_________________________________________________________________________

© Spring 2004, The Saigon Center of Techniques and Technology – – - 3

Common Locations

• /usr/local/samba/...

– bin/

• Samba binaries and associated shell scripts

– lib/

• configurations files and code pages

– var/

• log files for smbd and nmbd

– var/locks/

• browse.dat, wins.dat, pid files for smbd and nmbd

Last updated: 06 December 2004

4

_________________________________________________________________________

© Spring 2004, The Saigon Center of Techniques and Technology – – - 4

Common Locations (Con’t)

• /usr/local/samba/...

– private/

• MACHINE.SID, smbpasswd file

– swat/

• HTML and image files for SWAT

– man/

• man pages

Last updated: 06 December 2004

5

_________________________________________________________________________

© Spring 2004, The Saigon Center of Techniques and Technology – – - 5

Sharing Directories

• Simple file shares can be created with a few

parameter settings: – Path: the actual directory of disk to share

– read only: determine whether or not smbd will allow write

access to the share

• Users are assigned the most restrictive access to files

and directories – If a file share assigns write access to a user, but the

permissions at the file system level do not allow it, the user is denied write access

Last updated: 06 December 2004

6

_________________________________________________________________________

© Spring 2004, The Saigon Center of Techniques and Technology – – - 6

Restricting Access

• hosts allow / hosts deny:

– akin to the tcp_wrappers /etc/host.allow and

/etc/hosts.deny

– e.g. hosts allow = 192.168.1.

• valid users / invalid users:

– list of users which will be allowed (denied) access to the

current service

• max connections:

– integer value specifying the maximum number of concurrent

client connections

Last updated: 06 December 2004

7

_________________________________________________________________________

© Spring 2004, The Saigon Center of Techniques and Technology – – - 7

[homes]

• One of the three built-in smb.conf sections.

• Used to provide a fast and easy method of sharing

user home directories.

• When smbd receives a SMBtconX request

– it attempts to locate an explictly defined share: in smb.conf

that matches the name

– if [homes] is defined, smbd assumes the resource named is a username and searches the local system password file

– if a match is located, smbd copies the [homes] services and

renames it to the located username

Last updated: 06 December 2004

8

_________________________________________________________________________

© Spring 2004, The Saigon Center of Techniques and Technology – – - 8

Enabeling smbfs Support

• Not officially part of Samba in past times.

• Andrew Tridgell has recently become the maintainer

of smbmount utilities.

• Requires kernel support: – compiled into the kernel

– loaded as a module

Last updated: 06 December 2004

9

_________________________________________________________________________

© Spring 2004, The Saigon Center of Techniques and Technology – – - 9

Samba’s Printing Styles

• Samba currently supports 8 different printing styles

out of the box.

• Samba attempts to determine the host's native

printing style during the configure script.

• The style can be explicitly defined using the "printing"

parameter

• The printing style is used:

– to configure default values for the various print commands

– to parse the output from the "lpq command"

Last updated: 06 December 2004

10

_________________________________________________________________________

© Spring 2004, The Saigon Center of Techniques and Technology – – - 10

[printers] [printers]

• Analagous to [homes].

• Uses the "printcap name" value instead of /etc/passwd.

• load printers:

– should all the printer names from the printcap file be autoloaded for

displaying the server's list of shares?

• Service location steps:

– search for an explictly defined share

– if [homes] exists, search using the algorithm previously described

– if [printers] exists, search the printcap file for a match

– if a match is located, generate a printer service using the requested

name

Last updated: 06 December 2004

11

_________________________________________________________________________

© Spring 2004, The Saigon Center of Techniques and Technology – – - 11

Network Browsing and WINS Network Browsing and WINS

• Browse list:

– a list of servers within a group collected from server

announcement initiated by hosts

• Workgroups and domains are functionally equivalent

with respect to browsing.

• A server is defined as a host that is able to offer

shares via SMB.

• Master browser:

– a host that maintains a browse list for its workgroup

• Backup master browser:

– maintains a copy of the browse list from the master browser

Last updated: 06 December 2004

12

_________________________________________________________________________

© Spring 2004, The Saigon Center of Techniques and Technology – – - 12

Browser Elections Browser Elections

• Used to support servers coming on and off the

network.

• Clients registering the group name

#0x1e are able to participate in browse master elections.

Last updated: 06 December 2004

13

_________________________________________________________________________

© Spring 2004, The Saigon Center of Techniques and Technology – – - 13

LMHOSTS File LMHOSTS File

• SMB equivalent of /etc/hosts

• Samba's lmhosts format

– IP Name[#resource_byte]

– e.g. 192.168.1.73 TUMNUS#20

• n Differs from the format used by Windows clients

Last updated: 06 December 2004

14

_________________________________________________________________________

© Spring 2004, The Saigon Center of Techniques and Technology – – - 14

Windows Internet Name Service (WINS) Windows Internet Name Service (WINS)

• WINS is Microsoft's implementation of an RFC 1001/1002 compliant NetBIOS Name Server.

• Name registration and resolution requests are unicast

from clients directly to the WINS server: – able cross subnet boundaries

– NetBIOS name space is still flat

Last updated: 06 December 2004

15

_________________________________________________________________________

© Spring 2004, The Saigon Center of Techniques and Technology – – - 15

Windows 95 and 98 Windows 95 and 98

• Windows 95 and 98 are fundamentally the same OS.

• In order to access Samba servers, these clients

require: – Client for Microsoft Networks

– Some type of network adapter

– TCP/IP

Last updated: 06 December 2004

16

_________________________________________________________________________

© Spring 2004, The Saigon Center of Techniques and Technology – – - 16

Windows NT 4.0

• To access a Samba host, clients require

– Workstation service

– some type of network adapter

– TCP/IP

• Connecting to file shares and printers is similar to

Windows 9x with the exception that: – NT does allow for connecting as a user other than the one

currently logged in as: • e.g. net use x: \\server\share /user:username

– Samba does not support the automatic downloading of print

drivers to NT clients

Last updated: 06 December 2004

17

_________________________________________________________________________

© Spring 2004, The Saigon Center of Techniques and Technology – – - 17

Windows 2000 Windows 2000

• Is really NT 5.0

– not the next revision of Windows for Workgroups

• Samba will continue to function as a stable file and

print server for Windows 2000 clients – Top priority for the Samba developers

• SMB protocol has built in backwards compatibility

features

Last updated: 06 December 2004

18

_________________________________________________________________________

© Spring 2004, The Saigon Center of Techniques and Technology – – - 18

Summary Summary

• The simplest file share definition may only include a

path and a name, such as: [public]

path = /export/smb/public

• The smb.conf file is located in /etc/samba.

• The addition of the browseable parameter in smb.conf instructs smbd to include the share when displaying a list of available services.

Last updated: 06 December 2004

19

_________________________________________________________________________

© Spring 2004, The Saigon Center of Techniques and Technology – – - 19

Question & Answer Question & Answer

Last updated: 06 December 2004

20

_________________________________________________________________________

© Spring 2004, The Saigon Center of Techniques and Technology – – - 20

LPI 201 Intermediate Level Administration –– LPI 201 Intermediate Level Administration

Chapter 8 Chapter 8

Recovering a Linux System Recovering a Linux System

Last updated: 06 December 2004

1 © Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –Recovering a Linux System Chapter 08 - 1

Objectives Objectives

• Create an off-site backup storage plan

• Boot into different runlevels to performs system

maintenance and recovery

• Create a data CD using mkisofs and cdrecord

Last updated: 06 December 2004

2

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –Recovering a Linux System Chapter 08 - 2

Classifications : Data types DataData Classifications : Data types

• Ephemeral:

– data that is only present because of processes running at

the moment.

• Local:

– User specific: data or files that are individually created by

users on this system

– Service Specific: the custom data which makes a given

service useful

• General System:

– Overall file system and directory heirarchy.

Last updated: 06 December 2004

3

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –Recovering a Linux System Chapter 08 - 3

Full System and Data Backup Full System and Data Backup

• To restore from multiple tar files is a little involved:

– 1 . Boot with a rescue disc or diskette.

– 2 . Partition the drives with fdisk, sfdisk, or cfdisk.

– 3 . For each file system: mkfs.

– 4 . Mount the new rootfs up on /mnt or /mnt/target.

– 5 . Make your mountpoints thereon.

Last updated: 06 December 2004

4

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –Recovering a Linux System Chapter 08 - 4

Full System and Data Backup Full System and Data Backup

– 6 . Mount up all your other target file systems.

– 7 . cd into the top level of the target.

– 8 . Extract the tar file.

– 9 . Edit /etc/lilo.conf and /boot/grub/grub.conf (menu.lst).

– 10.chroot and run /sbin/lilo (or /sbin/grub-install).

– 11.Edit a few files and install a few other packages to adjust

the system to any hardware differences.

Last updated: 06 December 2004

5

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –Recovering a Linux System Chapter 08 - 5

Steps to create a data CD Steps to create a data CD

• Compile and install software pakage

• Create a ISO file system with mkisofs

• Test the .iso image by mounting it

• Write the data to the CD with cdrecord

Last updated: 06 December 2004

6

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –Recovering a Linux System Chapter 08 - 6

Compile and install software pakage Compile and install software pakage

• The package named: cdrtools • Locate at:

ftp://ftp.berlios.de/pub/cdrecord/

• Include commands:

mkisofs, isoinfo, cdrecord, ...

• Compile and install cdrtools to special location

• Add path to PATH variable

Last updated: 06 December 2004

7

ftp://ftp.berlios.de/pub/cdrecord/

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –Recovering a Linux System Chapter 08 - 7

Create a ISO file system with mkisofs Create a ISO file system with mkisofs

• Syntax:

mkisofs -o filename data_collection

• options:

– J : joliet filename format (64 chars) into iso9660 fs, like mkhydrid

command.

– r : the uid and gid on files and dirs is set to zero

– o filename : ISO file name, iso9660 format.

– b boot_image : special path and filename of boot image, be use

when making bootable CD

Last updated: 06 December 2004

8

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –Recovering a Linux System Chapter 08 - 8

Create a ISO file system with mkisofs Create a ISO file system with mkisofs

• Backup /etc directory to iso9660 format with long

filename.

mkisofs -J -r -o backup.iso /etc

• where:

– backup.iso : output iso file name (iso9660 format) – /etc : Data sources

Last updated: 06 December 2004

9

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –Recovering a Linux System Chapter 08 - 9

Test the .iso image Test the .iso image

• Linux has the ability to mount files as if they were disk

partitions.

• This feature is useful to check that the directory layout and file-access permissions of the CD image matches your wishes.

Last updated: 06 December 2004

10

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –Recovering a Linux System Chapter 08 - 10

Test the .iso image Test the .iso image

• Step test iso image 1. Create mount point

2. Mount iso image to mount point

3. Now, you can do something (read-only) on it.

4. unmount iso image

# mkdir /mnt/testiso # mount -t iso9660 backup.iso /mnt/testiso -o loop ....... # umount /mnt/testiso

Last updated: 06 December 2004

11

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –Recovering a Linux System Chapter 08 - 11

image to CD Write CD--image to CD Write CD

• Make sure CD-Writer availabe.

# cdrecord --scanbus

1,0,0 1,1,0 1,2,0 1,3,0 1,4,0 1,5,0 1,6,0 1,7,0

100) 'HL-DT-ST' 'CD-RW GCE-8160B ' '2.01' Removable CD-ROM 101) * 102) * 103) * 104) * 105) * 106) * 107) *

Last updated: 06 December 2004

12

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –Recovering a Linux System Chapter 08 - 12

image to CD Write CD--image to CD Write CD

• Write CD-image to CD

# cdrecord -v speed=8 dev=1,0,0 -data backup.iso

Last updated: 06 December 2004

13

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –Recovering a Linux System Chapter 08 - 13

Duplicate CD Duplicate CD

•

It is possible to make a 1:1 copy of a data CD

• But, can not duplicate Audio CD

• Copy direct CD to CD:

# cdrecord -v dev=1,0,0 speed=8 -isosize /dev/scd0

Last updated: 06 December 2004

14

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –Recovering a Linux System Chapter 08 - 14

Duplicate CD Duplicate CD

• Copy CD indirect: – Copy CD to image

# dd if=/dev/scd0 of=cdimage.img

– Write image to CD

# cdrecord -v dev=1,0,0 speed=8 -data cdimage.img

Last updated: 06 December 2004

15

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –Recovering a Linux System Chapter 08 - 15

Summary Summary

• Create an off-site backup storage plan

• Boot into different runlevels to performs system

maintenance and recovery

• Create a data CD using mkisofs and cdrecord

Last updated: 06 December 2004

16

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –Recovering a Linux System Chapter 08 - 16

Practice Practice

1. Compile and install cdrtools package to /opt/

17

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –Recovering a Linux System Chapter 08 - 17

Question & Answer Question & Answer

Last updated: 06 December 2004

18

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –Recovering a Linux System Chapter 08 - 18

LPI 201 Intermediate Level Administration –– LPI 201 Intermediate Level Administration

Chapter 9 Chapter 9

System Logging and Automation System Logging and Automation

Regular Expression Part 1: Regular Expression Part 1:

Last updated: 06 December 2004

1 © Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 1 - 1

Objectives Objectives

• Introducing Regular Expressions

• Primitives and Multipliers

• Anchors • Regular Expressions in sed • Regular Expressions in awk • Perl Regular Expressions

Last updated: 06 December 2004

2

Regular Expressions are, as we write further on in this chapter, the salt of all Unix systems. Regular Expression patterns are used in grep and family, Perl, awk and sed. In general, there are lots of similarities between the Regular Expression dialects used by each of these programs. When able to work with one dialect, it is quite easy to work with the other dialects too. There are, however, annoying and dangerous differences. In this section the commonalities will be described and the differences clarified. Unfortunately, portability is an issue: there are even different Regular Expression dialects for the same program on different operating systems. Linux Regular Expression support is less complicated, but even here differences exist. This point will gradually become clear in this section.

In keeping with common practice in Perl documentation, the term Perl will be used for the Perl language and perl will be used for the program.

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 1 - 2

Introducing Regular Expressions Introducing Regular Expressions

• A Regular Expression is a pattern.

• The pattern is matchedagainst some text.

• The text is frequently called the input.

• example:

grep 'loo*ny' tune.texts

Last updated: 06 December 2004

3

A Regular Expression is a pattern. The pattern is matched against some text. The result: the pattern will either fit or not fit. There is a lot to say about this pattern language. The text is frequently called the input. On the command line or in scripts, it can originate from an input stream (standard input) or from a set of files. In these cases the input will almost always be of a line of text. When using a programming language that supports Regular Expressions, the pattern will probably be matched against the contents of a buffer.

Note

When using Regular Expressions on the command line, such as with grep, you must

protect the Regular Expression from expansion by the shell. Putting a single (forward) quote on each side of the Regular Expression will do the job:

grep 'loo*ny' tune.texts

This (rather long) section will discuss Regular Expressions in detail. First, the Regular

Expression language (including variants) is introduced. Next, Regular Expressions are applied using specific programs. So, take a deep breath and start reading.

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 1 - 3

Primitives and Multipliers Primitives and Multipliers

• Two parts of a Regular Expression are the primitive

and the multiplier.

• They frequently occur in pairs.

• Example:

E{4}

Primitive

Multiplier

# echo "Lots of EEEE's" | egrep 'E{4}'

Last updated: 06 December 2004

4

The parts of a Regular Expression most commonly used are the primitive and the multiplier. They frequently occur in pairs. An example of such a pair is:

E{4}

In this example, E is the primitive and {4} is the multiplier, meaning four times the

thing before it. This Regular Expression matches four consecutive capital E characters somewhere in the input.

Given the Regular Expression above, for example, and an input line such as the following:

I like lots of EEEE's the Regular Expression will match the four E's in the input.

Moreover, the complete line will match, since the Regular Expression fits somewhere in the line. A member of the grep family, such as egrep will show the input if the Regular Expression matches part or all of it. So, if egrep were to handle something like this

echo "Lots of EEEE's" | egrep 'E{4}'

then egrep would show the whole line.

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 1 - 4

A multiplier may be omitted, in which case a multiplier of one time is added automatically to each primitive. So the Regular Expression:

head

effectively means:

one h, followed by one e followed by one a followed by one d.

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 1 - 5

Primitives Primitives

• There are three types of primitives:

– a regular character

– the dot placeholder

– the character class

Last updated: 06 December 2004

6

The character class is the most complicated of these.

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 1 - 6

A digit, letter or other character A digit, letter or other character

• Any regular character can be a primitive.

– What is the primitive here ?

4{2}

• Other characters can also be used as primitives:

ls -la | egrep ' {11}'

• Some characters have special meanings:

– the curly brace { }

– the backslash \ : Escape characters

Last updated: 06 December 2004

7

Any regular character can be a primitive. This includes letters (like E) and digits. For instance, in the following example the capital E is the primitive:

E{4}

What is the primitive here:

4{2}

The correct answer is 4.

Other characters can also be used as primitives in a Regular Expression. For example:

ls -la | egrep ' {11}'

(the Regular Expression is a space followed by multiplier {11}). This will show lines containing at least 11 consecutive spaces.

Some characters have special meanings, such as the curly brace ({). These characters must be escaped (they are preceded by a backslash (\)) to remove the special meaning. This will be discussed further later on.

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 1 - 7

The dot The dot

• As a primitive

• placeholder for any character

• Example:

a.c

– allows for one a, followed by any single character, followed

by one c

a.{3}

– one a followed by threearbitrary characters.

• Remember: dot . differ question ?

Last updated: 06 December 2004

8

The dot has a special meaning. As a primitive, it can be a placeholder to allow any character in that position. For instance:

a.c

allows for one (remember the one-time multiplier being added?) a, followed by any single character, followed by one c. So echo reading the abc | egrep 'a.c' fits.

And what will this fit on?

a.{3}

The answer: one a followed by three arbitrary characters.

Note

To find a literal dot, use the \. primitive.

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 1 - 8

Character classes Character classes

• is a primitive

• a placeholder for a number of characters

• It is more restrictive than the dot placeholder

• start with a [ and end it with ]

Last updated: 06 December 2004

9

A character class is a placeholder for a number of characters. It is more restrictive than the

dot placeholder, since it allows you to specify the characters the placeholder should match.

Keep in mind that the character class is a primitive, it is a placeholder for one of the specified characters.

To set up a character class, start with a [ (opening square bracket) and end it with ] (closing

square bracket). Any combination of sets and ranges can be specified between these square brackets.

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 1 - 9

in a character class A character setset in a character class A character

• specify a set of characters in a character class

[aouyA]

• the character class is a primitive. So, it can be

multiplied [aouyA]{2}

• Example:

# echo "reading the abc" | egrep 'a[bB]c'

Last updated: 06 December 2004

10

To specify a set (a series) of characters in a character class, simply put them between the square brackets. So,

[aouyA]

specifies a set consisting of an a (both lower and upper case), an o, a u or a y.

Remember that the character class is a primitive. That is, it can be multiplied:

[aouyA]{2}

This specifies two characters, each of which may be one of the characters inside the square brackets.

The order of the characters in the set is irrelevant. That is, [abcABC] is the same as the [aAbBcC] character class. So,

echo reading the abc | egrep 'a[bB]c'

fits, as would

echo reading the aBc | egrep 'a[bB]c'

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 1 - 10

in a character class range in a character class

A character range A character

• A character rangeconsists of two characters

separated by a minus sign. [0-9]

[a-z]

• Ranges can be combined

[a-z0-9]

• Some sets can be written as ranges

[ghiGHI]{2}

[g-iG-I]{2}

Last updated: 06 December 2004

11

A character range consists of two characters separated by a minus sign. This specifies one digit:

[0-9]

This is one lower case letter:

[a-z]

Ranges can be combined:

[a-z0-9]

Some sets can be written as ranges. For example, the Regular Expression:

[ghiGHI]{2}

can also be written as:

[g-iG-I]{2}

It matches two of the indicated characters.

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 1 - 11 For now, the order is defined by the ASCII character set (or derivatives like iso8859-x). In

Combinations of sets and ranges Combinations of sets and ranges

• Ranges and sets can be combined at will. – specifies one of the named vowels or a digit

[aouy0-9]

– one matches three characters

[A-Z][aouy0-9][A-Z]

• Example:

# echo "This is JaN" | egrep '[A-Z][aouy0-9][A-Z]'

Last updated: 06 December 2004

12

Ranges and sets can be combined at will. This specifies one of the named vowels or a digit:

[aouy0-9]

And this one matches three characters:

[A-Z][aouy0-9][A-Z]

Which one? Answer: three characters, the first and the third an uppercase letter with a vowel or digit in the middle. In the following example the Regular Expression matches:

echo This is JaN | egrep '[A-Z][aouy0-9][A-Z]'

Note

To include a minus sign in a character set, place it at the beginning or the end of the character class.

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 1 - 12

The inverted character class The inverted character class

• It is often desirable to exclude certain characters from

a character class.

• The inverted character class starts with a caret ^

[^0-9]

[^aeouy0-9]

Last updated: 06 December 2004

13

It is often desirable to exclude certain characters from a character class. That's where the inverted character class comes in.

The inverted character class starts with a caret (^) as the first character after the opening square bracket ([) that starts the character class. The caret sign at the beginning inverts (negates) the meaning of the character class. Where

[0-9]

fits one digit,

[^0-9]

will fit any character that is not a digit.

This, of course, can be extended with every character class:

[^aeouy0-9]

will fit anything that is neither a vowel nor a digit.

Note

To include a caret sign in a character class, do not put it at the beginning.

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 1 - 13

POSIX character classes POSIX character classes

• The POSIX character classes form an extension to the

character classes [[:keyword:]]

• The keyword is a word describing the class [[:alnum:]] same with [a-zA-Z0-9]

• To match a blank or a tab sign

[[:blank:]]

• the complete set of white-space characters

[[:space:]]

Last updated: 06 December 2004

14

The POSIX character classes form an extension to the character classes that were discussed above. They come in the following format:

[[:keyword:]]

The keyword is a word describing the class, for example, alnum or blank. Current

implementations of GNU grep and egrep, as well as GNU awk (gawk), come with manpages that describe the POSIX character classes. Let's look at some examples.

Note

The outer square brackets are the delimiters of any character class. The inner square brackets and colons plus the keyword define the POSIX character set.

A typical POSIX character class is

[[:alnum:]]

It fits exactly one alphanumeric character. It is the same as

[a-zA-Z0-9]

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 1 - 14

If you want to match a blank or a tab sign (a subset of the wider set called white space), this one might come in handy:

[[:blank:]]

If the complete set of white-space characters is to be covered, choose this one:

[[:space:]] It matches each white-space character, including space, tab, formfeed and carriage return. More on white space later.

Multiple POSIX-character-set definitions can be combined in one character class:

[[:punct:][:space:]]

This one fits a character which is either a punctuation character (such as a semicolon or a plus sign) or a white-space character.

The POSIX character set specifiers can be combined with any other character-class element, as shown below:

[0-9]{4}[[:space:],\.:][0-9]{2}

In words: four digits, then one character that can be either a white-space character, a comma, a dot or a colon, followed by two digits.

Since the POSIX character class extensions are extensions to the regular character class, it can be inverted the same way:

[^[:digit:]]

This matches one non-digit. It is identical to:

[^0-9]

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 1 - 15

Multipliers Multipliers

• Works in conjunction with a primitive.

• The multiplier is placed after the primitive.

• There are several multipliers

• *

• ?

• +

• { }

Last updated: 06 December 2004

16

A multiplier (also known as quantifier) works in conjunction with a primitive. The multiplier is placed after the primitive. Together they form a specification of how many of which characters are wanted.

Unfortunately, before multipliers can be explained in detail, the two main variants in Regular Expressions, most affecting the way multipliers are spelled, need to be introduced first. The classic Regular Expressions are the ones implemented in grep and sed. The modern, or extended Regular Expressions, on the other hand, are the ones used by awk, egrep, Perl and flex, to name a few. The multipliers shown so far were in the extended style. For this reason, egrep, was used in examples.

There are several multipliers that will be discussed here. They are the *, ? and +

multipliers (and their classic counterparts *, \? and \+, yes, the * is the same in both cases). There are also the curly-brace multipliers we already met. These are part of the POSIX standard. All relevant GNU programs have support for curly-brace multipliers on board; even versions for classic Regular Expression exist. In GNU awk (gawk) a special --posix option must be given to turn on support for curly-brace multipliers.

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 1 - 16

The * multiplier The * multiplier

• The * is the oldest multiplier around.

• It is implemented in every Regular Expression

language.

• The * means: zero or more times.

ba*c

Last updated: 06 December 2004

17

The * is the oldest multiplier around. As far as I know, it is implemented in every Regular Expression language. It is the same in classic and extended variants.

The * means: zero or more times. So,

means zero or more a's. This may not be that meaningful, since it is always true. In larger combinations, however, its use can be more meaningful:

ba*c

That is, one b, and one c, possibly with a's between them.

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 1 - 17

The ? multiplier The ? multiplier

• The ? multiplier means zero or one time.

• Example:

egrep '

Last updated: 06 December 2004

18

The ? multiplier means zero or one time, in other words, present or not present. The extended version is ?, the version to use in classic Regular Expressions is \?.

An example: in sgml/xml sections start with

egrep '

(be careful with Linux egrep, sometime ? mean zero or more time ! )

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 1 - 18

The + multiplier The + multiplier

• The + multiplier means one or more times.

• This (extended) Regular Expression

–

ba+c

–

lo+ng

Last updated: 06 December 2004

19

The + multiplier means one or more times. The extended version is +, the version to use in classic Regular Expressions is \+.

This (extended) Regular Expression

ba+c

means one b, followed by at least one a, followed by a c.

To find long, regardless of how may o's it contains, use the (extended) Regular Expression

lo+ng

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 1 - 19

The curly brace {} multiplier The curly brace {} multiplier

• The curly-brace multipliers come in four variants

• The general form is

{m,n}

– m is numbers indicating the lowest

– n is numbers indicating the highest

• Example

[0-9]{2,4}

• To specify a minimum number of times, omit the n

{m,}

• To specify a maximum number of times, omit the m

{,n}

Last updated: 06 December 2004

20

In the extended variant, a normal set of curly braces is used, as in {4}. In the classic variant, a matching pair of \{ and \} (each curly brace is prefixed by a backslash) is used, as in \{4\}.

The curly-brace multipliers come in four variants (only the extended Regular Expressions will be shown). The general form is

{m,n}

In this, both m and n are numbers indicating the lowest (m) and the highest (n) number. It matches at least m, but at most n times. For instance, to match at least two, but at most four digits:

[0-9]{2,4}

To specify a minimum number of times, omit the n (but not the comma!):

{m,}

For instance, to match at least five digits:

[0-9]{5,}

egrep -n '.{78,}' file1 file2 ...

© Summer 2004, The Saigon Center of Techniques and Technology And to show lines that are at least 78 characters long (show also line numbers), select LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 1 - 20

Multiplier Summary Multiplier Summary

extended RE

classic RE

meaning

zero or more times

zero or one time

one or more time

at least m, but maximally n times

{m,n}

\{m,n\}

up to n times

{,n}

\{,n\}

at least m times

{m,}

\{m,\}

exactly m times

{m}

\{m\}

Last updated: 06 December 2004

21 © Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 1 - 21

Portable multipliers Portable multipliers

Perl

awk, egrep

sed, grep

meaning

zero or more times

zero or one time

one or more times

at least m, but maximally n times

{m,n}

up to n times

{,n}

at least m times

{m,}

exactly n times

{n}

Last updated: 06 December 2004

22 © Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 1 - 22

Anchors Anchors

• The ^ begin anchor

# echo Once upon ... | grep '^Once'

– The $ end anchor

# ls -lF | grep '/$'

• Matching complete text

# ^[0-9]+$'

Last updated: 06 December 2004

23

The ^ begin anchor

The ^ (caret) is the anchor that specifies the beginning of the input. If you attach ^ at the

beginning of your Regular Expression, matches will then only be found at the beginning of the input.

In the example below there is a match:

echo Once upon ... | grep '^Once'

There is, however, no match in this one:

echo He said: Once ... | grep '^Once'

The $ end anchor

The $ (dollar sign) is the anchor that specifies placement at the end of the input. The $ is typically attached to the end of the Regular Expression. The Regular Expression will then only match strings at the end of the input.

In the next example all lines that end in a slash (that is, containing directory names) will match:

ls -lF | grep '/$'

In this example, there will not be a match, since pwd does not put a slash at the end of its output:

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 1 - 23

Anchors Anchors

• The \b word-boundary anchor

# echo "The one and only" | grep '\bone'

• The \< and \> word-boundary anchors

# echo The onesided page is gone | grep '\

Last updated: 06 December 2004

24

The \b word-boundary anchor

The \b word-boundary anchor fits on either side of a word. This one matches:

echo The one and only | grep '\bone'

This one does not:

echo gone with the wind | grep '\bone'

The \< and \> word-boundary anchors

The \< and \> word-boundary anchors fit, respectively, on the left and right word boundaries. They can be used together, but this is not mandatory.

In:

echo The onesided page is gone | grep '\'

the Regular Expression will not match.

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 1 - 24

Anchors Anchors

anchor

awk

egrep

Perl

grep, sed

meaning

yes

beginning

yes

ending

yes

word boundary

yes

left word boundary

yes

right word boundary

Last updated: 06 December 2004

25

Notes

^ and $ in Perl can have a slightly different meaning in conjunction with newlines.

GNU awk (gawk) does support \< and \>

GNU grep and egrep do support word boundaries. Other grep's might not do this.

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 1 - 25

Regular Expressions in sedsed Regular Expressions in

• The sed program uses the classic Regular Expression

version.

• Operations work on a line that has been read from a

file or standard input.

Last updated: 06 December 2004

26

This section will only describe what Regular Expressions look like in sed.

The sed program uses the classic Regular Expression version. Probably because of a fear of breaking old software, few new features have been added, not even in GNU sed.

Remember that sed is line oriented. Operations work on a line that has been read from a file or standard input. The global modifier for the substitute command works on a line basis. When global is turned on, the replacement is done as often as possible in each line. When global is kept off, then the replacement will work once in each input line.

As described earlier, parsing white space is a burden in sed. Luckily, GNU sed 3.02

supports the POSIX character-set primitives, so [[:space:]]\+ can be used to match one or more white-space characters.

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 1 - 26

Regular Expressions in awkawk Regular Expressions in

• There are at least three variants of awk for Linux.

• GNU awk (gawk) is the most sophisticated.

Last updated: 06 December 2004

27

This section will only describe what Regular Expressions look like in awk.

There are at least three variants of awk for Linux. Of those, GNU awk (gawk) is the most sophisticated. The gawk distribution comes with great documentation.

All awk variants accept the extended Regular Expressions described earlier, but, currently, without the curly-brace multipliers (called interval expressions in awk terminology). Keep in mind that gawk has some extensions that the other awk variants do not accept, such as POSIX character-class extensions and the \< and \> word anchors. The interval expressions are supported in gawk, but are not enabled by default for portability with other awk implementations. To enable these, use either the --posix or the --re-interval option.

You might have wondered why most awk variants do not have support for word anchors.

Since each awk split its input into fields (which are normal words by default) these fields can be checked with normal begin and end anchors. The following checks, for example, if the second field (word) begins with an uppercase letter:

echo The Ape | awk '$2 ~ /^[A-Z]/ {print $2 ": starts upcase"}'

Creating a white-space character class in awk is easy: [ \t] (as was shown earlier). In gawk,

the POSIX white-space character classes can be used instead. This is, of course, not portable to other awks.

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 1 - 27

Perl Regular Expressions Perl Regular Expressions

in Perl

description

same in awk

a digit

[0-9]

a non-digit

[^0-9]

a white-space character

[ \t\f\r]

a non-white-space character

[^ \t\f\r]

a “word” character

[a-zA-Z0-9_]

a non-“word” character

[^a-zA-Z0-9_]

Last updated: 06 December 2004

28

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 1 - 28

Summary Summary

• Introducing Regular Expressions

• Primitives and Multipliers

• Anchors • Regular Expressions in sed • Regular Expressions in awk • Perl Regular Expressions

Last updated: 06 December 2004

29

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 1 - 29

Question & Answer Question & Answer

Last updated: 06 December 2004

30

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 1 - 30

LPI 201 Intermediate Level Administration –– LPI 201 Intermediate Level Administration

Chapter 9 Chapter 9

Part 2: Part 2: Perl language Perl language

Last updated: 06 December 2004

1

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 2 - 1

Objectives Objectives

• Introduction to Perl

• Perl basics

• Perl taint mode

• Perl modules

• CPAN

• Command line arguments

Last updated: 06 December 2004

2

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 2 - 2

Introduction to Perl Introduction to Perl

• Perl : Practical Extraction and Report Language

• The Perl language optimized for scanning text files;

• extracting information from those text files and

printing reports

• It can also be used for many system-management

tasks

• Perl combines features of C, sed, awk and sh • Perl is an interpreted language

Last updated: 06 December 2004

3

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 2 - 3

Perl basics Perl basics

• perl will use standard input by default

• execute Perl statements:

$ perl

print "hello world\n";

exit 0;

( Ctrl-D )

hello world

$

• execute Perl statements in file (script):

$ perl perlfile

Last updated: 06 December 2004

4

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 2 - 4

Perl basics Perl basics

• perlfile

#!/usr/bin/perl

statement1;

# a comment line

statement2;

statement3; # end of line comment

• block

if ($x > 5)

{

statement5;

statement6;

}

Last updated: 06 December 2004

5

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 2 - 5

Perl basics Perl basics

• print

– built-in function

– takes a list containing one or more arguments and prints

them

• example

$v = 10;

$h = 33.6;

print( "The area of a %f x %f rectangle is %10.3f\n", $v, $h, $v * $h );

Last updated: 06 December 2004

6

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 2 - 6

Perl basics Perl basics

• variables

– $var names a variable that holds a scalar data type

(integer, floating point or string).

– @var names a variable that holds an array of scalars or list. – %var names a variable that holds a hash of scalars.

• Scope of variables

– By default, all variables in Perl are globalvariables

– create private variables called lexical variables at any time

with the my operator

{

my($x, $y); # private variables for this block

}

Last updated: 06 December 2004

7

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 2 - 7

Perl basics Perl basics

• Single-quoted strings

print 'Some $verse here \n';

The output:

Some $verse here \n

• Double-quoted strings

my $verse = 'hallo there';

print "Some $verse here\n";

The output

Some hallo there here

Last updated: 06 December 2004

8

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 2 - 8

Perl basics Perl basics

• Array

– An array is a singly dimensioned vector of scalar quantities.

– An array containing “n” elements has indices starting at 0

and going to n-1.

() # the empty list, or array

(1,2,3) # an array of three integers

(1,'fred',27.1,3*5)

– references an array @vec = (3,4,5);

$sum = $vec[0] + $vec[2];

$vec[1] = $sum;

Last updated: 06 December 2004

9

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 2 - 9

Perl basics Perl basics

• Hash

– name always begins with a "%"

%box = ("len", 100, "height", 40, "colour", "blue");

%box = ("len" =>100, "height" =>40, "colour" =>"blue");

– references an hash

print "height is ", $box{'height'};

Last updated: 06 December 2004

10

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 2 - 10

Perl basics Perl basics

• A subroutine

– Perl has the ability to use user-defined subroutines or

functions.

– Defining a subroutine

my $n = 0; # file-wide variable $n

...

sub marine

{

$n += 1;

print "Hello, sailor number $n!\n";

}

– Calling a subroutine

&marine; # says Hello, sailor number 1!

Last updated: 06 December 2004

11

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 2 - 11

Perl basics Perl basics

• Numeric Comparisons

< less

<= less or equal

== equals

!= not equal

>= bigger or equal

> bigger

– If you compare strings using these numeric comparison operators, the strings are first converted to numeric quantities and the numeric quantities compared.

Last updated: 06 December 2004

12

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 2 - 12

Perl basics Perl basics

• String Comparisons

lt lesser than

le lesser or equal

eq equals (identical)

ne not equal

ge greater or equal

gt greater than

Last updated: 06 December 2004

13

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 2 - 13

Escape characters in Perl Escape characters in Perl

\t \n

tab newline

return

formfeed

backspace

alarm (bell)

\e \033

escape character octal character

double quote

literal backslash

Last updated: 06 December 2004

14

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 2 - 14

Perl basics Perl basics

• Pattern matching with Regular Expressions

– the variable: $some – the binding operator: =~ , !~ – a Regular Expression enclosed in slashes: /^[A-Z]/ if ($some =~ /^[A-Z]/)

{

# match: do something

}

– OR my @results = $input =~ /^([A-Z])([a-z]+)/;

if (@results)

{

Last updated: 06 December 2004

15

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 2 - 15

Perl basics Perl basics

• File handle – Read file open(IN, "

while ()

{

# most recent line read in $_

print $_;

}

close(IN);

– Another way, assign to array: my @allLines = ;

Last updated: 06 December 2004

16

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 2 - 16

Perl basics Perl basics

• File handle

– Write to file

open(OUT,">/opt/data.txt") || die "$0: unable to open";

print OUT ("this is text file.");

close(OUT);

Last updated: 06 December 2004

17

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 2 - 17

Perl basics Perl basics

• Proccess handles

open (WHO, "who|") || die "cannot start who";

my @whoLines = ;

close (WHO);

Last updated: 06 December 2004

18

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 2 - 18

Perl basics Perl basics

• loop: for, while

my $i; my $i = 0;

for($i = 0; $i < 20; $i++)

while ($i < 20)

{ {

print "\$i: $i\n"; print "\$i: $i\n";

$i++;

} }

@a = ("a"..."z","A"..."Z");

foreach $n (@a)

{

print $n;

}

Last updated: 06 December 2004

19

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 2 - 19

Perl basics Perl basics

• Logic/branching

if( $le < 10.0 )

{

print( "Length $le is too small!\n" );

}

elsif( $le > 100.0 )

{

print( "Length $le is too big!\n" );

}

else

{

print( "Length is just right!\n" );

}

Last updated: 06 December 2004

20

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 2 - 20

Perl taint mode Perl taint mode

• perl automatically enables a set of special security

checks

• Enable taint modeexplicitly by using the -T flag

#!/usr/bin/perl -T

Last updated: 06 December 2004

21

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 2 - 21

Here is a list of functions that Perl considers dangerous while in taint mode:

exec() -- executes a system command and passes the program flow control to it;

system() -- same as exec(), except it forks a child process first and waits for it to return; open() -- opens a file for input or output and associates a file handle with it; glob() -- expands a filename pattern to a full pathname according to the rules used by the shell; unlink() -- deletes one or more files; mkdir() -- creates a directory; chdir() -- changes the current directory; rmdir()-- deletes a directory; chown() -- changes the ownership information (UID and GID) of one or more files; chmod() -- changes the permissions of one or more files; umask() -- sets the umask that the process will use to mask file permissions; link() -- creates a new hardlink to a file; symlink() -- creates a new symbolic link to a file; kill() -- sends a signal to one or more processes; eval() -- parses and evaluates Perl code; truncate() -- truncates a file to a specified length; ioctl() -- manipulates device parameters of special files; fcntl() -- manipulates file descriptors; chroot() -- makes a directory the new root directory for all further pathnames that starting with '/'; setpgrp() -- sets the current process group for a specified PID; setpriority() -- sets the current priority of a process; syscall() -- performs a system call with the specified arguments; socket() -- opens a socket and attaches it to a file handle; socketpair() -- creates an anonymous pair of sockets; bind() -- binds a network address to a socket; connect() -- connects to a remote socket; _________________________________________________________________________

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 2 - 22

CPANCPAN

• CPAN: Comprehensive Perl Archive Network

• There are hundreds of free modules available

– Available modules include support for access to Oracle and

other databases (DBI);

– networking protocols such as HTTP, POP3 and FTP and

support for CGI.

• Start the CPAN shell (as root) # perl -MCPAN -e shell;

• CPAN site

– http://www.cpan.org

Last updated: 06 December 2004

23

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 2 - 23

Command line arguments Command line arguments

• pass parameter to script

$ cat test.pl

!#/usr/bin/perl

print $ARGV[0]

# ./test.pl "hello world"

hello world

Last updated: 06 December 2004

24

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 2 - 24

Summary Summary

• Introduction to Perl

• Perl basics

• Perl taint mode

• Perl modules

• CPAN

• Command line arguments

Last updated: 06 December 2004

25

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 2 - 25

Practice Practice

• Write script to get 3 numbers from command line

– Print the Max number.

– Print the Min number.

Last updated: 06 December 2004

26

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 2 - 26

Question & Answer Question & Answer

Last updated: 06 December 2004

27

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 2 - 27

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 2 - 28

LPI 201 Intermediate Level Administration –– LPI 201 Intermediate Level Administration

Chapter 9 Chapter 9

Part 3 Part 3

System Logging and Automation System Logging and Automation

Last updated: 06 December 2004

1 © Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 3 - 1

Objectives Objectives

• System logging

– syslogd

– klogd

• Automate tasks

– cron

– at

– anacron

Last updated: 06 December 2004

2

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 3 - 2

Introduction to System logging Introduction to System logging

• One integral part of any UNIX system are the logging

facilities.

• The majority of logging in Linux is provided by two

main programs: – syslogd : logging services to programs and applications.

– klogd : logging capability to the Linux kernel.

Last updated: 06 December 2004

3

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 3 - 3

Introduction to System logging Introduction to System logging

• klogd actually sends most messages to the syslogd facility but will on occasion pop up messages at the console (i.e. kernel panics).

• syslogd actually handles the task of processing most messages and sending them to the appropriate file or device, this is configured from within /etc/syslog.conf

• By default most logging to files takes place in

/var/log/

Last updated: 06 December 2004

4

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 3 - 4

Rotating log files Rotating log files

• A common way to keep logging information for a fixed

period is “rotation”.

• Example: rotation policy - if log file is called logfile

#!/bin/sh

cd /var/log

mv logfile.2 logfile.3

mv logfile.1 logfile.2

mv logfile logfile.1

cat /dev/null > logfile

chmod 600 logfile

• every day logfile.7 is lost as logfile.6 overwrites it.

Last updated: 06 December 2004

5

You keep backup files that are one day old, two days old, and so on. Each day a script or utility program renames the files to push older data toward the end of the chain.

One common problem is figuring out what to do with all of the log messages. If you do a lot of logging (particularly if everything is sent to a central server), you can fill up your filesystem faster than you think. The most obvious and direct solution is to remove them after a specific length of time or when they reach a particular size.

As its name implies, the goal of the logrotate program is to "rotate" log files. This could be as simple as moving a log file to a different name and replacing the original with an empty file.

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 3 - 5

Logrotate: manage log files:

A logrotate configuration file consists of a series of specifications for groups of logfiles to be managed.

# Example log rotation policy

errors sa-book@admin.com

rotate 5

weekly

/var/log/messages{

postrotate

/bin/kill –HUP ‘cat /var/run/syslogd.pid

endscript

}

/var/log/samba/*.log{

notifempty

copytruncate

sharedscripts

postrotate

/bin/kill –HUP ‘cat /var/run/syslogd.pid

endscript

}

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 3 - 6

Linux log files Linux log files

• For the most part, linux packages send their logging

information to files in the /var/log directory.

• Some distribution logs are also stored in /var/adm • Special log files

– /var/log/wtmp : contains a record of users’ logins and

logouts. Should be rotated .

– /var/log/lastlog : it records only the time of last login

for each user. No need to be rotated.

Last updated: 06 December 2004

7

Special log files

o /var/log/wtmp : contains a record of users’ logins and logouts, entries that indicate when the system was rebooted or shut down. Should be rotated.

# who /var/log/wtmp

o /var/log/lastlog it records only the time of last login for each user. Is binary file, indexed by UID. No need to be rotated because its size stay constant unless new users log in.

# lastlog

o checksecurity utility runs every day (from etc/cron.daily/standard) to enumerate the systems complement of setuid programs( setuid.changes, setuid.today, setuid.yesterday).

Download (debian): http://packages.debian.org/unstable/admin/checksecurity

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 3 - 7

Configuring syslogd Configuring syslogd

• The configuration file /etc/syslog.conf • Basic format is:

selector action

mail.info

/var/log/maillog

• Within the selector, “mail” is the facility and “info”

is the level of priority

• /var/log/maillog is the action

Last updated: 06 December 2004

8

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 3 - 8

Facilities Facilities

• Supported facilities in Linux:

Syslog Facility

Associated Subsystem

authpriv

cron

cron subsystem

daemon

System server processes

kern

Linux kernel

lpr

Spooling subsystem

mail

Mail subsystem

news

News subsystem

Last updated: 06 December 2004

9

Locally-defined syslog facilities N runs from 0 to 7

localN

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 3 - 9

Unlike facilities, which have no relationship to each other, priorities are hierarchical. Possible priorities in Linux are (in increasing order of urgency): debug, info, notice, warning, err, crit, alert and emerg.

As with facilities, the wildcards * and none.

A priority may be preceded by either or both of the modifiers = and !

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 3 - 10

priorities priorities

Level

Keyword

Description

emerg

emergencies, System unusable

0 alert

alerts, Immediate action required

1 crit

Critical condition

2 err

Error conditions

3 warning

Warning conditions

4

5 notice

notifications, Normal but significant conditions

6 info

Informational messages

7 debug

Debugging messages

Last updated: 06 December 2004

11

Unlike facilities, which have no relationship to each other, priorities are hierarchical. Possible

priorities in Linux are (in increasing order of urgency): debug, info, notice, warning, err, crit, alert and emerg.

As with facilities, the wildcards * and none.

A priority may be preceded by either or both of the modifiers = and !

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 3 - 11

Example Example

.info;mail.none;authpriv.none;cron.none .info;mail.none;authpriv.none;cron.none

/var/log/messages /var/log/messages

• All messages of severity "info" and above are logged,

but none from the mail, cron or authentication.

selector

meaning

Mail.info

Select mail-related mesg. Of info prio. And higher

Mail.=info

Select only messages at info priority

Mail.info;mail.!err

Select only prio. Info, notice and warning

Select all priorities except warning

Mail.debug; mail.!=warning

Last updated: 06 December 2004

12

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 3 - 12

action action

• In practice, most log messages are written to files.

• You can send messages other places too.

Action

meaning

Filename

Writes the mesg. To file on the local machine

@hostname

Forward the msg. To syslogd on host name

@ipaddress

Forward the mesg. To the host at ip address ipaddress

\fifoname

Writes the mesg. To the named pipe fifoname

user1,user2,…

Writes the mesg. To user’s screen

*

Writes the mesg. To all users who are logged in

Last updated: 06 December 2004

13

In practice, most log messages are written to files. If you list the full path to a filename as a line's action in syslog.conf, messages that match that line will be appended to that file. (If the file doesn't exist, syslog will create it.)

You can send messages other places too. An action can be a file, a named pipe, a device file, a remote host or a user's screen.

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 3 - 13

Config file examples Config file examples

• stand-alone machine on a small network.

#emergencies: tell every one who is logged on *.emerge # important messages *.warning; daemon, auth.info,user.non /var/log/messages # printer errors lpr.debug

/var/log/lpd-errs

Last updated: 06 December 2004

14

The syslog config file can be customized for each host.

Choose stable machine as your logging server, one that is well secured and does not have many logins.

Some very large sites may want to add more levels to the logging hierarchy. Unfortunately syslog

retains the name of the originating host for only one hop. If a host “client” sends some log entries to host “server”, which sends them on to host “master,” master will see the data coming from server, not from client.

_________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________ _________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 3 - 14

Config file examples Config file examples

• a client machine on a larger network

@netloghost @netloghost

# emergencies tell everyone who is logged on *.ernerg;user.none # Forward important messages to the central logger *.warning; lpr,local.none daemon,auth.info # Send local staff to the central logger too

1ooa12.info; local0,local7,debug @netloghost

@boulder.colorado.edu

/var/log/lpd-errs

/var/log/sudo.log

#cardd logs through facility local1 – send to boulder local1.debug # keeping printer errors local lpr.debug #sudo logs to local2 – keep a copy here too local2.info # keeping kernel messages local kern.info

/var/log/kern.log

Last updated: 06 December 2004

15

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 3 - 15

Creating a central syslog server

In this example, I will configure our Solaris server environment to send syslog information to a remote Red Hat Linux 8.0 server.

Changes needed on the syslog server (Red Hat Linux 8.0):

vi /etc/sysconfig/syslog

Change: SYSLOGD_OPTIONS="-m 0"

To: SYSLOGD_OPTIONS="-m 0 -r -x"

This change to the syslog daemon enables logging from remote machines and disables DNS lookups on incoming syslog messages.

Changes needed on the syslog clients (Sun Solaris):

1. vi /etc/syslog.conf

Add the following lines:

# Send a copy to remote loghost *.info @loghost auth.* @loghost

Note: make sure that you do not have extra whitespace in the Solaris syslog.conf file. Separate the facility and severity from the location with either a single space or with tabs.

2. vi /etc/hosts

Remove any reference to "loghost." By default, Solaris will configure each host to be its own loghost.

3. Send the syslogd process a SIGHUP signal (kill -HUP pid_of_syslogd).

If you are using DNS, you will want to add a DNS A record for your "loghost" server. Since it may already have an entry in DNS, you may wish to use a DNS CNAME record.

If you are using NIS in your environment, you may want to add "loghost" to your NIS hosts map.

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 3 - 16

Command atat Command

• Execute commands at a specified time or run the

commands on a batch queue.

at [options]

• at reads commands from stdin or file (with –f

option) and executes them using user’shell.

Last updated: 06 December 2004

17

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 3 - 17

Examples atat Examples

• Run myprogram once at 6:15 p.m tomorrow:

$ at 6:15pm tomorrow

at> myprogram

at> ^D

• The ^D ( ctrl-D ) : sending the end-of-file

character to terminate the at command.

Last updated: 06 December 2004

18

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 3 - 18

Listing and Deleting at Jobs Listing and Deleting at Jobs

List jobs, it’s an alias for atq

– at –l

Removes jobs, it’s an alias for atrm

– at –d job_id

– atq

– atrm job_id

Like at -l Like at -d

• Example

# atq

14 2003-10-31 12:00 a root

# atrm 14

# atq

Last updated: 06 December 2004

19

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 3 - 19

cron's uses cron's uses

• Remove junk files from the system.

• Reload mail aliases

• Reset / Rotate log files

• Synchronize databases

• Checking network resources

Last updated: 06 December 2004

20

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 3 - 20

crontab crontab

• crontab is located in three places:

– /var/spool/cron

• contains user's crontabs

• based on username

– /etc/cron.d

• typically used by software installations

– /etc/crontab

• maintained by system administrator

Last updated: 06 December 2004

21

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 3 - 21

how cron works how cron works

• Cron reads all its config files into memory then sleeps.

• Every minute cron wakes

– checks for updates to its config files

– reloads updated files

– executes any tasks scheduled for that minute.

• Output of cron is emailed to the owner of the crontab

Last updated: 06 December 2004

22

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 3 - 22

crontab fields crontab fields

min hr day month wday [usr] cmd

– minute [0-59]

– hour [0-23]

– day [1-31]

– month [1-12]

– weekday [0-6 ] (0=Sunday)

Last updated: 06 December 2004

23

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 3 - 23

crontab format crontab format

• Comment lines start with #

• Fields are separated by whitespace

– whitespace is taken literally in the command field

• The command should notbe quoted

Last updated: 06 December 2004

24

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 3 - 24

crontab format crontab format

Options

Explanation

*

Is treated as a wild card. Meaning any possible value.

*/5

Is treated as ever 5 minutes, hours, days, or months. Replacing the 5 with another numerical value will change this option.

2,4,6

Treated as an OR, so if placed in the hours, this could mean at 2, 4, or 6 o-clock.

9-17

Treats for any value between 9 and 17. So if placed in day of month this would be days 9 through 17. Or if put in hours it would be between 9 and 5.

Last updated: 06 December 2004

25

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 3 - 25

command crontab command crontab

• Command to modify a users crontab • Flags

– e : Edit the current cron table – l : lists the contents of the user's crontab – r : removes the user's crontab – u : specifies a username (root only)

• Each user has a personal list of commands kept in the crontab file, stored in /var/spool/cron/ directory

Last updated: 06 December 2004

26

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 3 - 26

Examples Examples

• To execute myprogram once per day at 6:15 a.m , use

this crontab entry:

15 6 * * * myprogram

• To execute at 6:15 and 18:15 on the 1st and 15th of

the month, use:

15 6,18 1,15 * * myprogram

Last updated: 06 December 2004

27

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 3 - 27

Administering atat and Administering

and crontab crontab

• Control and configuration files are kept in /etc

at.allow

Users allowed to use at

at.deny

cron.allow

cron.deny

crontab

Users denied use of at (only used if no at.allow) Users allowed to use cron Users denied use of cron System cron table

Last updated: 06 December 2004

28

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 3 - 28

Administering atat and Administering

and crontab crontab

• All crontab are stored in the cron spool directory

/var/spool/cron

• All at jobs are stored in the cron spool directory

/var/spool/at

• Do not edit the files in the spool directories.

– Use the at and crontab utilities to make changes

Last updated: 06 December 2004

29

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 3 - 29

anacron anacron

• anacron is a task scheduler similar to cron; • except that it does not require the system to run

continuously. – It can be used to run the daily, weekly, and monthly jobs

usually run by cron.

• To use the anacron service, you must have:

– the anacron RPM package installed – the anacron service must be running.

Last updated: 06 December 2004

30

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 3 - 30

format anacrontab format anacrontab

• Anacron tasks are listed in the configuration file

/etc/anacrontab.

period delay job-identifier command

frequency (in days) to execute the command

– period

delay time in minutes

– delay – job-identifier description of the task.

command to execute

– command

Last updated: 06 December 2004

31

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 3 - 31

how cron works how cron works

• Anacron executes the command specified in the

command field after waiting the number of minutes specified in the delay field.

• After the task is completed, Anacron records the date in a timestamp file in the /var/spool/anacron directory.

Last updated: 06 December 2004

32

For each tasks, Anacron determines if the task has been executed within the period specified in the period field of the configuration file. If it has not been executed within the given period, Anacron executes the command specified in the command field after waiting the number of minutes specified in the delay field.

After the task is completed, Anacron records the date in a timestamp file in the /var/spool/anacron directory. Only the date is used (not the time), and the value of the job-identifier is used as the filename for the timestamp file.

Environment variables such as SHELL and PATH can be defined at the top of /etc/anacrontab as with the cron configuration file.

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 3 - 32

anacrontab Example anacrontab Example

# /etc/anacrontab: configuration file for anacron

# These entries are useful for a Red Hat Linux system.

1 5 cron.daily run-parts /etc/cron.daily

7 10 cron.weekly run-parts /etc/cron.weekly

30 15 cron.monthly run-parts /etc/cron.monthly

Last updated: 06 December 2004

33

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 3 - 33

Summary Summary

• sysklogd can be configured for both local and remote

logging.

• In the syslog.conf file, every rule consists these two

fields: – selector field

– action field

• The three major process scheduling utilities: – at: executes a process once at a specified time

– cron: executes a process repeatedly at specified intervals

– anacron: like cron, but can schedule processes with within

smaller time intervals than what cron allows

Last updated: 06 December 2004

34

o selector field: defines the application/subsystem that generates the message and its severity

o action field: defines what to do with the messages (i.e. write the messages to a file, send the messages to a particular tty, etc.)

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 3 - 34

Question & Answer Question & Answer

Last updated: 06 December 2004

35

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 3 - 35

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration –System Logging and Automation Chapter 09, Part 3 - 36

LPI 201 Intermediate Level Administration –– LPI 201 Intermediate Level Administration

Chapter 10 Chapter 10

Troubleshooting Troubleshooting

Last updated: 06 December 2004

1

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Troubleshooting Chapter 10 - 1

Objectives Objectives

• Creating recovery disks

• Troubleshooting LILO

• General troubleshooting

• Troubleshooting system resources

• Troubleshooting environment configurations

Last updated: 06 December 2004

2

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Troubleshooting Chapter 10 - 2

Creating boot disks Creating boot disks

• command

mkbootdisk [options] kernel_version

mkbootdisk --device /dev/fd0 1.4.29

– select or build a kernel

– copy the kernel to a floppy disk

– create and populate a root filesystem

– compress the root filesystem

– copy the root filesystem to floppy disk

– set the ramdisk word

Last updated: 06 December 2004

3

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Troubleshooting Chapter 10 - 3

Creating loopback Creating

devices loopbackdevices

1. Create plainfile:

dd if=/dev/zero of=fsfile bs=512 count=200

• This creates a file that contains 100K of zero bytes, named

“fsfile”.

2. Associate one of the loopback devices with that file

losetup -e none /dev/loop0 fsfile

• The -e none is default and specifies that encryption

should not be used

Last updated: 06 December 2004

4

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Troubleshooting Chapter 10 - 4

Creating loopback Creating

devices loopbackdevices

3. Format and mount loopback device

mkfs -t ext2 /dev/loop0 100 mount -t ext2 /dev/loop0 /mnt

4. unmount device

umount /dev/loop0 losetup -d /dev/loop0

Last updated: 06 December 2004

5

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Troubleshooting Chapter 10 - 5

Troubleshooting LILO Troubleshooting LILO

(Chapter 03)

– : No part of LILO has been loaded – L : The first stage boot loader has been loaded – LI : The first stage boot loader was able to load

the second stage

– LIL : The second stage boot loader has started, but

it can't load descriptor table.

– LIL? : boot loader has been loaded at an incorect address – LIL- : The descriptor table is corrupt – LILO : All parts of LILO have been successfully loaded

Last updated: 06 December 2004

6

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Troubleshooting Chapter 10 - 6

General troubleshooting General troubleshooting

• Redhat:

– http://www.redhat.com/services/consulting/

• SuSE

– http://support.novell.com/linux/index.html

• Debian

– http://www.debian.org/doc/FAQ/ch-support.html

• Mandrake

– http://www.mandrakeexpert.com/index1.php

Last updated: 06 December 2004

7

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Troubleshooting Chapter 10 - 7

Troubleshooting system resources Troubleshooting system resources

• Core system variables

– HOME, LOGNAME, UID, MANPATH, PAGER,

PATH, PS1..4, SHELL

• /etc/shells

– lists the valid login shells on your system

• Shell startup environment

– /etc/profile

– ~/.bash_profile

– ~/.bash_login

– ~/.profile

Last updated: 06 December 2004

8

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Troubleshooting Chapter 10 - 8

Troubleshooting system resources Troubleshooting system resources

• Setting kernel parameters

– sysctl

• Shared libraries

– LD_LIBRARY_PATH environment variable – ldconfig creates a cache file – ldd resolve the matching filename

• lists the valid login shells on your system

– /etc/shells

Last updated: 06 December 2004

9

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Troubleshooting Chapter 10 - 9

Troubleshooting environment configurations Troubleshooting environment configurations

• /etc/inittab

• /etc/passwd

• /etc/shadow

• /etc/group

• /etc/profile

• /usr/sbin/cron

• /usr/bin/crontab

• /var/spool/cron/crontabs/

• /etc/login.defs

• /etc/syslog.conf

Last updated: 06 December 2004

10

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Troubleshooting Chapter 10 - 10

Summary Summary

• Creating recovery disks

• Troubleshooting LILO

• General troubleshooting

• Troubleshooting system resources

• Troubleshooting environment configurations

Last updated: 06 December 2004

11

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Troubleshooting Chapter 10 - 11

Question & Answer Question & Answer

Last updated: 06 December 2004

12

_________________________________________________________________________

© Summer 2004, The Saigon Center of Techniques and Technology LPI 201 – Intermediate Level Administration – Troubleshooting Chapter 10 - 12

Appendix: Student Notes

__________________________________________________________________________

The Linux Kernel.......................................................................................................................8 1. Kernel Components..........................................................................................................8 2. Compiling a Kernel..........................................................................................................10 3. Patching a Kernel............................................................................................................11 4. Customising a Kernel......................................................................................................14

System Startup........................................................................................................................19 1. Customising the Boot Process........................................................................................19 2. System Recovery............................................................................................................22 3. Customised initrd............................................................................................................25

The Linux Filesystem..............................................................................................................29 1. Operating the Linux Filesystem.......................................................................................29 2. Maintaining a Linux Filesystem.......................................................................................31 3. Configuring automount....................................................................................................33

Hardware and Software Configuration....................................................................................36 1. Software RAID................................................................................................................36 2. LVM Configuration..........................................................................................................40 3. CD Burners and Linux.....................................................................................................47 5. Configuring PCMCIA Devices.........................................................................................49

File and Service Sharing.........................................................................................................51 1. Samba Client Tools.........................................................................................................51 2. Configuring a SAMBA server..........................................................................................52 2. Configuring an NFS server..............................................................................................55 3. Setting up an NFS Client.................................................................................................58

System Maintenance...............................................................................................................60 1. System Logging..............................................................................................................60 2. Packaging Software........................................................................................................62

System Automation.................................................................................................................65 1. Writing simple perl scripts (using modules).....................................................................65 2. Using the Perl taint module to secure data.....................................................................66 3. Installing Perl modules (CPAN).......................................................................................67 4. Check for process execution...........................................................................................68 5. Monitor Processes and generate alerts..........................................................................69 6. Using rsync.....................................................................................................................72

LinuxIT Technical Training Centre

The Linux Kernel

___________________________________________________________________

The Linux Kernel

This module will describe the kernel source tree and the documentation available. We will also apply patches and recompile patched kernels. Information found in the /proc directory will be highlighted.

1. Kernel Components

Modules

Module Components in the Source Tree

In the kernel source tree /usr/src/linux, the kernel components are stored in various subdirectories:

Example

nfs ipx

Subdirectory Description ./drivers ./fs ./net

contains code for different types of hardware support pcmcia code for filesystem supported code for network support

These components can be selected while configuring the kernel (see 2. Compiling a Kernel).

Module Components at Runtime The /lib/modules//kernel directory, has many of the same subdirectories present in the kernel source tree. However only the modules that have been compiled will be stored here.

Types of Kernel Images

The various kernel image types differ depending only on the type of compression used to compress the kernel.

The make tool will read the /usr/src/linux/Makefile to compile

• A compressed linux kernel using gzip is compiled with: make zImage The compiled kernel will be:

/usr/src/linux/arch/i386/boot/zImage

LinuxIT Technical Training Centre

The Linux Kernel

___________________________________________________________________

• A compressed linux kernel using better compression is compiled with: make bzImage The compiled image will be:

/usr/src/linux/arch/i386/boot/bzImage • One can also use: make zdisk or make bzdisk to create compressed kernels on

a floppy. The compiled kernel will be written to:

/dev/fd0

Remember to put a floppy in the drive!

Documentation

Most documentation is available in the /usr/src/linux/Documentation directory. The main files are the following:

File 00-INDEX

Description Summary of the contents for each file in the Documentation directory

Configure.help Contains the help displayed when configuring a kernel

The Configure.help file also provides further information for when a kernel module doesn't load properly. Specific options and aliases for /etc/modules.conf are specified in that file.

Information about compiling and documentation is available in /usr/src/linux/README.

The version of the kernel is set at the beginning of the Makefile. VERSION = 2 PATCHLEVEL = 4 SUBLEVEL = 22 EXTRAVERSION = Make sure to add something to the EXTRAVERSION line like EXTRAVERSION=-test

This will build a kernel called 2.4.22-test Notice: You need the “-” sign in EXTRAVERSION or else the version will be 2.4.22test

LinuxIT Technical Training Centre

The Linux Kernel

___________________________________________________________________

2. Compiling a Kernel

Compiling and installing a kernel can be described in three stages.

Stage 1: configuring the kernel

Here we need to decide what kind of hardware and network support needs to be included in the kernel as well as which type of kernel we wish to compile (modular or monolithic). These choices will be saved in a single file:

/usr/src/linux/.config

Creating the .config file Command make config make menuconfig make xconfig

Description Edit each line of .config one at a time Edit .config browsing through menus (uses ncurses) Edit .config browsing through menus (uses GUI widgets)

When editing the .config file using any of the above methods the choices available for most kernel components are:

Do not use the module (n) Statically compile the module into the kernel (y) Compile the module as dynamically loadable (M)

Notice that some kernel components can only be statically compiled into the kernel. One cannot therefore have a totally modular kernel.

When compiling a monolithic kernel none of the components should be compiled dynamically.

Stage 2: compiling the modules and the kernel

The next table outlines the various 'makes' and their function during this stage. Notice that not all commands actually compile code and that the make modules_install has been included

Compiling Command

Description

LinuxIT Technical Training Centre

The Linux Kernel

___________________________________________________________________

make clean

makes sure no stale .o files have been left over from a previous build make dep adds a .depend with headers specific to the kernel components build the kernel make make modules build the dynamic modules make modules_install install the modules in /lib/modules/kernel-version/

Stage 3: Installing the kernel image

This stage has no script and involves copying the kernel image manually to the boot directory and configuring the bootloader (LILO or GRUB) to find the new kernel.

3. Patching a Kernel

Incremental upgrades can be applied to an existing source tree. If you have downloaded the linux-2.4.21.tgz kernel source and you want to update to a more recent kernel linux- 2.4.22 for example, you must download the patch-2.4.22.gz patch.

Applying the Patch

The patch file attempts to overwrite files in the 2.4.21 tree. One way to apply the patch is to proceed as follows:

cd /usr/src

zcat patch-2.4.22.gz | patch -p0

The -p option can strip any number of directories the patch is expecting to find. In the above example the patch starts with:

--- linux-2.4.21/... +++ linux-2.4.22/...

This indicates that the patch can be applied in the directory where the linux-2.4.21 is.

However if we apply the patch from the /usr/src/linux-2.4.21 directory then we need to strip the first part of all the paths in the patch. So that

--- linux-2.4.21/arch/arm/def-configs/adsagc +++ linux-2.4.22/arch/arm/def-configs/adsagc

LinuxIT Technical Training Centre

The Linux Kernel

___________________________________________________________________

becomes

--- ./arch/arm/def-configs/adsagc +++ ./arch/arm/def-configs/adsagc

This is done with the -p1 option of patch effectively telling it to strip the first directory.

cd /usr/src/linux-2.4.21

zcat patch-2.4.22.gz | patch -p1

Testing the Patch

Before applying a patch one can test what will be changed without making them: patch -p1 –dry-run < patchfile

Recovering the Old Source Tree

To make sure the old configuration (.config file) is saved copy the .config file to the /boot directory.

cp .config /boot/config-kernelversion

The patch tool has two ways of keeping track of the changed files:

1. You can apply the patch with the -b option

patch -b -p0 < patch-file

By default this option keeps all the original files and appends a “.orig” to them.

2. You can backup the old changed file to a directory of your choice

mkdir oldfiles

patch -B oldfiles/ -p0 < patch-file

This has the advantage of letting you create a backup patch that can restore the source

LinuxIT Technical Training Centre

The Linux Kernel

___________________________________________________________________

tree to it's original state.

diff -ur linux-2.4.21 oldfiles/linux-2.4.21 > recover-2.4.21- patch

NOTICE

Applying this recover-2.4.21-patch will have the effect of removing the 2.4.22 patch we just applied in the previous paragraph

Building the New Kernel after a patch

Simply copy the old .config to the top of the source directory.

cp /boot/config-kernelversion /usr/src/linux- kernelversion/.config

Next 'make oldconfig' will only prompt for new features.

make oldconfig

make dep

make clean bzImage modules modules_install

4. Customising a Kernel

Loading Kernel modules

Loadable modules are inserted into the kernel at runtime using various methods.

The modprobe tool can be used to selectively insert or remove modules and their dependencies.

LinuxIT Technical Training Centre

The Linux Kernel

___________________________________________________________________

The kernel can automatically insert modules using the kmod module. This module has replaced the kerneld module.

When using kmod the kernel will use the tool listed in /proc/sys/kernel/modprobe whenever a module is needed.

Check that kmod has been selected in the source tree as a static component:

grep -i “kmod” /usr/src/linux/.config

CONFIG_KMOD=y

When making a monolithic kernel the CONFIG_MODULES option must be set to no.

The /proc/ directory

The kernel capabilities that have been selected in a default or a patched kernel are reflected in the /proc directory. We will list some of the files containing useful information:

/proc/cmdline Contains the command line passed at boot time to the kernel by the bootloader

/proc/cpuinfo CPU information is stored here

/proc/meminfo Memory statistics are written to this file

/proc/filesystems Filesystems currently supported by the kernel. Notice that by inserting a new module (e.g cramfs) this will add an entry to the file. So the file isn't a list of all filesystems supported by the kernel!

/proc/partitions The partition layout is displayed with further information such as the name, the number of block, the major/minor numbers, etc

/proc/sys/ The /proc/sys directory is the only place were files with write permission can be found (the rest of /proc is read-only). Values in this directory can be changed with the sysctl

LinuxIT Technical Training Centre

The Linux Kernel

___________________________________________________________________

utility or set in the configuration file /etc/sysctl.conf

/proc/sys/kernel/hotplug Path to the utility invoked by the kernel which implements hotplugin (used for USB devices or hotplug PCI and SCSI devices)

/proc/sys/kernel/modprobe Path to the utility invoked by the kernel to insert modules

/proc/sys/overflowgid/uid Maximum number of users on a system. The filesystem uses 16 bits for the user and group fields, so the maximum is 2^16 = 65534 which is usually mapped to the user nobody or nfsnobody more recently

/proc/modules List of currently loaded modules, same as the output of lsmod

Example: Patch the linux-2.4.22-1.2149.nptl kernel to support Extended Attributes and Posix Access Control Lists (ACL) for ext2 and ext3 filesystems.

ACLs are beyond this course. All we need to know is that they provide a greater flexibility for directory and file permissions on the filesystem allowing, for example, several groups to access resources with different permissions.

WARNING

This patch will fail on older kernel versions (e.g linux-2.4.22-1.2115.nptl )

Install the 2.4.22-1.2149.nptl kernel and point the /usr/src/linux link to the new source. Then do:

cd /usr/src/linux

LinuxIT Technical Training Centre

The Linux Kernel

___________________________________________________________________

bzcat /usr/src/ea+acl+nfsacl-2.4.22-0.8.65.patch.bz2 | patch -p1 –dry-run

If there are no error messages then run patch with no –dry-run option. Next, we compile the new kernel:

Add EXTRAVERSION=-acl to the Makefile

make mrproper

cp configs/kernel-2.4.22-i686.config .config

make oldconfig (answer y to all questions relative to ACLs)

make dep bzImage modules modules_install

Quick test: Once you have rebooted with the new kernel, add the acl option into /etc/fstab on any EXT3 filesystem

LABEL=/usr /usr ext3 defaults,acl 1 2

You can then use the setfacl to add assign permissions for different groups on the same directory.

We first create two groups eng and sales: \

groupadd eng

groupadd sales

Then add a directory called /usrNEWS:

mkdir /usr/NEWS

The getfacl is a tool that lists ACL privileges. So before we do anything lets look at the following output:

getfacl /usr/NEWS

# file: share

# owner: root

# group: root

LinuxIT Technical Training Centre

The Linux Kernel

___________________________________________________________________

user::rwx

group::r-x

other::r-x

Next add rwx permissions on NEWS for the group sales:

setfacl -m g:sales:rwx NEWS/

List the ACL privileges:

getfacl NEWS/ # file: NEWS # owner: root # group: sales user::rwx group::r-x group:sales:rwx mask::rwx other::r-x

Finally add r_x permissions for the group eng and list the permissions:

setfacl -m g:eng:r-x NEWS/

getfacl NEWS/

# file: NEWS

# owner: root

# group: sales

user::rwx

group::r-x

group:sales:rwx

group:eng:r-x

mask::rwx

other::r-x

LinuxIT Technical Training Centre

The Linux Kernel

___________________________________________________________________

The kernel patch has worked. The above tools are not in the 201 objectives.

LinuxIT Technical Training Centre

System Startup

___________________________________________________________________

System Startup

Customising the boot process involves understanding how startup script are called. The chapter also describes common problems that arise at different points during the booting process as well as some recovery techniques. Finally we focus our attention on the “initial ram disk” (or initial root device) initrd stage of the booting process. This will allow us to make decisions as to when new initial ram disks need to be made. .

1. Customising the Boot Process

Overview of init

In order to prevent processes run by users from interfering with the kernel two distinct memory areas are defined. These are referred to as “kernel space memory” and “user space memory”. The init process is the first program to run in user-space.

Init is therefore the parent of all processes. The init program's configuration file is / etc/inittab

Runlevels

Runlevels determine which processes should run together. All processes that can be started or stopped at a given runlevel are controlled by a script (called an “init script” or an “rc script”) in /etc/rc.d/init.d

List of rc scripts on a typical system

anacron halt kudzu ntpd rusersd syslog ypxfrd

apmd identd lpd portmap rwalld vncserver

atd ipchains netfs radvd rwhod xfs

autofs iptables network random sendmail xinetd

crond kdcrotate nfs rawdevices single ypbind

functions keytable nfslock rhnsd snmpd yppasswdd

gpm killall nscd rstatd sshd ypserv

Selecting a process to run or be stopped in a given runlevel is done by creating symbolic links in the /etc/rc.d/rcN.d/ directory, where N is a runlevel.

LinuxIT Technical Training Centre

System Startup

___________________________________________________________________

Example 1: selecting httpd process for runlevel 3:

ln -s /etc/rc.d/init.d/httpd /etc/rc.d/rc3.d/S85httpd

Notice that the name of the link is the same as the name of the process and is preceded by an S for start and a number representing the order of execution.

Example 2: stopping httpd process for runlevel 3:

rm /etc/rc.d/rc3.d/S85httpd

ln -s /etc/rc.d/init.d/httpd /etc/rc.d/rc3.d/K15httpd

This time the name of the link starts with a K for kill to make sure the process is stopped when switching from one runlevel to another.

Starting Local scripts

We want to run a script at a given run level. Our script will be called printtotty10 and will simply print the message given as an argument to /dev/tty10.

/bin/printtotty10

#!/bin/bash

echo $1 > /dev/tty10

1. One way to have the script started at a specific run level is to add a line in /etc/inittab like

pr10:3:once:/bin/printtotty10 “Printtotty was started in inittab”

This is not always the best way to do this. What if many scripts need to be started? The inittab file would look messy.

2. We can write a custom rc-script. We follow the usage to call the script the same name as the actual tool we want to startup.

/etc/rc.d/init.d/printtotty10 #!/bin/sh

LinuxIT Technical Training Centre

System Startup

___________________________________________________________________

# chkconfig: 345 85 15 # description: This line has to be here for chkconfig to work ... \ #The script will display a message on /dev/tty10 #First source some predefined functions such as echo_success() ./etc/rc.d/init.d/functions

start() { echo -n "Starting printtotty10" /bin/printtotty10 "printtotty10 was started with an rc-script " echo_success echo }

stop() { echo -n "Stopping custom-rc" /bin/printtotty10 "The custom script has stopped" echo_success echo } case "$1" in start) start;; stop) stop;; esac exit 0

3. The prittotty10 script can be started at boot time by placing the command in / etc/rc.d/rc.local. The rc.local script is the last rc-script to be run.

Notice: When setting up a linux server as a router it is possible to switch on ip-forwarding at boot time by adding the following line to rc.local:

echo 1 > /proc/sys/net/ipv4/ip_forward

However it is better to use the sysctl mechanism to switch ip-forwarding on every time the network interface is started. This is done by adding the following line to /etc/sysctl.conf:

net.ipv4.ip_forward = 1

2. System Recovery

When a system crashes and fails to restart it is necessary to alter the normal booting

LinuxIT Technical Training Centre

System Startup

___________________________________________________________________

process. We will describe a few solutions here.

Overriding the INIT stage

This is necessary if the boot process fails due to a faulty init script. Once the kernel successfully locates the root file system it will attempt to run /sbin/init. But the kernel can be instructed to run a shell instead which will allow us to have access to the system before the services are started.

At the LILO or GRUB boot prompt add the following kernel parameter:

init=/bin/bash

At the end of the kernel boot stage you should get a bash prompt. Read-write access to the root filesystem is achieved with the following

mount /proc

mount -o remount,rw /

Errors at the end of the kernel stage

•

If the kernel can't mount the root filesystem it will print the following message:

Kernel panic: VFS: Unable to mount root fs on 03:05

The number 03 is the major number for the first IDE controller, and 05 is the 5th partition on the disk. The problem is that the kernel is missing the proper modules to access the disk.

We need to boot the system using an alternative method. The fix next involves creating a custom initrd and using it for the normal boot process.

Question: In the case above since the drive isn't a SCSI drive what could have caused the problem?

•

If the wrong root filesystem was parsed to the kernel by the boot loader (LILO or GRUB) then the INIT stage cannot start since /sbin/init will be missing

Kernel Panic: No init found. Try passing init= option to kernel

Again we need to boot the system using a different method, then edit the bootloader's

LinuxIT Technical Training Centre

System Startup

___________________________________________________________________

configuration file (telling the kernel to use another device as the root filesystem), and reboot.

In both scenarios above it isn't always necessary to use a rescue disk. In fact, it often is a case of booting with a properly configured kernel. But what happens if the we don't have the option? What if the bootloader was reconfigured with the wrong kernels using no initial root disks or trying to mount the wrong root filesystem?

This leads us to the next possible cause of booting problems.

Missconfigured Bootloaders

At this stage we need to use a rescue method to boot the system. We already know from 101 that any Linux distribution CD can be used to start a system in rescue mode. The advantage of these CDs is that they work on any Linux system.

We next describes a preventative method which can only recover a specific system. We will create a floppy rescue disk which we then use in the case of an emergency (simple!)

All we need is a floppy with a Linux kernel image that can boot, and this image must be told were the root filesystem on the hard drive is.

The following creates a bootable floppy which will launch a linux kernel image

dd if=/boot/vmlinuz of=/dev/fd0

Finally rdev is used to tell the kernel where the root filesystem is. The next command must be run on the system we wish to protect and the floppy with the kernel must be in the drive

rdev /dev/fd0 /dev/hda2

Bootloader Kernel Parameters

load_ramdisk=n

If n is 1 then load a ramdisk, the default is 0

prompt_ramdisk=n If n is 1 prompt to insert a floppy disk containing a ramdisk

Disable or limit the number of CPUs

nosmp or maxcpus=N

LinuxIT Technical Training Centre

System Startup

___________________________________________________________________

apm=off

Disable APM, sometime needed to boot from yet unsupported motherboards

init=

Defaults to /sbin/init but may also be a shell or an alternative process

root=

Set the root filesystem device (can be set with rdev*)

mem=

Assign available RAM size

vga=

Change the console video mode (can be changed with rdev*)

*The rdev manual pages say; “The rdev utility, when used other than to find a name for the current root device, is an ancient hack that works by patching a kernel image at a magic offset with magic numbers. It does not work on architectures other than i386. Its use is strongly discouraged. Use a boot loader like SysLinux or LILO instead”

Troubleshooting LILO

When installing LILO the bootloader mapper, /sbin/lilo, will backup the existing bootloader.

For example if you install LILO on a floppy, the original bootloader will be save to /boot/boot.0200

Similarly when changing the bootloader on an IDE or a SCSI disk the files will be called boot.0300 and boot.0800 respectively. The original bootloader can be restored with:

lilo -u

By default the second stage LILO is called /boot/boot.b and when it is successfully loaded it will prompt you with a “boot: ”.

Here the possible errors during the boot stage (taken from the LILO README)

• nothing

LILO is either not installed or the partition isn't active

• L The first stage loader has been loaded but the second stage has failed

• LI The second stage boot loader has loaded but was unable to execute

This could be cause if /boot/boot.b moved and /sbin/lilo wasn't rerun

LinuxIT Technical Training Centre

System Startup

___________________________________________________________________

• LIL The second stage boot loader has been started, but it can't load the descriptor table from the map file or the second stage boot loader has been loaded at an incorrect address

This could be cause if /boot/boot.b moved and /sbin/lilo wasn't rerun.

• LIL- The descriptor table is corrupt

This could be cause if /boot/map moved and /sbin/lilo wasn't rerun.

3. Customised initrd

In most cases a “customised initrd” requires running mkinitrd which will determine the kernel modules needed to support block devices and filesystems used on the root device.

The mkinitrd script

The following are methods used in the mkinitrd script to determine critical information about the root device and filesystem.

-The root filesystem type:

Using /etc/fstab the script determines which filesystem is used on the root device and the corresponding module (for example ext3 or xfs).

-Software RAID:

Using /etc/raidtab the mkinitrd script deduces the names of the raid arrays to start all the devices (even non root).

-LVM root device

Once the root device $rootdev is determined in /etc/fstab the major number is obtained from the following line:

root_major=$(/bin/ls -l $rootdev | awk '{ print $5 }')

If this corresponds to a logical volume, the logical volume commands are copied onto the ram disk.

LinuxIT Technical Training Centre

System Startup

___________________________________________________________________

The mkinitrd script will transfer all the required tools and modules to a file mounted as a loop device on a temporary directory. Once unmounted, the file is compressed and can be used as an initrd.

Example: As an example we will copy the content of an existing initrd to a new initrd and change the root filesystem type form ext3 to ext2..

1. Uncompress the current initrd cp /boot/initrd-your-kernel-version.img /tmp/initrd.img.gz

gunzip /tmp/initrd.img.gz

2. Mount the current initrd using a loop device mkdir /mnt/current

mount -o loop /tmp/initrd.img /mnt/current

3. Estimate the size needed for the new initrd: df -k /mnt/current

Filesystem 1K-blocks Used Available Use% Mounted on

/tmp/initrd.img 317 191 126 61% / mnt/current

4. Create a new image file called initrd-new.img of size 161K dd if=/dev/zero of=/tmp/initrd-new.img bs=1K count=317

5. Estimate the number of inodes needed in the current initrd: df -i /mnt/current

Filesystem Inodes IUsed IFree IUse% Mounted on

/tmp/initrd.img 48 33 15 69% /mnt/current

LinuxIT Technical Training Centre

System Startup

___________________________________________________________________

6. Create a filesystem on the file /tmp/initrd-new.img with 48 inodes mke2fs -F -m 0 -N 48 /tmp/initrd-new.img

7. Mount the file on a new directory and copy across all the files of the current initrd to the new one: mkdir /mnt/new

mount -o loop /tmp/initrd-new.img /mnt/new

(cd /mnt/current/; tar cf - .) | (cd /mnt/new; tar xf -)

8. Edit the /mnt/new/linuxrc file and delete the line where the ext3 module is inserted. Also replace the ext3 option by ext2 at the mount command.

9. Finally, unmount the /tmp/initrd-new.img then compress and rename it. gzip /tmp/initrd-new.img ; mv /tmp/initrd-new.img.gz /boot/initrd- test.img

gzip < /tmp/initrd-new.img > /boot/initrd-test.img

10. Create a new kernel entry in /etc/lilo.conf or /boot/grub/grub.conf instructing the bootloader to use the new initrd.

: grub.conf Sample

title linux (2.4.22) root (hd0,1) kernel /vmlinuz-2.4.22 ro root=LABEL=/ initrd /initrd-2.4.22.img

title broken? root (hd0,1) kernel /vmlinuz-2.4.22-1.2115.nptl ro root=LABEL=/ initrd /initrd-new.img

: lilo.conf Sample

LinuxIT Technical Training Centre

System Startup

___________________________________________________________________

image=/boot/vmlinuz-2.4.22-1.2115.nptl

initrd=/boot/initrd-2.4.22.img

read-only

label=linux

append=”root=LABEL=/” image=/boot/vmlinuz-2.4.22-1.2115.nptl

initrd=/boot/initrd-new.img

read-only

label=broken?

append=”root=LABEL=/”

LinuxIT Technical Training Centre

The Linux Filesystem

___________________________________________________________________

The Linux Filesystem

This objective covers most points seen in LPI 101. Configuring automount is a new feature where special attention has to be payed to the syntax.

1. Operating the Linux Filesystem

When adding new filesystems to the existing root filesystem the key file involved is / etc/fstab which assigns a mount point, a mount order and global options per device.

/etc/fstab options

Read only or read write

ro or rw

Do not respond to mount -a. Used for external devices CDROMs ...

noauto

Executables cannot be started from the device

noexec

Ignore SUID bit throughout the filesystem

nosuid

Special device files such as block or character devices are ignored

nodev

Do not update atimes (performance gain)

noatime

The device can be mounted only by it's owner

owner

user

Implies noexec, nosuid and nodev. A single user's name is added to mtab so that other users may not unmount the devices

users

Same as user but the device may be unmounted by any other user

Mount will also keep track of mounted operations by updating /etc/mtab. The content of this file is similar to another table held by the kernel in /proc/mounts.

Regular local filesystems

When the system boots all local filesytems are mounted from the rc.sysinit scrpt. The mount command will mount every thing in /etc/fstab that has not yet been mounted and that is not encrypted or networked:

mount -a -t nonfs,smbfs,ncpfs -O no_netdev,noloop,noencrypted

LinuxIT Technical Training Centre

The Linux Filesystem

___________________________________________________________________

When shutting down, all filesystem are unmounted by the halt script by scanning the / proc/mounts file with the help of some awk commands!

Swap Partions and SWAP files

At boot time, swap partitions are activated in /etc/rc.d/rc.sysinit

swapon -a

Similarly when the system shuts down swap is turned off in the halt rc-script:

SWAPS=`awk '! /^Filename/ { print $1 }' /proc/swaps`

[ -n "$SWAPS" ] && runcmd "Turning off swap: " swapoff $SWAPS

Example 1: Making a swap file of 10MB

dd if=/dev/zero of=/tmp/SWAPFILE bs=1k count=10240

mkswap /tmp/SWAPFILE

swapon /tmp/SWAPFILE

cat /proc/swaps

Filename Type Size Used Priority

/dev/hda6 partition 522072 39744 -1

/tmp/SWAPFILE file 10232 0 -2

Example 2: Making a swap partition of 16MB

1. Make a new partition (e.g /dev/hda16) of type swap (82) and size 16MB. Reboot

LinuxIT Technical Training Centre

The Linux Filesystem

___________________________________________________________________

2. Make a swap filesystem on the devices

mkswap /dev/hda16

3. Add the following to /etc/fstab

/dev/hda16 swap swap pri=-1 0 0

4. Make the swap partition available with swapon -a

Notice that if two swap partition are defined the kernel will automatically access them in “striped”mode, provided they have been mounted with the same priority determined by the pri= option in /etc/fstab

2. Maintaining a Linux Filesystem

This section covers a list of commands related to filesystem maintenance.

fsck - check and repair a Linux file system

Main options:

use alternative superblck

-b

check for bad blocks

-c

-f

force checking even when partition is marked clean

-p

automatic repair

-y

answer yes to all question

sync - flush filesystem buffers

Updates modified superblocks and inodes and executes delayed writes. The operating system keeps data in RAM in order to speed up operations. This may cause data to be lost in the event of a crash unless sync is executed. Sync will simply call the 'sync' system call. Another way of doing this is to use the 'ALT+sysreq+s' key combination

LinuxIT Technical Training Centre

The Linux Filesystem

___________________________________________________________________

badblocks - search a device for bad blocks

It is recommended NOT to use badblocks directly but to use the -c flag with fsck or mkfs.

Main options:

block size

-b

number of blocks tested at a time

-c

file with a list of known bad blocks, these blocks will be skipped

-i

output file, passed to mkfs

-o

mke2fs - create an ext2/3 filesyste

Main options:

blocksize

-b

number of bytes between consecutive inodes 'bytes-per-inode'

-i

number of inodes

-N

Percentage of blocks reserved for user root

-m

-c

Check for bad blocks

-l

Read bad blocks from file

-L

Set a volume LABEL

-j/-J Create journal (ext3)

-T

Optimise filesystem “type”, values are:

news largefile largefile4

one inode per 4kb block one inode per megabyte one inode per 4 megabytes

dumpe2fs - dump filesystem information

dumpe2fs prints the super block and blocks group information for the filesystem present on a device

debugfs - ext2 file system debugger

debugfs is used to test and repair an ext2 filesystem. The main options are:

open the filesystem as writeable

-w

-b

blocksize

LinuxIT Technical Training Centre

The Linux Filesystem

___________________________________________________________________

tune2fs - adjust tunable filesystem parameters on second extended filesystems

Main options:

-l

read the superblock

-L

set the device's volume LABEL

-m

change the filesystems reserved blocks for user root

-j or -J

set a journal

3. Configuring automount

Mounting can be automated using a mechanism called automount or autofs.

The /usr/sbin/automount is invoked with the rc-script /etc/init.d/autofs.

service autofs start

This script reads the configuration file /etc/auto.master also called a map. The map file defines mount points to be monitored by individual automount daemons.

Sample /etc/auto.master

/extra /etc/auto.extra

/home /etc/auto.home

/extra /home

automount

LinuxIT Technical Training Centre

The Linux Filesystem

___________________________________________________________________

When autofs is started it will invoke an instance of /usr/sbin/automount for each mount point defined in the master map /etc/auto.master. When the map file /etc/auto.master is changed it is necessary to restart autofs. For example if mount points have been deleted, then the associated automount daemon is terminated. Likewise, new daemons are started for newly defined mount points.

Multiple filesystems can be mounted on a single mount point. These filesystems as well as the mount options needed (filesystem type, read-write permissions, etc) are defined in a separate file.

Sample /etc/auto.extra

cdrom -fstype=iso9660,ro,user,exec,nodev,nosuid :/dev/cdrom

nfs -fstype=nfs,soft,intr,rsize=8192,wsize=8 192 192.168.3.100:/usr/local

/extra

cdrom

nfs

The CDROM will automatically be accessible in /extra/cdrom and the NFS share is mounted as soon as the /extra/nfs directory is accessed

NOTICE

In the above example:

The directories /extra/cdrom and /extra/nfs must not be created

New entries in /etc/auto.extra are immediately made available: adding 'new -fstype=ext3 :/dev/hda2' to the file will automatically make /extra/new available

By default a mounted device will stay mounted for 5 minutes: if we uncomment the 'cdrom' device in the map file /etc/auto.extra shortly after the CDROM has been accessed, then the device will still be available for approximately 5 minutes in / extra/cdrom

LinuxIT Technical Training Centre

Hardware and Software Configuration

___________________________________________________________________

Hardware and Software Configuration

This module will cover the configuration of components which need both kernel support and software tools.

1. Software RAID

RAID stands for “Redundant Array of Inexpensive Disks” and was originally designed to combine cheap hard disks together. RAID can either increase speed or reliability depending on the RAID level used.

RAID Levels

RAID-Linear

RAID-0 (stripe)

RAID-1 (mirror)

write 0

redundancy no

read write +

redundancy no

read write redundancy +

yes

RAID-5

RAID-4

read +

write -

redundancy yes

read +

write 0

redundancy yes

Spare Disks

LinuxIT Technical Training Centre

Hardware and Software Configuration

___________________________________________________________________

If spare disks are configured they will be used in the RAID array as soon as one of the array disks fail.

Kernel and software components

Software raid is handle by the following kernel module:

RAID0

raid0.o

RAID1

raid1.o

RAID4 or RAID5

raid5.o

The raidtools package will provide these most common tools:

query raid devices create md devices from instructions given in /etc/raidtab

/sbin/lsraid /sbin/mkraid /sbin/raidstart and raidstop

start and stop the md devices

Once a meta device has been successfully created the information can be found in

/proc/mdstats

Booting from a RAID root device (exercise)

1. Make two new partitions of the same size as the root device of type “Linux raid autodetect”.

One can make a smaller new root partition by checking the actual used space on the current root device

df -h /

Filesystem Size Used Avail Use% Mounted on

/dev/hda7 286M 71M 201M 27% /

Use fdisk to create the new partions (e.g /dev/hda14 and /dev/hda15) Reboot.

2. Configure software RAID 1 on these partitions

LinuxIT Technical Training Centre

Hardware and Software Configuration

___________________________________________________________________

/etc/raidtab

raiddev

/dev/md0

raidlevel 1

nr-raid-disks 2

nr-spare-disks 0

chunk-size 4

persistent-superblock 1

device /dev/hda14

raid-disk 0

device /dev/hda15

raid-disk 1

Use the raidtools to make the array and start it up:

mkraid /dev/md0

raidstart /dev/md0

Make an EXT2 filesystem on the new meta device and mount it on /mnt/sys:

mke2fs /dev/md0

mkdir /mnt/sys

mount /dev/md0 /mnt/sys

3. Copy all files on the current root device to the new root device:

(tar lcvf - /) | (cd /mnt/sys; tar xvf -)

The l option for tar is an instruction to stay on the same file system.

4. Prepare to reboot

The mkinitrd script will read /etc/raitab and /mnt/sys/etc/fstab to customise an initrd.

Edit /mnt/sys/etc/fstab and change the root device to /dev/md0 as well as the filesystem type to ext2.

LinuxIT Technical Training Centre

Hardware and Software Configuration

___________________________________________________________________

/mnt/sys/etc/fstab

/dev/md0 / ext2 defaults 1 1

Make the initial rootdisk and call it initrd-raid.img

Uncompress /boot/initrd-raid.img and mount it on a loop device to check that linuxrc will insert the correct modules.

Reconfigure LILO/GRUB to change the following

mkinitrd --fstab=/mnt/sys/etc/fstab /boot/initrd-raid.img $(uname -r)

lilo.conf Sample :

image=/boot/vmlinuz-2.4.22-1.2115.nptl

initrd=/boot/initrd-raid.img

read-only

root=/dev/md0

label=linux-raid

2. LVM Configuration

Logical Volume Management (LVM)

The Logical Volume Management framework allows one to group different block devices (disks, partitions, RAID arrays...) together as a single larger device, the volume group (VG).

Individual devices used to form a volume group are referred to as physical volumes (PV).

LinuxIT Technical Training Centre

Hardware and Software Configuration

___________________________________________________________________

Physical volumes once regrouped into a volume group lose their individual character. Instead the entire volume group is divided into physical extents (PE) of fixed size (4MB by default) from which logical volumes (LV) are created. A logical volume can be thought of as a partition.

Volume Group (VG)

Logical Volume

Physical Extent (PE)

/dev/hda10 /dev/hdb1 /dev/hdd1

Physical Volume (PV)

Kernel and software components

The LVM kernel module is lvm-mod.o. The software tools are installed by the lvm package which provides in particular /sbin/vgscan. This command will start the LVM environment by scanning all the volume groups and build the /etc/lvmtab as well as databases in / etc/lvmtab.d which are used by all the other LVM tools.

Main LVM tools :

PV tools:

pvcreate, pvmove, pvchange, pvdisplay, pvscan ...

VG tools:

vgcreate, vgremove, vgchange, vgdisplay, vgscan ...

LinuxIT Technical Training Centre

Hardware and Software Configuration

___________________________________________________________________

LV tools:

lvcreate, lvextend, lvreduce, lvremove, lvchange, lvscan ...

We won't need to use or know all the above tools. We will rather focus on the various LVM components (as depicted in the next diagram) and the commands needed to create these components: pvcreate,vgcreate and lvcreate.

Example:

Create a volume group called volumeA with three physical volumes ( 3 partitions in this case) and create a logical volume called lv0 of size 150MB initially.

1. Run vgscan to create the /etc/lvmtab file

2. Create three new partitions (say /dev/hda16, /dev/hda17, /dev/hda18) of 100MB each. Make sure you toggle the partition type to 8e (Linux LVM). Then reboot.

3. Prepare the physical volumes pvcreate /dev/hda16 pvcreate /dev/hda17 pvcreate /dev/hda18

4. Create a volume group called volumeA with the above physical volumes:

vgcreate volumeA /dev/hda16 /dev/hda17 /dev/hda18

This will create a directory called /dev/volumeA/. The default PE size of 4MB will be used, one can change this with the -s option.

5. Create a logical volume called lv0 of size 150MB on this volume group

lvcreate -L 150M -n lv0 volumeA

This will create the block device /dev/volume1/lv0

6. Make a filesystem on lv0 and mount it on /mnt/lvm

mkfs -t ext3 /dev/volumeA/lv0 mkdir /mnt/lvm mount /dev/volumeA/lv0 /mnt/lvm

LinuxIT Technical Training Centre

Hardware and Software Configuration

___________________________________________________________________

This wouldn't be very different from other partition types if it weren't for the possibility to change the logical volume's size at anytime. Let's first show how to reduce the existing 150MB logical volume lv0 with the esfsadm tool installed by the lvm package.

umount /mnt/lvm

e2fsadm -L 25 /dev/volumeA/lv0

NOTICE

The -L option refers to size in megabytes. The is the case with most LVM tools. The -l option can be used to specify logical extents (LE) instead. The default size of an LE is 4MB. .

The next section will show how to add a new physical volume (a disk) to a volume group and demonstrates how an existing logical volume can be made larger by including physical extents available in the volume group to itself. Once this is done the e2fsadm tool will resize the filesystem across the logical volume.

Extending the Volume Group with a RAID 0 device

So far we have:

VG = /dev/hda16 + /dev/hda17 + /dev/hda18

and we would like to add a RAID0 device to this

1. Create three more partitions (e.g /dev/hda19, /dev/hda20 and /dev/hda21) of size 50MB and of type “Linux raid autodetect” (fd) – reboot!

2. Edit /etc/mtab to add the following RAID 0 device:

raiddev /dev/md1

raid-level 0 nr-raid-disks 3

LinuxIT Technical Training Centre

Hardware and Software Configuration

___________________________________________________________________

nr-spare-disks 0 persistent-superblock 1 chunk-size 4 device /dev/hda19 raid-disk 0 device /dev/hda20 raid-disk 1 device /dev/hda21 raid-disk 2

3. Start the raid meta device:

mkraid /dev/md1

raidstart /dev/md1

4. Add this device to the Volume Group volumeA

Before adding the device to the volume group run pvscan to see which physical volumes are available. Notice that /dev/md1 is not listed.

We now prepare /dev/md1 as a PV (physical volume):

pvcreate /dev/md1

When running pvscan again the output should look like the following. Notice that /dev/md1 is now listed.

pvscan pvscan -- reading all physical volumes (this may take a while...) pvscan -- ACTIVE PV "/dev/md1" is in no VG [305.62 MB] pvscan -- ACTIVE PV "/dev/hda10" of VG "volumeA"[96 MB / 0 free] pvscan -- ACTIVE PV "/dev/hda11" of VG "volumeA"[96 MB / 0 free] pvscan -- ACTIVE PV "/dev/hda12" of VG "volumeA"[96 MB / 84 MB free]

We next add the device /dev/md1 to the volume group volumeA:

vgextend volumeA /dev/md1

pvscan -- total:4[611.46 MB] /in use:3[305.83 MB] /in no VG:1 [305.62 MB]

LinuxIT Technical Training Centre

Hardware and Software Configuration

___________________________________________________________________

At this stage the volume group has four devices:

VolumeA = /dev/hda10 + /dev/hda11 + /dev/hda12 + /dev/md1

We can take 50MB from /dev/md1 and add them to lv0 (unmount the volume first)

lvextend -L +50 /dev/volumeA/lv0 /dev/md1

The original lv0 volume had 150 megabytes. The + flag in front of the requested size has added 50MB to the logical volume, making it about 200 megabytes. But we haven't extended the filesystem across the entire logical volume yet.

The output of lvscan will show 80MB available. This corresponds to the 25 megabytes resizing done with e2fsadm on p. 21 plus the 50MB added by lvextend above

lvscan lvscan -- ACTIVE "/dev/volumeA/lv0" [80 MB] lvscan -- 1 logical volumes with 80 MB total in 1 volume group lvscan -- 1 active logical volume

The next command will extend the filesystem to 80 megabytes:

e2fsadm -L 80 /dev/volume/lv0

If you remount this volume on /mnt/lvm you can see the new available space with df.

REBOOT WARNING

The LVM tools need the lvm-mod.o module and in our case the metadevice /dev/md1. You need to create a new initrd with mkinitrd or add the following lines to a new initrd:

insmod /lib/lvm-mod.o

raidautorun /dev/md1

LinuxIT Technical Training Centre

Hardware and Software Configuration

___________________________________________________________________

REBOOT WARNING

The volume group is then activated with vgscan from the rc.sysinit script.

Booting from a logical volume root device

As with software RAID we are going to investigate some issues we need to consider when using LVM on the root device.

First make sure the volume we have created previously is mounted. If it isn't then do

mount /dev/volumeA/lv0 /mnt/lvm

Next we archive the root device in the same way as we did for RAID:

tar clvf - / | (cd /mnt/lvm/; tar xvf -)

Edit /mnt/lvm/etc/fstab and enter

/dev/volumeA/lv0 / ext2 defaults 0 1

Edit /etc/lilo.conf or /etc/grub.conf to add a new entry where the kernel points to the new root logical volume. For a 2.4.22 kernel an additional entry in /etc/grub.conf looks like this:

title lvm-root

root (hd0,1)

kernel /vmlinuz-2.4.22 ro root=LABEL=/

initrd /initrd-2.4.22-lvm.img

All we need is the initrd initrd-2.4.22-lvm.img.

Once again we will run mkinitrd with –-fstab= which we will use to make the script read our new fstab file /mnt/lvm/etc/fstab. We test this:

mkinitrd --fstab=/mnt/lvm/etc/fstab /boot/initrd-lvm.img $(uname -r)

LinuxIT Technical Training Centre

Hardware and Software Configuration

___________________________________________________________________

If we mount this initial ram disk we can see that this is going to work by looking at the linuxrc script.

linuxrc echo "Loading lvm-mod.o module" insmod /lib/lvm-mod.o echo Creating block devices mkdevices /dev echo Scanning logical volumes vgscan echo Activating logical volumes vgchange -ay

----snip---

3. CD Burners and Linux

Hardware detection

The tools available on the commandline to burn CDs assume that the CD writer is a SCSI device. However most cheaper CD burner are IDE devices and we need a ide-scsi.o module to drive the CD burner device.

If you run cdrecord with the -scanbus option you will see that the tool is looking for a SCSI device.

If the CD burner is attached as a secondary master (./dev/hdc) then the following entry in

/etc/modules.conf will enable the ide-sci module for this device :

/etc/modules.conf (from the CD-Writing HOWTO) options ide-scsi=/dev/hdb options ide-cd ignore=hdb alias scd0 sr_mod pre-install sg modprobe ide-scsi # load ide-scsi before sg pre-install sr_mod modprobe ide-scsi # load ide-scsi before sr_mod pre-install ide-scsi modprobe ide-cd # load ide-cd before ide-scsi

The device will be seen as /dev/scd0 and can be added to /etc/fstab with it's own mount point.

The following command shows that the hardware has been correctly detected:

LinuxIT Technical Training Centre

Hardware and Software Configuration

___________________________________________________________________

cdrecord -scanbus Cdrecord 2.0 (i686-pc-linux-gnu) Copyright (C) 1995-2002 J(cid:31)rg Schilling Linux sg driver version: 3.1.24 Using libscg version 'schily-0.7' cdrecord: Warning: using inofficial libscg transport code version (schily - Red Hat-scsi-linux-sg.c-1.75-RH '@(#)scsi-linux-sg.c 1.75 02/10/21 Copyright 1997 J. Schilling'). scsibus0: 0,0,0 0) 'PHILIPS ' 'CDRW48A ' 'P1.3' Removable CD-ROM 0,1,0 1) * 0,2,0 2) * 0,3,0 3) * 0,4,0 4) * 0,5,0 5) * 0,6,0 6) * 0,7,0 7) *

Iso9660 Filesystem and burning CDs

Store all the data that need to be copied in a separated directory (e.g backups/). We next need to create an isoimage of this directory as follows:

mkisofs -o baskups-image.iso backups/

Check the image file by mounting it as a loop device:

mount -o loop backups-image.iso /mnt

ls /mnt

umount /mnt

Finally, burn the CD with cdrecord. From the output of cdrecord -scanbus on the previous page we see that the CD writer device is seen as dev=0,0,0 so we type:

cdrecord -v dev=0,0,0 backups-image.iso

Copying Bootable CDs

This is useful for example when copying the first disk for a Linux distribution.

Put the bootable CD into the CDROM tray. Do not mount the disk! Then type:

LinuxIT Technical Training Centre

Hardware and Software Configuration

___________________________________________________________________

dd if=/dev/cdrom of=distro-inst1.iso

Once this is done you can update the image with rsync before burning it, this will fix data corruptions that could have been copied from the CD:

rsync -av ftp.somesite.org::/path-to-iso/distro-inst1.iso .

5. Configuring PCMCIA Devices

The cardmgr utility monitors the PCMCIA slots. It will scan the /proc/devices file searching for the pcmcia entry. If this entry isn't there then cardmgr will exit.

In order to get the kernel to write an entry into /proc/devices it is necessary to load the relevant modules. Only once kernel support is enabled will cardmgr work properly. The module names are kept in the following configuration files:

For RedHat like distributions: /etc/sysconfig/pcmcia

For Debian like distributions: /etc/pcmcia.conf

The main module is called pcmcia_core and uses two other modules called yenta_socket and ds.

One can start cardmgr on the commandline after having inserted the above kernel modules

modprobe pcmcia_core

modprobe yenta_socket

modprobe ds

cardmgr

cardmgr[18772]: watching 2 sockets

But it is best to use the rc-script provided with the pcmcia-cs package:

LinuxIT Technical Training Centre

Hardware and Software Configuration

___________________________________________________________________

/etc/rc.d/init.d/pcmcia restart

The configuration file with a database of possible devices (e.g modems, wireless network interfaces, memory cards ...) is called /etc/pcmcia/config.

To get infornation about your pcmcia card use the cardctl utility. Put the card into the pcmcia slot and run:

cardctl info

....snip....

PRODID_1="Xircom"

PRODID_2="CardBus Ethernet 10/100 + Modem 56"

PRODID_3="CBEM56G"

....snip....

We can check that this card is listed in /etc/pcmcia/config. The next table shows the information relevant to this card, in particular the xircom_cb module needed.

/etc/pcmcia/config – section relevent to scanned card

card "Xircom CBEM56G-100 CardBus 10/100 Ethernet + 56K Modem"

version "Xircom", "*", "CBEM56G"

bind "xircom_cb" to 0

LinuxIT Technical Training Centre

File and Service Sharing

___________________________________________________________________

File and Service Sharing

This module covers SAMBA and NFS. The objectives state a few specific implementations such as file servers and printer shares.

1. Samba Client Tools

nmblookup

nmblookup trainer-1

querying trainer-1 on 192.168.3.255

192.168.3.101 trainer-1<00>

smbpasswd

smbpasswd -a USER

add a samba user

smbpasswd -e USER

enable a samba user

smbtar

Script using smbclient to archive SMB shares directly to tape

smbclient

smbclient //HOST/SHARE Logs onto the specified share

smbclient -L //HOST

List all available shares

Output of smbstatus

LinuxIT Technical Training Centre

File and Service Sharing

___________________________________________________________________

Samba version 2.2.7a-security-rollup-fix

Service uid gid pid machine

----------------------------------------------

dean dean dean 3106 trainer-1 (192.168.3.101) Mon Nov 26 13:34:54 2003

IPC$ nobody nogroup 3106 trainer-1 (192.168.3.101) Mon Nov 26 13:34:45 2003

IPC$ nobody nogroup 3106 trainer-1 (192.168.3.101) Mon Nov 26 13:34:53 2003

dean dean dean 3106 trainer-1 (192.168.3.101) Mon Nov 26 13:35:14 2003

netlogon dean dean 3106 trainer-1 (192.168.3.101) Mon Nov 26 13:34:54 2003

public nobody nogroup 3145 drakelap (192.168.3.100) Mon Nov 26 13:35:34 2003

IPC$ nobody nogroup 3106 trainer-1 (192.168.3.101) Mon Nov 26 13:34:54 2003

No locked files

2. Configuring a SAMBA server

The SAMBA server configuration file smb.conf is usually in /etc/samba/. Within the '[global]' options, parameters such as the 'WORKGROUP = ' can be set.

The SAMBA server uses two daemons called nmbd and smbd implementing NMB and SMB services respectively. Both daemons are started with the single rc-script:

/etc/rc.d/init.d/smb start

The LanManager host file lmhosts

This file is usually in the same directory as the smb.conf file and is read by nmbd to resolve netBIOS hostnames. The file content is similar to /etc/hosts:

10.0.0.20 accounts

Shared Directories

We will define one share called 'readshare' which is readable and another called 'rw- share' which has read-write permissions but is only accessible for user 'tux':

The smb.conf options

LinuxIT Technical Training Centre

File and Service Sharing

___________________________________________________________________

[readshare]

comment = Read-only Directory

path = /usr/local/news/

guest only = yes

browseable = yes # this is optional

[rw-share]

comment = Read-write Share for tux

path = /usr/local/documents

browseable = yes

guest ok = yes

writeable = yes

valid users = tux

Sharing Printers

We choose to export all printers defined with CUPS on the Linux server. The following configuration will enable this:

The smb.conf options

[global]

printcap name = cups

load printers = yes

printing = cups

# printing without filters

[printers]

comment = All Printers defined using CUPS

path = /var/spool/samba

browseable = no

guest ok = yes # allow 'guest account to print'

writable = no

printable = yes

create mode = 0700 # printer drivers must be on the client side print command = lpr-cups -P %p -o raw %s -r

LinuxIT Technical Training Centre

File and Service Sharing

___________________________________________________________________

Implementing WINS with Samba?

On a NetBIOS network machine names are resolved using “Windows information network services” or WINS. Clients can either use broadcasts to query host names or be configured to use a WINS server. This server reduces the amount of traffic on the network due to broadcasts. SAMBA as a WINS server To enable WINS in SAMBA the following option is set in smb.conf

wins support = yes

Windows clients can then be configured to use the SAMBA server as a WINS server.

Second WINS server A NetBIOS network generally only has one WINS server. If a second server is configured then the servers should be able to synchronise their host information. One can configure SAMBA to register on an existing network as a second WINS server by giving it the address of this server with the option:

wins server =

NOTICE

The options 'wins support' and 'wins server' are mutually exclusive. The 'wins server' option registers the SAMBA server with an existing WINS server and enables WINS capabilities, there is no need to set 'wins support' as well.

Samba server as a Domain Controller

Options selected in /etc/samba/smb.conf:

security = users

domain master = yes

local master

preferred master = yes

domain logon = yes

LinuxIT Technical Training Centre

File and Service Sharing

___________________________________________________________________

[netlogon]

path=/var/lib/samba/netlogon

writable = no

public = no

Notice: You don't need to have a logon script. This netlogon share is something the Windows client needs to connect to even if it is empty

2. Configuring an NFS server

The NFS server runs the following daemons:

rpc.nfsd rpc.mountd

These services are started with the nfs sevice:

/etc/init.d/nfs start/stop/status/restart/reload

In addition rpc.statd is used to notify the client when the NFS service is unexpectedly interrupted, and rpc.lockd allows clients to lock files accessed on the server.

These services are started with the nfslock service:

/etc/init.d/nfslock start/stop/status/restart

Programs using remote procedure calls (RPC) use specific program numbers listed in / etc/rpc. When a RPC service is started it will tell portmap which port number it is using as well as its program number.

It is necessary for portmap to be running before starting any NFS service

RPC clients connect to the portmap service, although it is possible to work around portmap if the RPC program number is known.

The /etc/exports file

Syntax:

directory () ()

LinuxIT Technical Training Centre

File and Service Sharing

___________________________________________________________________

/etc/exports common options:

Description

Option

Read only. There is also the read-write option rw

no_root_squash override the default (root_squash) where root is mapped to user

nobody

async

the server writes to disk at predefined intervals (may cause data loss)

sync

use sync rather than async when exporting a directory read-write

User Mappings

Once a remote directory is mounted on the local client one would expect local users to access their files as if the directory was locally mounted. However this will only be the case if UIDs on both the local and remote systems correspond.

Client

Server

root=0 tux=500 penguin=600

NFS is generally used in an environment where UIDs are common between the server and the clients.

Anonuid and Anongid

It is possible, using anonuid and anongid options to assign a unique anonymous UID or GID per exported directory. Users mounting that share will be given the rights of that anonymous ID on the server. For example, everybody accessing the share bellow will inherit the right of the remote user with UID=150 and GID=100

/share *(rw,anonuid=150,anongid=100) Root Squashing

By default the root user on the client system will be mapped to the user nobody on the server. This option is disabled in /etc/exports with the no_root_squash option

LinuxIT Technical Training Centre

File and Service Sharing

___________________________________________________________________

Client

Server

UID=65534 GID=65534

root UID=0 GID=0

Finally, it is possible to map all users from any client to the user nobody with the all_squash option.

TCPwrappers

The portmap tool has been compiled with libwrap giving us the option to control access through /etc/hosts.allow and /etc/hosts.deny.

strings `which portmap ` |grep hosts.allow

Using exportfs and nfsstat

The exportfs command with no arguments will show all exported directories.

exportfs options -r -u -a -o

re-read /etc/exports and export all directories listed unexport all shares (until exportfs -r is called) applies to all exports specify directories not listed in /etc/exports

The nfsstat displays statistics about NFS server and client activity. The information is read from two files:

contains information about NFS client activity

/proc/net/rpc/nfs /proc/net/rpc/nfsd contains information about the NFS server

nfsstat options -s -c -n -r -o

show only server statistics shpw only client statistics print NFS statistics only print RPC statistics only print statistics for specific utility (nfs,rpc,net,fh,rc)

LinuxIT Technical Training Centre

File and Service Sharing

___________________________________________________________________

3. Setting up an NFS Client

Mount options soft

hard

intr nolock rsize=n wsize=n

When a major timeout happens send the calling program an I/O error, rather than retry indefinitely. When a major timeout happens, report “server not responding” and continues to reconnect indefinitely unless the intr option is also specified If the first mount fails retry subsequent mounts in the background (default is fg) Allows NFS requests to be interrupted Sometimes needed with older NFS servers Set communication block sizes for read and write. The default is 1024 bytes. On a clear network the speed may be improved by setting n to 8192

ERRORS mount: RPC: Program not registered

Possible cause The remote NFS server is not running Wrong directory

mount: IP:share failed, reason given by server: Permission denied

The showmount tool can view NFS shares available on a remote host. The main options are:

showmount -a server lists client IP and directory mounted

showmount -e server lists the content of /etc/exports from the server

showmount -d server lists only the exported directories on the server

LinuxIT Technical Training Centre

System Maintenance

__________________________________________________________________

System Maintenance

This module covers the syslogd similarly to LPI 102. The added emphasis is on remote logging and name resolution. Sotware packaging is covered here to. We will see how to make our own RPM package.

1. System Logging

Stopping and Starting syslogd

The syslogd daemon is responsible for system logging. It is started as a service:

/etc/rc.d/init.d/syslogd start/stop/status/restart/condrestart

The following lines are from the syslogd rc-script:

if [ -f /etc/sysconfig/syslog ] ; then . /etc/sysconfig/syslog

The /etc/sysconfig/syslog file defines the following default variables:

SYSLOGD_OPTIONS="-m 0" KLOGD_OPTIONS="-2"

Configuration File

The configuration file is /etc/syslog.conf with the following format:

FACILITY.PRIORITY ACTION

Facilities auth, authpriv, cron,daemon, kern, lpr, mail, mark, news, security (same as auth), syslog, user, uucp and local0 to local7 Priorities debug, info, notice, warning,err, crit, alert, emerg The following are deprecated: error (same as err), warn (same as warning), panic (same as emerg)

LinuxIT Technical Training Centre

System Maintenance

__________________________________________________________________

Actions Flat file Terminal Username Host

Full path to a file, usually in /var/log/ use /dev/ttyN to output logs to if Username is logged in, send logs to the user's tty send logs to a remote host. Prepend the remote host's IP with a @ sign.

Sending logs to a remote server

A seen above the local syslogd can send logs to a remote host (say 192.168.10.33) running a syslogd. Assume we want to send all logs to this remote host, this would be the syntax:

*.* @192.168.10.33

Configuring syslogd to accept remote logs

In this case we want remote systems to send their logs to our server. The only option that needs to be added at startup is -r.

Edit /etc/sysconfig/syslog and add the -r option to the SYSLOGD_OPTIONS variable

SYSLOGD_OPTIONS="-r -m 0"

Then restart the syslog service.

Name resolution

Once a server has been setup as a remote logging server it will accept logs from hosts on the network. By default these hosts will appear with an IP address in the logs unless the hosts are listed in /etc/hosts. This is due to the fact that syslogd cannot use DNS services. In fact syslogd has not been compiled with libresolv.so, as seen below:

ldd syslogd libc.so.6 => /lib/i686/libc.so.6 (0x40024000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000)

ldd ping libresolv.so.2 => /lib/libresolv.so.2 (0x40024000) libc.so.6 => /lib/i686/libc.so.6 (0x40035000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000

LinuxIT Technical Training Centre

System Maintenance

__________________________________________________________________

2. Packaging Software

Here is an overview of the specfile and its sections

Desciption Summary Name Version Release Copyright Group Source BuildRoot

A summary of what the package provides Name of the package Package version Package release Copyright agreement under which the package is released The package group (Amusement, Documentation ...) Path to the archive containing source and files Path to the temporary (fake) root filesystem

Define a variable that can be referenced later in the SPEC file

Macros and Section %define %description Paragraph type description for the package (usually longer than Summary

%prep %setup %patch

The preparation section, includes unpacking the source archive and patching Unpack the source archive Apply patches if needed

%build

The build section, includes commands to run in the BUILD directory and execute the next commands (make, ...)

%install

The install section, includes command to copy files from the BUILD directory to the fake $RPM_BUILD_ROOT directory

%clean

Delete all files in $RPM_BUILD_ROOT

%files %doc %config

List of files in the package List which files are part of the documentation List which files are configuration files

LinuxIT Technical Training Centre

System Maintenance

__________________________________________________________________

Example: Copy fstab to /tmp/etc/fstab

We can build a simple RPM package that installs an fstab file into /tmp/etc/. The spec file will look like this:

#This is the Header section Summary: Installs a fstab file to /tmp/etc %define name tmp-fstab %define version 0.2 %define release 1 Name: %{name} Version: %{version} Release: %{release} Copyright: Freely distributable Group: Documentation Source: %{name}-%{version}.tar.gz Packager: Adrian Thomasset

#The BuildRoot directory is a temporary replacement for root (/) while the package is being built. BuildRoot: /var/tmp/rpm-%{name}/

%description This package copies a file called fstab to /tmp/etc/

%prep #The %setup macro simply opens the archived files from SOURCES into BUILD and changes #directory to it (/../../BUILD/%{name}-%{version}/ %setup

#All the work is done here: $RPM_BUILD_ROOT is a reference to the variable defined using the %BuildRoot command earlier %install

rm -rf $RPM_BUILD_ROOT mkdir -p $RPM_BUILD_ROOT/tmp/etc/ install -m 644 fstab $RPM_BUILD_ROOT/tmp/etc/fstab

%clean rm -rf $RPM_BUILD_ROOT #Define which files must be copied to the binary RPM package. The $RPM_BUILD_ROOT is #taken as the root directory %files /tmp/etc/fstab %defattr(-,adrian,adrian)

All that is left to do is to prepare the source. In this case we need to create a directory called

LinuxIT Technical Training Centre

System Maintenance

__________________________________________________________________

tmp-fstab-0.2 containing fstab. Notice that the name and the version correspond to the name and version defined in the SPEC file

mkdir tmp-fstab-0.2

cp /etc/fstab tmp-fstab-0.2/

Next we archive the directory and copy this to the SOURCES directory

tar cvzf tmp-fstab-0.2.tar.gz tmp-fstab/

cp tmp-fstab-0.2.tar.gz /path/to/SOURCES/

LinuxIT Technical Training Centre

System Automation

___________________________________________________________________

System Automation

This module covers most scripting objectives for LPI 201. You do not need to learn a new language such as perl or bash. All that is expected is to accurately describe what a script is doing. Knowing the exact syntax for a specific scripting language is not expected.

The best way to train for this is to go through a few examples. For this we will implement the suggested automated tasks in the LPI objectives.

1. Writing simple perl scripts (using modules)

The online documentation for perl is contained in the perldoc package. The man pages are split into sections. For example the perlintro section can be accessed with:

man perlintro or

perldoc perlintro

Here is a summary of this perldoc.

Perl scripts must be readable and executable. The first line of the script must point to the interpreter. For example if which perl returns /usr/bin/perl, then the first line in a script should be: #!/usr/bin/perl

There are three variable types which can be declared and referenced as in the following script:

# Scalars my $VARIABLE = “value”; #declare VARIABLE print (“$VARIABLE \n”); #print VARIABLE

LinuxIT Technical Training Centre

System Automation

___________________________________________________________________

# Arrays my @ARRAY = (“color1”,”color2”,”color3”); # declare ARRAY $index=0 # print ARRAY while ($index < @ARRAY) {

print (“element of $index is @ARRAY[$index] \n”); $index++;

}

# Hashes or Associative Arrays ({key,value} pairs)

my %HASH=(“color1”, “blue”,”color2”, “red”, “color3”, “white”); foreach $key (keys %HASH) {

print (“The key $key corresponds to the value $HASH{$key} \n”);

}

@color_rank = sort keys %HASH; # assign the keys to an array

2. Using the Perl taint module to secure data

The taint module is used to check that external variables supplied by the user cannot be used to exploit the system. This module is automatically used when running scripts that have the setuid or setgid bit turned on. It is possible to force a perl script to switch the taint module on with the -T option.

For example the system call bellow will allow any user to read files with read access :

insecure.pl #!/usr/bin/perl $FILENAME=ARGV[0] # this is the equivalent to $1 in bash system(“/usr/bin/less”, $FILENAME);

If the script is set SUID root or if the -T option is used then the taint module will be called and this script will not execute.

check-secure.pl #!/usr/bin/perl -T $FILENAME=ARGV[0] # this is the equivalent to $1 in bash system(“/usr/bin/less”, $FILENAME);

LinuxIT Technical Training Centre

System Automation

___________________________________________________________________

In fact the check-secure.pl script isn't secure, it simply won't run with SUID root or the -T option. Here is a version of insecure.pl which works around the taint mechanism and is VERY INSECURE !!

if (open (FILE,"$FILENAME")) { $line = ; while ($line ne "") { print ($line); $line = ; } }

3. Installing Perl modules (CPAN)

Read the following perldoc pages for information on perl modules

man perlmod

A set of specific functions such as file or array manipulations can be written as modules and imported into new scripts with the directive:

use module

The modules can be downloaded from www.cpan.org and build as follows:

Unpack the archive and type

perl Makefile.pl make make test make install

This can also be done with the commandline

perl -MCPAN -e “install MODULENAME”

LinuxIT Technical Training Centre

System Automation

___________________________________________________________________

Modules are installed in subdirectories of /usr/lib/perl. One can check if a specific module is installed with:

perl -MMODULENAME -e 1

For an example application using perl modules see the Appendix.

4. Check for process execution

Searching through the output of ps for a process using grep will sometimes return a positive status even though the process is not running!

This is due to the fact that the grep process itself is sometimes printed out by ps. As in the example below:

ps au|grep junk root 13643 0.0 0.2 1724 600 pts/1 S 11:22 0:00 grep junk

Needless to say, there aren't any pre-installed tools called junk in general, so the above line would return a positive evaluation in a script!

There is a work around for this problem.

Use pgrep

This tool will search the output of ps for the PIDs of all processes that match the search criteria. For example:

ps aux | pgrep -u root httpd

will match all httpd processes run by user root. One can also use pgrep like grep with a single keyword.

LinuxIT Technical Training Centre

System Automation

___________________________________________________________________

Use |grep -v grep

By piping the output of ps into grep -v grep one can prevent grep from matching itself. This will not work however if the process you are monitoring contains the string grep.

ps aux | grep smbd | grep -v grep

5. Monitor Processes and generate alerts

This objective gives us the opportunity to use bash's control flow capabilities to make decisions when checking for the status of a given process.

Say we want to check that the smbd daemon is running, then restart it and send a message if it is stoped and do nothing if it is still running. The following script will do this:

#!/bin/bash PROCESS=smb if ps aux | grep "$PROCESS" | grep -v grep >/dev/null ; then echo Process $PROCESS is running else echo Process $PROCESS is stopped – Restarting it ... /etc/rc.d/init.d/smb start > /dev/null fi

Checking the response from a host using ping

#!/bin/bash while (true) do

#get the times from 10 ping outputs x=$(ping -c 10 $1 | cut -d"=" -f4 | tail +2|head | sed "s/ms//")

#loop through the times to check which ones are longer than 14ms for times in $x do dectimes=$(echo $times | cut -d. -f1) # get an integer if [ $(($dectimes-14)) -gt 0 ]; then

LinuxIT Technical Training Centre

System Automation

___________________________________________________________________

echo Time exceeded 14ms: $times fi done done

Schedule scripts that parse log files and email them

We can use a perl script to run last in order to read /var/run/utmp and get it to search for the string still which will match all logged users and mail the line to root.

#!/usr/bin/perl

$LOGFILE="/tmp/lastlog"; $line="0"; system("last> $LOGFILE");

open (MAIL, "| mail root");

if (open (FILE,"$LOGFILE")) { while ($line ne "") { $line=; if ($line =~ still) { print MAIL $line; } } }

close MAIL;

If this script needs to run every hour and it is called /usr/bin/last-log.pl, then you can create a symbolic link in /etc/cron.hourly pointing to it.

Monitor changed files and generate email alert

A 128-bit fingerprint (or “message-digest) for a file can be computed with md5sum.

The foillowing script will check the MD5 checksums for all the files in /etc and compare the output from each run with diff. If there are any differences the changed files are mailed to user root

LinuxIT Technical Training Centre

System Automation

___________________________________________________________________

#!/bin/bash touch /tmp/md5old touch /tmp/md5new mv /tmp/md5new /tmp/md5old

for files in $(find /etc -type f ) do md5sum $files >> /tmp/md5new done x=$(diff /tmp/md5old /tmp/md5new)

if [ -z "$x" ]; then break else echo $x |mail root fi

Notice that the first time you run this script all the files will be seen as changed!

Checking valid MD5 fingerprints can be done from the STDIN or from a list of pre-computed sums using md5sum -c (--check). We first compute these sums with

find /etc -type f | xargs md5sum > etc-md5.dat

We next pass the content of etc-md5.dat to md5sum -c.

If for example we delete a few blank lines in /etc/sysctl.conf we can see that something has changed with:

md5sum -c etc-md5.dat | grep -v OK

/etc/sysctl.conf: FAILED

md5sum: WARNING: 1 of 1906 computed checksums did NOT match

Write a script that notifies administrators when somebody logs in or out

It may not be a good idea to mail all this information but it is possible to gather it and possibly format it using XML or HTML.

Here we read from a list of users we wish to monitor /etc/checks and send an email as soon as they are logged in.

LinuxIT Technical Training Centre

System Automation

___________________________________________________________________

This can run through a cron every minute. This does imply that when somebody from the list is logged in, an email every minute would be sent!

#!/bin/bash for luser in $(cat /etc/checks) do x=$(last |grep $luser|grep still) if [ -n "$x" ]; then echo User $luser is logged in | mail root; fi done

6. Using rsync

Rsync works like an optimised rcp or scp command. It will copy to the destination directory only the files that are missing or have been changed in the source directory. Even with changed files rsync will send only the difference between the two files.

The syntaxes are:

rsync SRC HOST:/DEST

rsync HOST:/SRC DEST

One can change the value of the remote shell variable RSYNC_RSH used by rsync :

export RSYNC_RSH=ssh

Here is an example script using rsync to keep “Fedora Updates” updated on the local server:

LinuxIT Technical Training Centre

System Automation

___________________________________________________________________

#!/bin/sh

cd /var/ftp/pub/updates/fedora

( date echo echo "=== Sync Files ===" rsync -vaz --delete --delete-excluded --exclude="*/debug/*" rsync://rsync.mirror.ac.uk:873/download.fedora.redhat.com/pub/fedora/linux/core/up dates/1/ linux/core/updates/1/ 2>&1 echo "=== Sync Files Done ===" echo date ) | mail -s "Fedora Updates Sync Results" andrew@anvil.org