System Tweak

Finding Some System Info

Load, statistics and messages -- useful to find out what is happening in your system

# displays and updates the top cpu processes
Code:top

# displays your processors related statistics -- install via apt-get install sysstat
Code:mpstat 1

# displays your virtual memory statistics
Code:vmstat 2

# display I/O statistics (2 s intervals) -- also installed via sysstat
Code:iostat 2

# Last 500 kernel/syslog messages
Code:tail -n 500 /var/log/syslog

# Last 500 kernel/syslog messages
Code:tail -n 500 /var/log/messages

# System warnings messages see syslog.conf
Code:tail /var/log/warn

Running kernel and basic system information

# Get the kernel version -- Use uname --help for variations of this command
Code:uname -a

# Show how long the system has been running and CPU load
Code:uptime

# system's host name
Code:hostname

# Display the IP address of the host
Code:hostname -i

# Description of the file system hierarchy
Code:man hier

# Show system reboot history
Code:last reboot

# Full release info of any LSB distribution
Code:lsb_release -a

# Get Debian version (yes ubuntu gives one, it is based on debian)
Code:cat /etc/debian_version

Information about your Hardware

Kernel detected hardware

# Detected hardware and boot messages
Code:dmesg

# information about installed hardware
Code:lsdev

# Read the BIOS
Code:dd if=/dev/mem bs=1k skip=768 count=256 2>/dev/null | strings -n 8

# Shows PCI devices
Code:lspci

# Shows USB devices
Code:lsusb

# Shows DMI/SMBIOS info (hardware information from the BIOS)
Code:dmidecode

# Show a list of all devices with their properties
Code:lshal

# Shows a list of connected hardware
Code:lshw

# Information about your CPU
Code:nano -w /proc/cpuinfo

# Info about your hardware memory
Code:nano -w /proc/meminfo

# Displays your physical memory
Code:grep MemTotal /proc/meminfo

# Watch changeable interrupts continuously -- Watch -n1 is very useful to watch any changing file
Code:watch -n1 'nano -w /proc/interrupts'

# Used and free memory (-m for MB)
Code:free -m

# Configured devices
Code:nano -w /proc/devices

Commands for Users and Controlling them

# Shows the active user id with login and group
Code:id

# Shows last logins on the system
Code:last

# Shows who is logged on the system
Code:who

# Shows last boot time
Code:who -b

# Add group "admin"
Code:groupadd admin

# Adds user Joe Bloggs to group admin
Code:useradd -c "Joe Bloggs" -g admin -m joe

# Add existing user to group
Code:usermod -a -G <group> <user>

# Delete user joe
Code:userdel Joe

To temporarily prevent logins system wide (for all users but root) use nologin.
The message in nologin will be displayed (might not work with ssh pre-shared keys).

# Replace Sorry no login now for the message you would like to display
Code:echo "Sorry no login now" > /etc/nologin

Limits

Some applications require higher limits on open files and sockets. The default limits are sometimes too low.

Per shell/script

The shell limits are governed by ulimit. The status is checked
with ulimit -a. For example to change the open files limit from
1024 to 10240 do:
Code:ulimit -n 10240

The ulimit command can be used in a script to change the limits for the script only.

Per user/process

Login users and applications can be configured in /etc/security/limits.conf. For example:

Code:nano -w /etc/security/limits.conf

# Limit user processes
Code:*   hard    nproc   250

# Limit application open files
Code:asterisk hard nofile 409600

System wide limits

Kernel limits are set with sysctl. Permanent limits are set in /etc/sysctl.conf.

# View all system limits
Code:sysctl -a

# View max open files limit
Code:sysctl fs.file-max

# Change max open files limit
Code:sysctl fs.file-max=102400

# port range
Code:echo "1024 50000" > /proc/sys/net/ipv4/ip_local_port_range

Code:nano -w /etc/sysctl.conf

# Permanent entry in sysctl.conf
Code:fs.file-max=10240

# How many file descriptors are in use
Code:nano -w /proc/sys/fs/file-nr

Runlevels


Once booted, the kernel starts init which then starts rc which starts all scripts belonging to a runlevel.
The scripts are stored in /etc/init.d and are linked into /etc/rc.d/rc#.d with # the runlevel number.

The actual runlevel can be changed with init. For example to go from 3 to 5:

# Enters runlevel 5
Code:init 5

Runlevel list

* 0 Shutdown and halt

* 1 Single-User mode (also S)

* 2 Multi-user without network

* 3 Multi-user with network

* 5 Multi-user with X

* 6 Reboot

The above is for what most linux OSs do, however, debian acts slightly differently:

* 0 System Halt

* 1 Single User

* 2 Full multi-user mode (Default)

* 3-5 Same as 2

* 6 System Reboot

Debian and systems based on Debain, use the command update-rc.d to manage runlevels scripts.
Default is to start in 2, 3, 4 and 5 and shutdown in 0, 1 and 6.

in the following examples, i will use sshd (the ssh server)

# Activates samba in the default runlevel
Code:update-rc.d sshd defaults

# For use with explicit arguments
Code:update-rc.d sshd start 20 2 3 4 5 . stop 20 0 1 6 .

# Disable sshd for all runlevels
Code:update-rc.d -f sshd remove

# Shuts down and halts the system -- 'now' can be replace with a number # (# minutes from now)
Code:shutdown -h now

Also, debian tends to want to use a tool called 'telinit' to switch init modes.
i.e. #switches to runlevel 3
Code:telinit 3

Also, take caution. Whilst you can issue telinit 6 to issue a shutdown... it really is not recommended.


Resetting your root password

EDIT: Has been removed due to this forum policy: http://ubuntuforums.org/showthread.php?t=765414 (thanks Rocket2DMn)

Kernel

Kernel modules

# Lists all modules loaded into the kernel
Code:lsmod

# To load a module -- I used ndiswrapper here
Code:modprobe ndiswrapper

# To unload a module
Code:rmmod nidiswrapper

Compiling your Kernel

# Changes Directory to the current kernel source -- verify the directory with ls -l, and create if necessary with ln -s
Code:cd /usr/src/linux

# Cleans everything, including config files
Code:make mrproper

# Reuses the old .config if it exists
Code:make oldconfig

# or xconfig or gconfig, depending on personal preference
Code:make menuconfig

# Creates a compressed kernel image
Code:make

# Compiles the modules
Code:make modules

# Installs the modules
Code:make modules_install

# Installs the kernel
Code:make install

Listing and manipulating processes

Listing and manipulating processes

Each process has a unique number, a PID. A list of all running processes is retrieved with ps.

# List of all running processes
Code:ps -A

# An Extensive list of all running processes
Code:ps -auxefw

Many handy uses of ps involve a pipe (|) and grep:

# Looks for a process with cron -- may not actually be running, use ps -A for running commands.
Code:ps axww | grep cron

# an example output -- in this case, cron is not running
Code:15087 pts/2    R+     0:00 grep --colour=auto cron

# Finds all ssh PIDs without the grep pid
Code:ps aux | grep 'ss[h]'

# Find the PIDs of processes by (part of) name
Code:pgrep -l sshd

# Echos the PID of your shell
Code:echo $$

# List processes using port 22 on TCP
Code:fuser -va 22/tcp

# List processes accessing the /home partiton
Code:fuser -va /home

# Trace system calls and signals
Code:strace df

# Display the last 50 used commands
Code:history | tail -50

Priorities

Change the priority of a running process with renice. Negative numbers have a higher priority,
the lowest is -20 and "nice" have a positive value.

# Stronger priority on PID 1070
Code:renice -5 1070

# Output would look something like the following:
Code:1070: old priority 0, new priority -5

Start the process with a defined priority with nice. Positive is "nice" or weak,
negative is strong scheduling priority. Make sure you know if /usr/bin/nice or
the shell built-in is used (check with which nice).

# Stronger priority (/usr/bin/nice)
Code:nice -n -5 top

# Weaker priority (/usr/bin/nice)
Code:nice -n 5 top

# tcsh builtin nice (same as above)
Code:nice +5 top

While nice changes the CPU scheduler, an other useful command ionice will schedule the disk IO.
This is very useful for intensive IO application which can bring a machine to its knees while
still in a lower priority. You can select a class (idle - best effort - real time),
the man page is short and well explained.

# set idle class for pid 123
Code:ionice c3 -p123

# Run firefox with best effort and high priority
Code:ionice -c2 -n0 firefox

# Set the actual shell to idle priority
Code:ionice -c3 -p$$

For example, the last command is very useful to compile or debug a large project.
Every command launched from this shell will have a lower priority and will not disturb the system.
$$ is your shell pid (use echo $$ to see the PID of your shell.).

Background/Foreground

When started from a shell, processes can be brought in the background and back to the foreground with <Ctrl>-<Z> (^Z),
bg and fg. For example start two processes, bring them in the background, list the processes with jobs and bring one
in the foreground.
Code:ping google.com > ping.log

# ping is suspended (i.e stopped) with <Ctrl>-<Z>
Code:^Z

# put in background and continues running
Code:bg

# List processes in background
Code:jobs -l

# example output:
Code:

[1]  - 36232 Running                       ping google.com > ping.log
[2]  + 36233 Suspended (tty output)        less /var/log/messages

# Bring process number 2 back in foreground
Code:fg %2

Use nohup to start a process which has to keep running when the shell is closed (it is immune to hangups).

# Also needs the '&' to sub-task the ping
Code:nohup ping -i 60 > ping.log &

Using top effectively

The program top displays running information of processes.

# runs top from the terminal
Code:top

While top is running press the key h for a help overview. Useful keys are:


* u [user name] To display only the processes belonging to the user. Use + or blank to see all users

* k [pid] Kill the process with pid.

* 1 To display all processors statistics (Linux only)

* R Toggle normal/reverse sort.


Signals/Kill

Terminate or send a signal with kill or killall. Kill kills something with a PID, killall with the process name, see
man kill and man killall for more info.

Code:ping -i 60 google.com > ping.log &

#PID of the ping process
Code:[1] 4712

# same as kill -15 4712
Code:kill -s TERM 4712

# Kill HUP processes by exact name
Code:killall -1 httpd

# Kill TERM processes by (part of) name
Code:pkill -9 http

# Kill TERM processes owned by www
Code:pkill -TERM -u www

# Kill every process accessing /home (to umount)
Code:fuser -k -TERM -m /home

Important signals are (use anything above signal 3 with caution):


* 1 HUP (hang up)

* 2 INT (interrupt)

* 3 QUIT (quit)

* 9 KILL (non-catchable, non-ignorable kill)

* 15 TERM (software termination signal)

File Systems
Permissions

Change permission and ownership with chmod and chown.
The default umask can be changed for all users in /etc/profile.
The default umask is usually 022. The umask is subtracted from 777,
thus umask 022 results in a permission 0f 755:

1 --x execute # Mode 764 = exec/read/write | read/write | read
2 -w- write # For: |-- Owner --| |- Group-| |Oth|
4 r-- read
ugo=a u=user, g=group, o=others, a=everyone


# MODE is of the form [ugoa]*([-+=]([rwxXst]))
Code:chmod [OPTION] MODE[,MODE] FILE

# Restrict some-log -rw-r-----
Code:chmod 640 /var/log/some-log

# Same as above
Code:chmod u=rw,g=r,o= /var/log/some-log

# Recursive remove other readable for all users
Code:chmod -R o-r /home/*

# Set SUID bit on executable (use with caution)
Code:chmod u+s /path/to/app

# Find all programs with the SUID bit
Code:find / -perm -u+s -print

# Change the user and group ownership of a file
Code:chown user:group /path/to/file

# Change the group ownership of a file
Code:chgrp group /path/to/file

# Change permissions to 640 for all files
Code:chmod 640 `find ./ -type f -print`

# Change permissions to 751 for all directories
Code:chmod 751 `find ./ -type d -print`

Disk information

# information about the IDE/ATA disk
Code:hdparm -I /dev/sda

# Display and manipulate the partition table
Code:fdisk /dev/sda

# Display the disk SMART info (must have a SMART compatible disk) -- install with apt-get install smartmontools
Code:smartctl -a /dev/sda

System mount points

# Show mounted file-systems on the system
Code:mount | column -t

# display free disk space and mounted devices (use -h to show in human readable format)
Code:df

# Show all registered partitions
Code:nano -w /proc/partitions

Disk usage

# Directory sizes as listing
Code:du -sh *

# Total directory size of the current directory
Code:du -csh

# Sort everything by size in kilobytes
Code:du -ks * | sort -n -r

# Show files, biggest last
Code:ls -lSr

Find out who has which files opened

This is useful to find out which file is blocking a partition which has to be unmounted and gives a typical error of:

# umount impossible because a file is locking home
Code:umount /home/
umount: unmount of /home
failed:    Device busy

A lot of linux OSs can use fstat for this sort of thing, ubuntu appears not to have it, i will look for another tool for this


Find opened log file (or other opened files), say for your Xorg log file:

Code:ps ax | grep Xorg | awk '{print $1}'

#example output:
Code:1252

The file with inum 212042 is the only file in /var:
Code:find -x /var -inum 212042

# example ouptut:
Code:/var/log/Xorg.0.log

Find opened files on a mount point with fuser or lsof:

# List processes accessing /home
Code:fuser -m /home

Code:lsof /home

#example output:
COMMAND   PID    USER   FD   TYPE DEVICE    SIZE     NODE NAME
tcsh    29029 joe  cwd    DIR   0,18   12288  1048587 /home/joe (guam:/home)
lsof    29140 joe  cwd    DIR   0,18   12288  1048587 /home/joe (guam:/home)

About an application:

Code:ps ax | grep Xorg | awk '{print $1}'

# example output:
Code:3324

Code:lsof -p 3324

#example output
COMMAND   PID    USER   FD   TYPE DEVICE    SIZE    NODE NAME
Xorg    3324 root    0w   REG        8,6   56296      12492 /var/log/Xorg.0.log

About a single file:
Code:lsof /var/log/Xorg.0.log

# example output
COMMAND  PID USER   FD   TYPE DEVICE  SIZE  NODE NAME
Xorg    3324 root    0w   REG    8,6 56296 12492 /var/log/Xorg.0.log

Mount/remount a file system

For example, mounting your cdrom drive. If it is listed in /etc/fstab:
Code:mount /dev/cdrom

if not

# typical ubuntu cdrom mount command
Code:mount -t auto /dev/cdrom /media/cdrom

# typical IDE
Code:mount /dev/hdc -t iso9660 -r /media/cdrom

# typical SCSI cdrom
Code:mount /dev/scd0 -t iso9660 -r /media/cdrom

# typical SCSI -- must have ntfs-3g tool installed either install with apt-get
Code:mount /dev/sdc0 -t ntfs-3g /media/windows

# or you could swap ntfs-3g with ntfs if you have the ntfs kernel module installed.
Code:mount /dev/sdc0 -t ntfs /media/windows

Entry in /etc/fstab:

Code:/dev/cdrom   /media/cdrom  auto noauto,fs=cdfss,ro,procuid,nosuid,nodev,exec 0 0

Remount

Remount a device without unmounting it. Necessary for fsck for example

Code:mount -o remount,ro /

Copy the raw data from a cdrom into an iso image:
Code:dd if=/dev/cdrom of=file.iso

Add swap on-the-fly

Suppose you need more swap (right now), say a 2GB file /swap2gb.

Code:dd if=/dev/zero of=/swap2gb bs=1024k count=2000

# create the swap area
Code:mkswap /swap2gb

# activate the swap.
Code:swapon /swap2gb

# when done deactivate the swap
Code:swapoff /swap2gb

# remove the swap
Code:rm /swap2gb

Mount a samba share

Suppose we want to access the SMB share myshare on the computer smbserver,
the address as typed on a Windows PC is \\smbserver\myshare\.
We mount on /mnt/smbshare. Warning> cifs wants an IP or DNS name, not a Windows name.

# List the shares
Code:smbclient -U user -I 192.168.16.229 -L //smbshare/

Code:mount -t smbfs -o username=winuser //smbserver/myshare /mnt/smbshare

Code:mount -t cifs -o username=winuser,password=winpwd //192.168.16.229/myshare /mnt/share

Additionally with the package mount.cifs it is possible to store the credentials
in a file, for example /home/user/.smb:

username=winuser
password=winpwd


And mount as follow:

Code:mount -t cifs -o credentials=/home/user/.smb //192.168.16.229/myshare /mnt/smbshare

Mount an image

# Mount a CD image
Code:mount -t iso9660 -o loop file.iso /mnt/image

# Mount an image with ext3 fs[code]mount -t ext3 -o loop file.img /mnt/image [/code]Create and burn an ISO image

This will copy the cd or DVD sector for sector. Without conv=notrunc,
the image will be smaller if there is less content on the cd.
See below and the dd examples.

Code:dd if=/dev/cdrom of=/tmp/mycd.iso bs=2048 conv=notrunc

Use mkisofs to create a CD/DVD image from files in a directory.
To overcome the file names restrictions: -r enables the Rock Ridge
extensions common to UNIX systems, -J enables Joliet extensions used
by Microsoft systems. -L allows ISO9660 filenames to begin with a period.

Code:mkisofs -J -L -r -V TITLE -o imagefile.iso /path/to/dir

Also use cdrecord with Linux as described above. Additionally it is possible to
use the native ATAPI interface which is found with:

Code:cdrecord dev=ATAPI -scanbus

And burn the CD/DVD as above.
dvd+rw-tools

The dvd+rw-tools package can do it all and includes growisofs to burn CDs or DVDs.
The examples refere to the dvd device as /dev/dvd which could be a symlink to /dev/scd0
(typical scsi on Linux), although you will probably want /dev/cdrom.

# -dvd-compat closes the disk
# Burn existing iso image

Code:growisofs -dvd-compat -Z /dev/dvd=imagefile.iso

# Burn directly
Code:growisofs -dvd-compat -Z /dev/dvd -J -R /p/to/data

Convert a Nero .nrg file to .iso

Nero simply adds a 300Kb header to a normal iso image. This can be removed with dd.

# Backup .nrg file first before doing this just in-case
Code:dd bs=1k if=imagefile.nrg of=imagefile.iso skip=300

Convert a bin/cue image to .iso

The little bchunk programhttp://freshmeat.net/projects/bchunk/ can do this.
Can install from the ubuntu repos with sudo apt-get install bchunk

Code:bchunk imagefile.bin imagefile.cue imagefile.iso

Create a file based image

Code:dd if=/dev/zero of=/usr/vdisk.img bs=1024k count=1024

Code:mkfs.ext3 /usr/vdisk.img

Code:mount -o loop /usr/vdisk.img /mnt/mountpoint

# Cleanup
Code:umount /mnt; rm /usr/vdisk.img

Linux with losetup

/dev/zero is much faster than urandom, but less secure for encryption.

Code:dd if=/dev/urandom of=/usr/vdisk.img bs=1024k count=1024

# Creates and associates /dev/loop0
Code:losetup /dev/loop0 /usr/vdisk.img

Code:mkfs.ext3 /dev/loop0

Code:mount /dev/loop0 /mnt

# Check used loops
Code:losetup -a

Code:umount /mnt

# Detach
Code:losetup -d /dev/loop0

Code:rm /usr/vdisk.img

Create a memory file system

A memory based file system is very fast for heavy IO application. How to create a 64 MB partition mounted on /memdisk:


Code:mount -t tmpfs -osize=64m tmpfs /memdisk

Disk performance

Read and write a 1 GB file on partition sda2 (/home)
Code:time dd if=/dev/sda2 of=/dev/null bs=1024k count=1000

Code:time dd if=/dev/zero bs=1024k count=1000 of=/home/1Gb.file

Code:hdparm -tT /dev/hda