UNIX Concepts

COMPSCI 386

Topics

● History of C and UNIX

● The GNU Project

● Linux

● Command-Line Basics

● UNIX-Style File System

Multics Multiplexed Information and Computing Service

● Time-sharing system for mainframes (1965)

● virtual memory and other modern OS features● MIT, Bell Labs, and General Electric

● First (?) OS written in a high-level language (PL/1). Influential but not a commercial success.

● Ken Thompson and Dennis Ritchie worked on the project until Bell Labs dropped out. (Thompson at Google since 2006, where he co-invented Go.)

● They continued on their own. Kernighan and others at Bell Labs joined the effort. Designed UNIX for DEC PDP-7. Later reimplemented on the PDP-11.

UNIX

● The goal was to create a time-sharing OS with a simple UI and simple components that can be arranged and composed in a variety of ways.

● Philosophy in a nutshell: modularity.

– Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface. (McIlroy)

– The 17 Rules

● Originally written in assembly, but a high-level language would be needed for its proper development.

C

● First, Ken Thompson invented B.– a typeless, stripped-down version of BCPL

– compiler squeezed into 8K bytes of memory

● Then C was born as B-with-types.

● Easy to retarget C compiler:– made UNIX highly portable

– sparked its prominence in industry and academia

● C was standardized by ANSI in 1989.

C

● Why are almost all operating systems written in C?● Speed

– Code maps efficiently to machine instructions.● no threads, templates, nested functions, function or operator

overloading, exception handling; limited support for OOP.– Not much runtime support is needed.

● no dynamic type checking● no bounds checking or garbage collection

● Low-level access to memory (and hardware) thanks to weakly typed pointers

The GNU Project

● UNIX was proprietary, licenced to universities and corporations by A&T.

● In 1984, Richard Stallman (* * *) at the MIT AI Lab launched the GNU project to develop a free UNIX-like OS.

● He wrote gcc, emacs, and other GNU tools that are still in widespread use.

● Created the Free Software Foundation to sponsor GNU development and pursue free software ideals.

Open Source

● Free software: like free speech or free beer?– The freedom to view, copy, modify, and redistribute source

code without having to pay royalties or other fees.

● Open source movement– Best way to produce safe and reliable software?

– Enlist the cooperation of developers from around the world in a free exchange of ideas, opinions, and code.

● Stallman wrote the first copyleft licence (GNU GPL)

Linux

● GNU's kernel was never finished.● Linux released in 1991.

– The goal was to create a UNIX-like OS for PC.

– Just a kernel, but can be packaged with GNU tools and libraries for complete OS, called properly GNU/Linux.

● There are now hundreds of distributions.– Linux kernel with GNU tools and libraries, third-party

software, package manager, desktop environment.

User Interfaces

● batch, shell (CLI), GUI● Two strategies for implementing a shell:

– Interpret and execute each command. (DOS) ● If user types rm myfile.txt, shell makes necessary system calls.

– Does not interpret the command. Just invokes a system program to perform the work. (UNIX)

● If user types rm myfile.txt, the shell invokes a system program named rm with myfile.txt as a parameter.

● Two major advantages to this approach.

Bourne Shell (1977)

● /bin/sh● expressive scripting language ● can use scripts as filters● I/O redirection and pipes● globbing and command substitution● scripts invoked as commands by their filename● synchronous/asynchonous execution of commands

C Shell (1979)

● /bin/csh (or improved version: /bin/tcsh)

● scripting language with C-like syntax

● filename and command completion

● history, aliases, and other features for interactive use

Korn Shell (1983)

● /bin/ksh

● job control

● command aliasing

● command history

● better for interactive work than C shell, but had to be licensed from AT&T

Bash Shell (1989)

● /bin/bash

● Bourne-Again SHell

● Written for GNU Project

● Default shell for Linux and macOS

Command-Line Basics

● What shell am I using?

● Navigating the file system

● Viewing and finding files

● File permissions

● Copying and moving files

● Grepping

● Pipes and redirection

● Hard links and soft links

BASH Variables

● Environment variables– inherited by child processes

– /etc/bash.bashrc (for interactive non-login shells)

– SHELL, USER, HOME, PATH, PWD

● Shell variables– not inherited by child processes

– ~/.bashrc (for interactive non-login shells)

– PS1, BASH_VERSION, HISTSIZE

What Shell Am I Using?

> printenv SHELL/bin/bash

> echo $SHELL/bin/bash

> ps $$PID TTY STAT TIME COMMAND3877 pts/1 Ss 0:00 /bin/bash

> ps -p $$PID TTY TIME CMD3877 pts/1 00:00:00 bash

We can use ps to obtain info about all processes in the system.

Navigating the File System

● Listing directory contents with ls– options lapF

– man ls

– pwd

– cd

– ls .. ls ../.. ls / ls ~

● Change to parent, root, and home without specifying absolute paths.

Filesystem Hierarchy Standard

● Defines directory structure for UNIX-like systems.● Most Linux distributions are FHS-compliant.

Navigating the File System

● /bin (command binaries)

● /boot (files used by the kernel during start-up)

● /dev (device files)

● /etc (system configuration files)

● /home (home directories for individual users)

● /lib (shared libraries and kernel modules)

● /lost+found (recovered files, as after system crash)

● /media (mount point for removable media)

Navigating the File System

● /mnt (mount point for temporary file system)

● /opt (add-on software not part of default installation)

● /proc (virtual file system containing system info)

● /root (home directory for system administrator)

● /sbin (binaries for system maintenance and admin tasks)

● /tmp (temporary files)

● /usr (user-land programs and data)

● /tmp (temporary files, like lock files)

● /var (system log files)

Viewing Files

● cat /etc/shells

● cat bash_lab/episode4.txt

● more or less

Determining File Types

● file /etc/shells

● file /etc/p*

● file /bin/cat

Finding Files

Finding Files

File Permissions

- rwx r-x --x

type user group other

File Permissions

● Do a long listing after each of the following:– cd art

– chmod u-w 1*

– chmod g+w [2-4].jpg

– chmod u=rwx,go=r 5*

– chmod 751*

File Permissions

File Permissions

● Default file creation mask– umask

– umask -S

● Type umask 135 and touch a file.

● umask is a built-in command. – man umask

Copying and Moving Files

● cp /etc/shells .

● cp /etc/shells ~

● cp -r bash_lab/art ~

Try to remove art.

Grepping

● Using grep – the most common UNIX filter.

– grep womp episode4.txt

– grep “the droids” episode4.txt

– grep “ sand “ episode4.txt

– grep -c LUKE episode4.txt

– grep [A-Z]-[0-9] episode4.txt

– grep ^Eight episode4.txt

Pipes and Redirection

● A pipe connects one process to another.● Redirection connects a process to a file.● grep ^Luke episode4.txt | grep Ben

● grep droid episode4.txt | wc -l

● man less | less

● ls /etc > etc

● history > history

Pipes and Redirection

● Exercise

– Write a C program that prompts the user for two integers and displays the average. Compile and test it.

– Create a text file with input for the program.

– Run the program with input redirected to the file.

– Redirect the output to a file.

Hard Links

● Create directories tempdir1 and tempdir2.● Create a text file in tempdir1 and display contents.

Hard Links

● Make a hard link and display the link count.

● Move the link to tempdir2 and delete the text file.● Change to tempdir2.● cat indignation

● Surprised?

UNIX File System

● Directories do not contain files. That's an illusion.● A directory is a file containing a list of (name, inum) pairs.

art 1409792

episode4.txt 1391495

episode5.txt 1391768

● Each inum is an index into a table of inodes.● Do a directory listing with i and ail options.

UNIX File System

● The inode table contains inodes.

protection modeowner and group

file sizelink count

timestamps

direct blocks

indirect blocksdouble indirecttriple indirect

DATA

DATA

DATA

DATA

DATA

DATA

DATA

DATA

Note that file name not stored.

UNIX File System

struct inode {

umode_t i_mode;unsigned short i_opflags;kuid_t i_uid;kgid_t i_gid;unsigned int i_flags;

⋮

}

● /usr/src/linux-headers-4.4.0-34/include/linux/fs.h

Hard Links

● ln episode4.txt e4

art 1409792episode4.txt 1391495episode5.txt 1391768

e4 1391495

● Move link to another directory.

● Now you can find the file with -inum option.

Soft Links

● Make a soft (symbolic) link.

● Move soft link to tempdir2.● Remove original file (yoda).

● Now in tempdir2, try: cat slink.

Hard vs. Soft Links

● Which is like a Windows shortcut?● Hard links

– more space efficient

– faster access to link target

● Soft links– can link to directories

– can link across file systems