shell scriptingnavigating shellmv [options] [source] [destination]-either renames source to destination if destination if a file-or moves source to destination if destination is a foldercp [options] [source] [destination]-the same except it makes copiesls [options] [directory]-if no directory specified, used working directory- -l for long listing, -a for showing hidden files/folders, -t for sorting by timewhen a script is executed, a child shell is spawned for the process

first line states which shell to use#! /bin/sh, or #! /bin/bashthese can in fact point to any program; all it does is invoke the program listed with any flags you pass to it, and then appends the script filepath to the argument list

basicsshell variables must start with _ or a-z or A-Z, and can contain _ and alphanumeric-_myvar999="value"-mynum=2dereference using $ (and curly braces optionally)-$_myvar999, ${mynum}command line arguments are dereferenced with $digit-echo the first arg is $1-echo the tenth arg is ${10}-any double digit number must be enclosed in brackets

comment with #

globbing (don't confuse with regex)* = any amount of characters? = one of any characters[abc] = one of those characters abc[^abc] = not one of those characters abc

` ` (backticks, expand as shell command, called command subtitution, same thing as $( ))' ' (single quotes, do not expand at all)" " (double quotes, only expand backticks, $, and \)

&& will run the second command if the first one succeeds (if first one returns 0)|| will run the second command if the first one fails (if first one doesn't return 0)

special variables$# (num args given to current process)$@ (command-line args to current process, in " ", expands to individual arguments)$* (same as above, but in " ", expands to single arguments)$- (options given to shell on invocation)$? (exit status of previous command)$$ (PID of shell process)$0 (name of shell program, in other words the path invoked to reach the script)$HOME$IFS (defaults to tab, space, newline)$PATH$PWD (print working directory)$LINENO (line number of script that just ran, as in the one that LINENO is on)$PS1 (primary command prompt string, default $)$PS2 (prompt string for line continuation, default >)$PS4 (prompt string for execution tracing with set +x, default +)

arraysarray[10]=0 # initializes first value, or:set array[10] # or:array=(one two three)# or:declare -a arraydelete an array like this:unset arraydereference with ${array[2]}get all elements with ${array[@]}, or ${array[*]}${#array[@]}# gives you length of array, which means how many non-null elements there are # you can use this on ordinary variables (just exclude the brackets) to get # the number of characters in a string${array[@]:0:2}# gives you 2 elements starting from 0

arithmeticn=2n=`expr $n + 1`# backtick evaluation# if you're going to use multiplication, use '*' to avoid globbinglet n=n+1# no spaces allowed unless you put quotes around "n = n + 1"$((n+1))# arithmetic expansion((n += 1))# carries out the operation, but if you ever want to use this return value you# must prefix it with $

control structuresif [ 5 -gt 1 ]; thenecho "5 is greater than 1"elif [ 0 -gt 1]; thenecho "certainly not possible"elseecho "not possible"fi

if test 5 -gt 1; thenecho "5 is greater than 1"fi

case "$variable" ina ) echo "letter a";; # don't forget the double semi-colon[0-9] ) echo "digit";;[[:upper:]] ) echo "uppercase";;esac

COUNT=6while [ $COUNT -gt 0 ]; doecho "Value of count is: $COUNT"let COUNT=COUNT-1done

until [ $COUNT -eq 5]; do # repeat while condition is falseecho "Value of count is : $COUNT"let COUNT=COUNT+1done

# when parsing a string for tokens, they are separated by the $IFSfor i in `ls *.txt`; doecho "$i"done

# writes out each word produced by lstemp=`ls`for f in $temp; doecho $fdone

boolean expressionstest/[ ] operations:strings:- s1 = s2 (true if equal)- s1 != s2 (true if not equal)- -n s1 (true if not null)- -z s1 (true if null)arithmetic:- exp1 -eq exp2 (==)- exp1 -ne exp2 (!=)- exp1 -gt exp2 (>)- exp1 -ge exp2 (>=)- exp1 -lt exp2 (write stdout to file>append stedout to file2>write stderr to file2>>append stderr to file

regexmetacharacters:\ = either turns off special meaning of next character or activate a special meaning^ = match if succeeding expression is at the beginning of line, eg. ^apple$ = match if preceding expression is at the end of line, eg. apple$* = matches 0 or more of the preceding expression+ = matches 1 or more of the preceding expression? = matches 0 or 1 of the preceding expression\{i\} = matches exactly i (between 0 and 255) of the preceding expression-eg. [ab]\{5\} looks for 5 as and bs in some order\{i,j\} = matches between i and j, inclusive, of the preceding expression\{i,\} = matches i or more of the preceding expressionall of the six above are greedy; they will match as many as possible| = matches either the expression before or after. = matches any character, including newline[list] = matches any character in list-[aeiou] for all vowels, [a-z] for all lowercase characters-[^0-9] for all characters except numbers (^ in lists is the inverse operator)-to include ], make it the first character (after ^ if necessary)-to include -, make it the first or last character-to include ^, make it after the first character-every other character is treated literallyPOSIX bracket expressions include (it should look like [[:alnum:]0-5]):[:alnum:], [:alpha:], [:blank:] (space and tab), [:cntrl:] (control character), [:digit:], [:graph:] (nonspace characters), [:lower:], [:print:] (printable characters), [:punct:], [:space:] (whitespace characters), [:upper:], [:xdigit:] (hex digits)backreference with \( and \)-anything enclosed can be retrieved with \1 through \9, in order of occurrence

in basic regex, ? + { } | ( ) must be escaped for special meaningin extended regex, they must be escaped for no meaning

in basic regex, ^ is special only at the beginning (same line of thought applies to $)in extended regex, ^ is special everywhere

in basic regex, * is not special if it's at the beginning

chmodsymbolic mode: chmod [ugoa][+-=][rwxXst] file-u = owner, g = group, o = others, a = all-+ = adds to existing bits, - = removes from existing bits, = = adds existing bits and deletes unmentioned ones (basically it sets it)-r = read, w = write, x = execute, X = execute/search only if directory or some user,s = set user/group id bitalready has execute permissions, s = set user/group id on execution, t = restricted deletion flag or sticky bit-eg. chmod a-rw myfile-deletes read and write permissions from all 3 groupsnumeric mode: chmod xxxx file-one to four octal digits (0 to 7), omitted digits are assumed to be leading zeroes-digit 1: set user id (4), set group id (2), restricted deletion flag/sticky bit (1)-digit 2 - 4: read (4), write (2), execute (1)-digit 2: owner, digit 3: group, digit 4: othersuser/group id bit means that any user gains the permissions associated with the user group, often used temporarily to "elevate" a normal user's permissions

cmpcmp [options] file1 [file2]-file can be substituted with - for stdin-exit status is 0 if bytes are equal in files, 1 if different, 2 for errors

diffdiff [options] [files]-first file is original, second file is modified- -u = unified, prints 3 lines of unified context--- path/to/original_file ''timestamp''+++ path/to/modified_file ''timestamp''@@ -line,numLinesInHunk +line,numLinesInHunk @@-old stuff+old stuffunchanged stuff-in the "hunk header" line, the sign refers to the original and modified-should refer to the same hunk-first number is the line number the hunk begins on-second number is the number of lines in that hunk (considering the changes)

findfind [options] [paths] [expression]-expression is made up of operators-numeric arguments specified with +n, -n, n for greater than, less than, or equal to- -amin nlast accessed n minutes ago- -atime nlast accessed n days ago (fraction ignored, +1 searches for 2 days or older)- -cmin n, -ctime n, -mmin n, -mtime n, change/modified versions of above- -empty empty and is either regular file or directory- -exectuableexecutable files and searchable directories- -name patternpattern is a shell pattern- -iname patterncase insensitive version- -perm modefile's permissions are exactly mode (octal or symbolic)- -perm -modeall the bits set in mode are set in file- -perm /modeany of the bits set in mode are set in file- -readable- -regex patternpattern is a regex pattern- -size n[cwbkMG]file is n bytes, b = 512-byte blocks (default), c = bytes, w = two-word bytes, k = kilobytes, M = megabytes, G = Gigabytes- -type [bcdpflsD]b = block special, c = character special, d = directory, p = named pipe,f = regular file, l = symbolic link, s = socket, D = door (Solaris)- -writeable

grepprints all lines matching PATTERN, which is written in regexgrep [options] PATTERN [file]-if no files specified or - is the filename, reads from stdin- -E for extended regex (or use egrep)- -v/--invert-match to print non-matching lines- -F for fast grep (no regex) (or use fgrep)

head/tailhead [options] [files]-by default, prints first 10 lines of each file to stdout- -c=[-]Kprints first K bytes, with leading -, prints all but last K bytes- -n=[-]Ksame as above but with linestail is the same, but you specify +K for every line after the Kth line

linkshard links point to physical data, symbolic links point to a file name-if you change the original file name, symbolic link will no longer function, but ahard link will work just fineln [options] [target] [link name]-defaults to hard link- -s/--symbolic for symbolic links- -f/--force to remove existing destination files (link name)

localeLC_COLLATE = sorting orderLC_TIME = date formatLC_CTYPE = character class determination for pattern matchingLC_MESSAGES = current language for output messagesamong other things: LC_CTYPE, LC_NUMERIC, LC_MONETARY, LC_MESSAGES, LC_PAPER, LC_NAME, LC_ADDRESS, LC_TELEPHONE, LC_MEASUREMENT, LC_IDENTIFICATION, LC_ALLsedstream editor; does transformations on an input stream in one passs/// command is a regex command of interestsed [options] 's/regexp/replacement/flag' [input files]-eg. sed 's/:.*//' myFile, prints out each line with everything after a colon deleted-g flag will apply replacements to all matches, not just the first-a number flag will apply the replacement only to the nth occurrence- -r will use extended regex instead of basic regexsed [options] '/regexp/d'-deletes any lines matching regexp

sortsort [options] [file]-write sorted concatenation of all files to stdout- -u = w/o -c, output only the first of an equal run (unique values only)

stracestrace [options] [command]-prints out system calls to stderr or to output filename

timetime [options] [command]-real: total time elapsed in real life-user: total time spent executing in user space-sys: total time spent executing in kernel space

touchtouch [options] [files]-updates access and modification time to current time

trtr [options] set1 [set2]-translate, squeeze, or delete characters from stdin and print to stdout- -c = complement of set1- -d = delete characters in set1, do not translate- -s = squeeze repeats into a single occurrence- -t = first truncate set1 to length of set2

wcwc [options] [files]-prints newline, word, and byte count for each file, and total lines if more than 1-no file or - as file means read from stdin- -c, bytes only, -m, chars only, -l, newline only, -L, length of longest line, -w, word only

Makefile# are comments# with no args, first rule is called# this entire thing is a ruletarget: prerequisitescommandswhen calling make, you can specify a specific rule as an argumentif you're compiling, know that -c will tell gcc not to link but just to make an object file-o will tell it what file to output todeclare variables like this:CFLAGS=-g -I. -std=c++0x -cuse variables like:$(CFLAGS)./configure checks details of machines before installation (like package dependencies) and then generates the Makefilemake runs Makefile, which will compile all code and create executables in current directorymake install copies executables to system directories

Pythonobject-oriented scripting languagecompiled to bytecode and interpreted

basicsbackticks converts an object to a string representation, so for example:val = 1print `val`# prints "1"

strings are immutable

everything in Python can be treated as a logical operator-[], 0, "", None are all examples of false-most other values are true-you can use short circuit logic to easily return value

control structures (for loops under data structures)if x < 0:...elif x == 0:...else:...

while x > 0:...

try:f = open('myfile.txt')s = f.readline()except IOError as e:print "I/O Error({0}): {1}".format(e.errno, e.strerror)except ValueError as e:print "Could not convert data into integer"except:print "Unexpected error"else: # executed if try does not throw errorprint "Successful"

data structurestuplesimmutable arrays (though they can contain mutable objects)t = 1, 2, 3t[1] # 2u = t, (3, 4) # nested tuples

listslist = [123, 0, 'dynamic typing']print t[0] # outputs 123for i in list:print i # prints out each element's valuefor i in range(len(list)):print i # prints out each index of listlist[0:2] # all elements 0 to 1 (end is exclusive)list[1:]# all elements 1 and afterlist[:2]# all elements up to 2 (exclusive)list[-1]# first element from the endlist[:]# essentially copying a listlen(list)list1 + list2 # concatenationlist.append(5)list.reverse()list.count(5) # returns 1, only one instance of 5list.insert(0, 5) # index, then objectdel list[0]max(list)min(list)list(tuple) # converts tuple to liststrings are actually just lists

sets# You cannot delete from a setlist = [1, 1, 2, 2]s = set(list) # contains [1, 2]s.add(4)1 in s # returns true3 in s # returns falses2 = set([1, 3])# to make all of these actually change the sets, append _update to the function# except for union, that's just the update() functions1.intersection(s2) # returns set([1]), equivalent to s1 & s2s1.union(s2) # returns set([1, 2, 3, 4]), equivalent to s1 | s2s1.symmetric_difference(S2) # returns set([2, 3, 4]), equivalent to s1 ^ s2s1.difference(s2) # returns set([2, 4]), equivalent to s1 - s2for item in set:print item # prints items in set in no particular orderdictionariesa = {'key1': 1, 'key2': 2, 'key3':3} # to make, use curly braces or constructorb = dict(key1=1, key2=2, key3=3)a == blen(b)b['key1'] # 1b['key1'] = 4del b['key1']key1 in b # return falsekey1 not in b # return true, equivalent to not key1 in bfor key in b:print key # prints out each key in no particular orderfor key, value in b.items():print k, 'corresponds to', v # iterate over pairs

I/Of = open('path/to/file', 'w')# second arg is mode, r = read, w = write, a = append, r+ = read# and writef.read()# optional size arg to read at most n bytes, otherwise reads entire filef.readline()# newline is at the end of the string unless the file doesn't end in newline;# hence an empty string always means end of file reachedfor line in f:print line, # empty comma means print w/o newlinef.close()f.write(string)# must be a string; use str(value) to convertf.seek(offset, from_what = 0)# go to byte in file

input(prompt)# actually attempts to evaluate the expression; usually you want raw_inputraw_input(prompt)

Python 3 only:print(x, y, sep="\n", end=" ")# comma separated arguments, can specify separator and end

exceptionsby default, the message of an exception is found in its args attributeraise Exception('something bad happened')# construction and raising of exception; you can have# multiple args and access them as a tuple wouldArithmeticError# arising from overflow error, zero division error, floating point error, etc.EOFError# arising from input() or raw_input() reading in nothing before reaching EOFIOError# some IO error, has two args as an error: errno and strerrorIndexError# sequence subscript out of rangeException# base class for all exceptions

modulesimport mymodulemymodule.myfunction()# must call things through modulefrom module import myfunctionmyfunction()# importing specific things, no need for module

import string{0}refers to first positional argument{}implicitly refers to the first positional argument{name}references keyword argument name{0.weight}references weight attribute of first argument{name[0]}references first element of keyword argument name{0:}right justify (default for numbers){0:=}forces leading 0s between sign and digits{0:^}center justify{0:+}forces sign to be included for both positive and negative numbers (default is neg only){0: }leading space for positive number, minus sign for negative numbers{0:#}valid only for binary, octal, and hex; forces leading 0b, 0o, or 0x{0:,}commas for thousands places{0:2}minimum width 2{0:02}minimum width 2, enables zero padding{0:10.3}minimum width 10, for floats specifies 3 digits after decimal{0:s}string{0:^8.2f}8 characters width min, 2 decimal places, floating, center-justified

other type specifiers:bbinaryccharacterddecimalooctalxhexadecimal, lowercaseXhexadecimal, uppercasenlocale aware decimal or floateexponential, lowercaseeexponential, uppercaseffloating point, lowercaseFfloating point, uppercasegchooses between e and fGchooses between E and F%multiplies number by 100 and displays in f, followed by a % sign

string.strip()# strips leading and trailing whitespace, or whatever char you pass in as argumentstring.rstrip()# trailing stripstring.lstrip()# leading stripstring.lower()# lowercasestring.upper()# uppercase

import syssys.stdin# .read() and .readline() function just as they do in filessys.stdout.write(string)# you can set stdout and stderr to a file to write to filessys.stderr.write(string) print >> sys.stderr, 'some error'# if you prefersys.argv# list with arguments, argv[0] is the program name

import randomrandom.randrange(stop)# pretend the args are just like the range functionrandom.randint(a, b)# a