Linux Advanced Text Processing Tools

script
Log my current session in the text terminal into text file typescript . The log finishes when I type exit or press d.

emacs
(in X-terminal) The emacs text editor. Advanced and sophisticated text editor. Seems for gurus only: "emacs is not just an editor, it is a way of living". Emacs surely seems rich or bloated, depending on your point of view. There are likely 3 versions of emacs installed on your system: (1) text-only: type emacs in a text (not X-windows) terminal (I avoid this like fire); (2) graphical-mode: type emacs in an X-windows terminal (fairly usable even for a newbie if you decide to take some time to learn it); and (3) X-windows mode: type "xemacs" in an X-windows terminal.

vi
The famous (notorious?) "vi" text editor (definitely not recommended for newbies). To exit "vi" (no changes saved) use these five characters: :q! I use the pico text editor and don't ever need vi (well, unless I have to unmount the /usr subsystem and modify/edit some configuration files, then vi is the only editor avialable). To be fair, modern Linux distributions use vim (="vi improved") in place of vi, and vim is somewhat better than the original vi used to be. The GUI version of vi is also available (type gvim in an X terminal).

Experts do love vi, but vi is definitely difficult unless you use it every day. Here is a non-newbie opinion on vi (http://linuxtoday.com/stories/16620.html):

"I was first introduced to vi in 1988 and I hated it. I was a freshman in college... VI seemed archaic, complicated and unforgiving... It is now 12 years later and I love vi, in fact it is almost the only editor I use. Why the change? I actually learned to use vi... Now I see vi for what it really is, a powerful, full featured, and flexible editor..."

A short introduction to basic vi commands and modes can be viewed at http://www.thelinuxgurus.org/vitut.shtml.

For your entertainment, you may also want to try the even more ancient ed editor (just type ed on the command line).

nano
This is a brand new (March 2001) GNU replacement for pico. Works and looks like pico, but it is smaller, better, and licenced as expected for a decent Linux piece of software (i.e., GPL). Not yet included with RH7.0 or MDK7.2, but expect it soon to be.

khexedit
(in X terminal) Simple hexadecimal editor. Another hexadecimal editor is hexedit (text based, less user=friendly). Hex editors are used for editing binary (non-ASCII) files.

diff file1 file2 > patchfile
Compare contents of two files and list any differences. Save the output to the file patchfile.

sdiff file1 file2
Side-by-side comparison of two text files. Output goes to the "standard output" which normally is the screen.

patch file_to_patch patchfile
Apply the patch (a file produced by diff, which lists differences between two files) called patchfile to the file file_to_patch. If the patch was created using the previous command, I would use: patch file1 patchfile to change file1 to file2.

grep filter
Search content of text files for matching patterns. Definitely worth to learn at least the basics of this command.

A simple example. The command:

cat * | grep my_word | more

will search all the files in the current working directory (except files starting with a dot) and print the lines which contain the string "my_word".

A shorter form to achieve the same may be:

grep my_word * |more

The patterns are specified using a powerful and standard notation called "regular expressions".

There is also a "recursive" version of grep called rgrep. This will search all the files in the current directory and all its subdirectories for my_word and print the names of the files and the matching line:

rgrep -r my_word . | more

In regular expressions, most characters just match themselves. The exceptions are the "metacharacters" that have special meaning.

In regexpr, the special characters are: "" (backslash), "." (dot), "*" (asterisk), "[" (bracket), "^" (caret, special only at the beginnig of a string), "$" (dollar sign, special only at the end of a string). A character terminating a pattern string is also special for this string.

The backslash, "" is used as an "escape" character, i.e., to quote a subsequent special character.
Thus, "" searches for a backslash, "." searches for a dot, "*" searches for the asterisk, "[" searches for the bracket, "^" searches for the caret even at the begining of the string, "$" searches for the dollar sign even at the end of the string.

Backslash followed by a regular (non-special) character may gain a special meaning. Thus, the symbols < and > match an empty string at the beginning and the end of a word, respectively. The symbol  matches the empty string at the edge of a word, and B matches the empty string provided it's not at the edge of a word.

The dot, ".", matches any single character. [The dir command uses "?" in this place.] Thus, "m.a" matches "mpa" and "mea" but not "ma" or "mppa".

Any string is matched by ".*" (dot and asterisk). [The dir command uses "*" instead.] In general, any pattern followed by "*" matches zero or more occurences of this pattern. Thus, "m*" matches zero or more occurances of "m". To search for one or more "m", I could use "mm*".

The * is a repetition operator. Other repetition operators are used less often--here is the full list:
* the proceding item is to be matched zero or more times;
+ the preceding item is to be matched one or more times);
? the preceding item is optional and matched at most once);
{n} the preceding item is to be matched exactly n times;
{n,} the preceding item is to be matched n or more times);
{n,m} the preceding item is to be matched at least n times, but not more than m times.

The caret, "^", means "the beginning of the line". So "^a" means "find a line starting with an "a".

The dollar sign, "$", means "the end of the line". So "a$" means "find a line ending with an "a".

Example. This command searches the file myfile for lines starting with an "s" and ending with an "n", and prints them to the standard output (screen):

cat myfile | grep '^s.*n$'

Any character terminating the pattern string is special, precede it with a backslash if you want to use it within this string.

The bracket, "[" introduces a set. Thus [abD] means: either a or b or D. [a-zA-C] means any character from a to z or from A to C.

Attention with some characters inside sets. Within a set, the only special characters are "[", "]", "-", and "^", and the combinations "[:", "[=", and "[.". The backslash is not special within a set.

Some useful categories of characters are: [:upper:] =upper-case letters, [:lower:] =lower-case letters, [:alpha:] =letters meaning upper+lower, [:digit:] =0 to 9, [:alnum:] = alpha and digits, [:space:] =whitespace meaning ++ and similar, [:graph:] =graphically printable characters except space, [:print:] =printable characters including space, [:punct:] =punctuation characters meaning graphical characters minus alpha and digits, [:cntrl:] =control characters meaning non-printable characters.

Example. This command prints lines containing a capital letter followed by a digit:

dir -l | grep '[[:upper:]][[:digit:]]'

tr
(=translation). A filter useful to replace all instances of characters in a text file or "squeeze" the white space.

Example :

cat my_file | tr 1 2 > new_file

This command takes the content of the file my_file, pipes it to the translation utility tr, the tr utility replaces all instances of the character "1" with "2", the output from the process is directed to the file new_file.

sed
(=stream editor) I use sed to filter text files. The pattern to match is typically included between a pair of slashes // and quoted.

For example, to print lines containing the string "1024", I may use:

cat filename | sed -n '/1024/p'

Here, "sed" filters the output from the cat command. The option "-n" tells sed to block all the incoming lines but those explicitly matching my expression. The sed action on a match is "p"= print.

Another example.

cat filename | sed '/.*o$/d' > new_file

In this example, lines ending the an "o" will be deleted. I used a regular expression for matching any string followed by an "o" and the end of the line. The output (i.e., all lines but those ending with "d") is directed to new_file.

To search and replace, I use the sed 's' action, which comes in front of two expressions:

cat filename | sed 's/string_old/string_new/' > newfile

A shorter form for the last command is:

sed 's/string_old/string_new/' filename > newfile

gawk
(=GNU awk. The awk command is a traditional UNIX tool.) A tool for processing text files, in many respects similar to sed. Perl can do all that gawk can, and more, so I don't bother with gawk. For simple tasks, I use sed, for more complicated tasks, I use perl.

cvs
Concurrent versions system. Try: info cvs for more information. Useful to keep the "source code repository" when several programmers are working on the same computer program.

cervisia
(in X-terminal). A GUI front-end to the cvs versioning system.

file -z filename
Determine the type of the file filename. The option -z makes file look also inside compressed files to determine what the compressed file is (instead of just telling you that this is a compressed file).

To determine the type of content, file looks inside the file to find particular pieces of contents ("magic numbers")--it does not just look at the filename extension like MS Windows does. The "magic numbers" are stored in the text file /usr/share/magic--really impressive database of filetypes.

strings filename | more
Display the strings contained in the binary file called filename. "strings" could, for example, be a useful first step to a close examination of an unknown executable.

od
(=octal dump). Display contents as octal numbers. This can be useful when the output contains non-printable characters. For example, a filename may contain non-printable characters and be a real pain.

Examples:
dir | od | more
cat my_file | od |more
od my_file

wc
(=word count) Print the number of lines, words, and bytes in the file.

Examples:
dir | wc
cat my_file | wc
wc myfile

cksum filename
Computing the CRC (="cyclic redundancy check") for file filename to verify its integrity.

md5sum
Computed a md5 checksum (128-bit) for file filename to verify its integrity.

mkpasswd -l 10
Make a hard-to-guess, random password of the length of 10 characters.

sort -f filename
Arrange the lines in filename according to the ascii order. The option -f tells sort to ignore the upper and lower character case. The ascii character set is (see man ascii):

Dec Hex Char Dec Hex Char Dec Hex Char Dec Hex Char
---------------------------------------------------------------------------
0 00 NUL '' 32 20 SPACE 65 41 A 97 61 a
1 01 SOH 33 21 ! 66 42 B 98 62 b
2 02 STX 34 22 " 67 43 C 99 63 c
3 03 ETX 35 23 # 68 44 D 100 64 d
4 04 EOT 36 24 $ 69 45 E 101 65 e
5 05 ENQ 37 25 % 70 46 F 102 66 f
6 06 ACK 38 26 & 71 47 G 103 67 g
7 07 BEL 'a' 39 27 ' 72 48 H 104 68 h
8 08 BS '' 40 28 ( 73 49 I 105 69 i
9 09 HT ' ' 41 29 ) 74 4A J 106 6A j
10 0A LF ' ' 42 2A * 75 4B K 107 6B k
11 0B VT 'v' 43 2B + 76 4C L 108 6C l
12 0C FF 'f' 44 2C , 77 4D M 109 6D m
13 0D CR ' ' 45 2D - 78 4E N 110 6E n
14 0E SO 46 2E . 79 4F O 111 6F o
15 0F SI 47 2F / 80 50 P 112 70 p
16 10 DLE 48 30 0 81 51 Q 113 71 q
17 11 DC1 49 31 1 82 52 R 114 72 r
18 12 DC2 50 32 2 83 53 S 115 73 s
19 13 DC3 51 33 3 84 54 T 116 74 t
20 14 DC4 52 34 4 85 55 U 117 75 u
21 15 NAK 53 35 5 86 56 V 118 76 v
22 16 SYN 54 36 6 87 57 W 119 77 w
23 17 ETB 55 37 7 88 58 X 120 78 x
24 18 CAN 56 38 8 89 59 Y 121 79 y
25 19 EM 57 39 9 90 5A Z 122 7A z
26 1A SUB 58 3A : 91 5B [ 123 7B {
27 1B ESC 59 3B ; 92 5C '' 124 7C |
28 1C FS 60 3C < 93 5D ] 125 7D }
29 1D GS 61 3D = 94 5E ^ 126 7E ~
30 1E RS 62 3E > 95 5F _ 127 7F DEL
31 1F US 63 3F ? 96 60 `
64 40 @

If you wondered about the control characters, here is the meaning of some of them on the console (Source: man console_codes). Each line below gives the code mnemonics, its ASCII decimal number, the key combination to produce the code on the console, and a short description:
BEL (7, G) bell (=alarm, beep).
BS (8, H) backspaces one column (but not past the beginning of the line).
HT (9, I) horizonal tab, goes to the next tab stop or to the end of the line if there is no earlier tab stop.
LF (10, J), VT (11, K) and FF (12, L) all three give a linefeed.
CR (13, M) gives a carriage return.
SO (14, N) activates the G1 character set, and if LF/NL (new line mode) is set also a carriage return.
SI (15, O) activates the G0 character set.
CAN (24, X) and SUB (26, Z) interrupt escape sequences.
ESC (27, [) starts an escape sequence.
DEL (127) is ignored.
CSI (155) control sequence introducer.

uniq
(=unique) Eliminate duplicate lines in sorted input. Example: sort myfile | uniq

fold -w 30 -s my_file.txt > new_file.txt
Wrap the lines in the text file my_file.txt so that there is 30 characters per line. Break the lines on spaces. Output goes to new_file.txt.

fmt -w 75 my_file.txt > new_file.txt
Format the lines in the text file to the width of 75 characters. Break long lines and join short lines as required, but don't remove empty lines.

nl myfile > myfile_lines_numbered
Number the lines in the file myfile. Put the output to the file myfiles_lines_numbered.

rev filename > filename1
Print the file filename, each line in reversed order. In the example above, the output is directed to the file filename1.

shred filename
Repeatedly overwrite the contents of the file filename with garbage, so that nobody will ever be able to read its original contents again.

paste file1 file2 > file3
Merge two or more text files on lines using as delimiter (use option "d=" to specify your own delimiter(s).

Example. If the content of file1 was:
1
2
3
and file2 was
a
b
c
d
the resulting file3 would be:
1 a
2 b
3 c
d

des -e plain_file encrypted_file
(="Data Encryption Standard") Encrypt plain_file. You will be ask for a key that the program will use for encryption. Output goes to encrypted_file. To decrypt use
des -d encrypted_file decrypted_file.

gpg
"Gnu Privacy Guard"--a free equivalent of PGP ("Pretty Good Privacy"). gpg is more secure than PGP and does not use any patented algorithms. gpg is mostly used for signing your e-mail messages and checking signatures of others. You can also use it to encrypt/decrypt messages. http://www.gnupg.org/ contains all the details, including a legible, detailed manual.

To start, I needed a pair of keys: private and public. The private key is used for signing my messages. The public key I give away so that others can use it to verify my signatures. [One can also use a public key to encrypt a message so it can only be read using my private key.] I generated my keypair using this command:
gpg --gen-key

My keys are stored in the directory ~/.gnupg (encrypted using a passphrase I supplied during the key generation). To list my public key in plain text file, I use:
gpg --armor --export my_email_address > public_key_stan.gpg

which created a file public_key_stan.gpg containing something like this:

-----BEGIN PGP PUBLIC KEY BLOCK-----
Version: GnuPG v1.0.1 (GNU/Linux)
Comment: For info see http://www.gnupg.org

mQGiBDmnzEYRBACoN438rxANaMfCy5bfj6KWM0/TR6x6HZ0gpmhGeuouM/SOR2IU
/G30NdCuzHeFs93BhtY0IdzoEMtMyZHnvdhZC2bx/jhgaaMbEaSsXwRhVB0xVPYx
rHbsgSULHYzRFF34MS3/Lse3QWfWxzA7I0lbXB7nLwZKZqaNONRFRR42owCg60hV
TDPEB2N0llMyt12R4ZByFSsEAJ1tE7pb9b6TP7cw21vkIjc+BI2uSzn/B4vNlCWK
TTuZHVv0w0jFcbd8DB0/1tlZUOrIzLSqJyQDGiNn258+7LetQ+LKG/1YKbiAcosz
4QirBuLIeF2M9GuXYCwZypE3Dwv+4YupvybR31CgLTJ8p4sKqC5n0eSr2oSrtdHZ
yuJtA/9v2HcebOncfCNOK+cVRmcTB1Frl/Gh/vNCfeZyXaJxlqDfCU2vJHtBemiE
AtcfZHB/iHy0DM68LfRJSAIFAa5um9iWHh5/vWCGZLqtpwZ7kyMw+2D6CFkWATsy
wQA1g1VcGkNc14Crrd36qf60bI+b8pn2zDhwZtLsELsXyXkNhbQmU3RhbiBKIEts
aW1hcyA8U3RhbktsaW1hc0B3ZWJoYXJ0Lm5ldD6IVgQTEQIAFgUCOafMRgQLCgQD
AxUDAgMWAgECF4AACgkQt+ZBooH8bHd2kwCghAt9aKIk0mRJv+g7YcRPotVtrwkA
n1a4xEVEyaKgKoMaJnopf69K9+vouQENBDmnzH4QBADgFpLP+tWZPnVYg47cn+9b
XQRjdOtNsDE6BYH872/sR1oCrdH6k+gXFOiZxRZ3PElK2/olo59kh5xa9aBxNdEC
FuXJN0UelmhOFbDtqVksIqVWyYfXnLz+wtcXg0Q0L0q8vY4IuTzw2WkV6EkM+/x8
6UhA2XVaMJKBdRKFSVilbwADBQP+JCzLj5HDgpRvf+KM72nzSg7sp8Tki7nF9wNA
PODK0SeQgI3dwXYyF6AVenlETE/3xRWoYQN1bxVZsOex9vzqPrQC3dR0NBljd74r
kfXwUTl2fNQX4N9iuVCo2gCGbi5+gfEk1GhsWDsq0z40f+18k+XBdWmY8sCNiolT
tnvm1QeIRgQYEQIABgUCOafMfgAKCRC35kGigfxsd9SGAJ9/FWSkEfgbE/Yc46d8
Ef1gYg3I1ACff3oLeAMeGGO79gW6UGp9RJ6mRao=
=X1k2
-----END PGP PUBLIC KEY BLOCK-----

Now, I can e-mail my public key to the people with whom I want to communicate securely. They can store it on their pgp system using;
gpg --import public_key_stan.gpg

Even better, I can submit my public key to a public key server. To find a server near me, I used:
host -l pgp.net | grep wwwkeys

and to submit the key, I did (can take a couple of minutes, and I am connected to the Internet):
gpg --keyserver wwwkeys.pgp.net --send-keys linux_nag@canada.com

The "wwwkeys.pgp.net" is the key server I selected, and "linux_nag@canada.com" is my email address that identifies me on my local key ring. I need to submit myself only to one public key server (they all synchronize).

Now, I can start using gpg. To manually sign a plain text file my_message, I could use:
gpg --clearsign my_message

This created file my_message.asc which may contain something like:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello World!

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.1 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE5p9+3t+ZBooH8bHcRApn/AJ9kx9+pU3GJBuvJN9Bo3bW3ku/5PwCgquht
mfrPrt7PQtdmGox72jkY0lo=
=rtK0
-----END PGP SIGNATURE-----

To verify a signed message, I could do:
gpg --verify my_message.asc

If the contents of the signed section in my_message.asc was even slightly modified, the signature will not check.

Manual signing can be awkward. But, for example, kmail can sign the electronic signatures automatically for me.

"docbook" tools
Docbook is the incoming standard for document depository. The docbooks tools are included with RH6.2 (and later) in the package "jade" and include the following converters: db2ps, db2pdf, db2dvi, db2html, db2rtf which convert docbook files to: postscript (*.ps), Adobe Portable Document Format (*.pdf), device independent file format (*.dvi), HyperText Markup Language (*.html), and Rich Text Format (*.rtf), respectively.

"Document depository" means the document is in a format that can be automatically translated into other useful formats. For example, consider a document (under development) which may, in the future, need to be published as a report, a journal paper, a newspaper article, a webpage, perhaps a book, I (the author) am still uncertain. Formatting the document using "hard codes" (fonts, font sizes, page breaks, line centering, etc.) is rather a waste of time--styles vary very much between the particular document types and are publisher-dependent. The solution is to format the document using "logical" layout elements which may include the document title, chapter titles, subchapters, emphasis style, picture filenames, caption text, tables, etc. Thats what "docbook" does--it is a description of styles (using xml, a superset of html, and a close relative of sgml)--a so-called stylesheet. The logical layout is rendered to a physical appearance when the document is being published.

This section will be expanded in the future as we learn to use docbook.

7.2 Simple Programming under Linux

perl
Powerful and widely used scripting language, very popular among gurus. Perl looks cryptic yet it is quite straight-forward if you need to achieve simple tasks. Think of perl as a swiss-army knife for simple programming. Here is how Eric Reymond (famous Linux guru) describes perl: "Perl, of course, is the 800-pound gorilla of modern scripting languages. It has largely replaced shell as the scripting language of choice for system administrators, thanks partly to its comprehensive set of UNIX library and system calls, and partly to the huge collection of Perl modules built by a very active Perl community. The language is commonly estimated to be the CGI language behind about 85% of the ``live'' content on the Net. Larry Wall, its creator, is rightly considered one of the most important leaders in the Open Source community, and often ranks third behind Linus Torvalds and Richard Stallman in the current pantheon of hacker demigods."

How do I write a simple perl script?

I may use pico (or any other text editor of my choice) to type in a simple perl script:

pico try_perl

The example script below does nothing useful, except illustrates some features of perl:

#!/usr/bin/perl -w
# a stupid example perl program
# the lines starting with # are comments except for the first line
# names of scalar variables start with $
$a=2;
$b=3;
# each instruction ends with a semicolon, like in "c"
print $a**$b," ";
$hello_world='Hello World';
print $hello_world," ";
system "ls";

The first line tells the shell how to execute my text file. The option "-w" causes perl to print some additional warnings, etc. that may be useful for debugging your script. The next 3 lines (starting with #) are comments. The following lines are almost self explanatory: I assign some values to two variables ($a and $b), put $a to power $b and print the result. The " " prints a new line, just like in the "c" programming language. Then I define another variable to contain the string "Hello World" and, in the next line, I print it to the screen. Finally, I execute the local operating system command "ls", which on Linux prints the listing of the current directory content. Really stupid script.

After saving the file, I make it executable:
chmod a+x try_perl

Now, I can run the script by typing:
./try_perl

python
Modern and very elegant object oriented interpreter. Powerful and (arguably) more legible than perl. Very good (and large) free handbooks by G. van Rossum (the Python creator) are available on the net (try: http://www.python.org/doc/ for browsing or ftp://ftp.python.org for downloading).

How do I write a simple Python program?

Edit a text file that will contain your Python program:
pico try_python

Type in some simple python code to see if it works:

#!/usr/bin/env python
print 2+2

The first line (starting with the "pound-bang") tells the shell how to execute this text file--it must be there (always as the first line) for Linux to know that this particular text file is a Python script. The second line is a simple Python expression.

After saving the file, I make it executable:
chmod a+x try_python

after which I can run it by typing:
./try_python

Python is an excellent, and very modern programming language. Give it a try, particularly if you like object oriented programming.

Here is a slightly longer, but still (hopefully) self-explanatory python code:

#!/usr/bin/env python
# All comments start with a hash
# This program converts human years to dog years

# get the original age
age = input("Enter your age (in human years): ")
print # print a blank line

# check if the age is valid using a simple if statement
if age < 0:
print "A negative age is not possible."
elif age < 3 or age > 110:
print "Frankly, I don't believe you."
else:
print "That's the same as a", age/7, "year old dog."

tcl
(Pronounced "tickle".) Popular scripting language.

A simple tcl program?

#!/usr/bin/tclsh
puts stdout {Hello World!}

wish
(type in X-terminal ) A front-end to Tk, an X-windows extension of tcl. Often used for building front-ends of a program.

How do I write a simple GUI program (using Tk)?

Tk is a GUI extension of the easy yet powerful tcl programming language. For example, I may use pico to create a text file that will contain a simple tk program:

pico try_tk

and type in a simple example of tk code to see if it works:

#!/usr/bin/wish
button .my_button -text "Hello World" -command exit
pack .my_button

The first line (starting with the "#!" pound-bang) tells the shell what utility to use to execute my text file. The next two lines are an example of a simple tk program. First, I created a button called "my_button" and placed it at the root of my class hierarchy (the dot in front of "my_button"). To the button, I tied the text "Hello World" and a command that exits the program (when the button is pressed). Last line makes my program's window adjust its size to just big enough to contain my button.

After saving the file, I make it executable:
chmod a+x try_tk

after which I can run it by typing (in the X-terminal, because it requires X-windows to run):
./try_tk

Tk is very popular for building GUI front ends.

gcc filename.c
GNU C compiler. Quite straight-forward if you know C. Extensive free manuals are available on the net.

How do I compile a simple C program?

Start your favourite text editor and type in your source code. For example, I may use pico:
pico hello.c

and type in the Kerningham and Richie (the creators of "c") intro example of a C program:

#include
void main(void) {
printf("hello world ");
}

I save the file and then invoke the GNU C compiler to compile the file "hello.c":
gcc hello.c

The gcc compiler produces an executable binary file "a.out", which I can run:
./a.out

g++ filename.C
GNU C++ compiler. The capital "C" is often used for C++ sources. If you need an "integrated development envrionment" (IDE), kdevelop is really something you would probably like to look at.

kdevelop
(type in X-terminal) Integrated development environment for K. It is really worth downloading (if it does not come with your distribution).

glade
(type in X-terminal) A graphical builder of user interfaces.

"Glade is an interface builder developed by Damon Chaplin. It allows graphical and interactive construction of Gnome/Gtk graphical user interfaces. From Glade, the generated interface can be saved in a xml file or directly exported to C code to be included in a C source tree. Glade also allows to define the name of the handlers - functions - to be attached to the various event of the interface. For example the function (name) to be called when a specific menu item is pressed." (From: http://linuxtoday.com/news_story.php3?ltsn=2000-07-16-013-04-PS-GN)

What "C" functions are available for programming under Linux?
Too many for a newbie like myself. I started by studying the header files (*.h) in the directory /usr/include and all its subdirectories. To find a header file that contains a prototype for a given function (e.g., cosh) I would do something like:

cd /usr/include
grep -H "cosh" *.h

guile
An implementation of "Scheme" programming language. Scheme is a modern dialect of LISP.

Silly guile examples:
(+ 1 1)
(system "ls")
(display "hello ")
(define a 7)
(exit)

g77
GNU FORTRAN. If you are into FORTRAN, you may want to check: http://studbolt.physast.uga.edu/templon/fortran.html to find a FORTRAN compiler that suits your particular needs under Linux.

expect
Scripting language for "programmed dialog". See man expect.

kylix
This is a brand-new (Feb.2001) commercial offering from Borland (aka Inprise). In short, it is a Linux port of the famous object-oriented Pascal ("Delphi"). kylix is unlikely to be on your Linux distribution CD, you must pay for it, but if you want the best rapid application development (RAD) platform with a code portablity between Linux and MS Windows, large number of pre-built components, etc., kylix is likely the best. In my opinion, Delphi is significanly better than MS Visual Basic.

make
Run the "make" utility to build (compile, link, etc) a project described in the Makefile found in the current directory.

yes
Generate a never-ending output of strings containing "yes" (it does end when is pressed). Sounds like a silly utility, but it can be used to write simple program on the command line. This amusing example determines the frequency of digits in 100 000 radom numbers (the whole command is a single line):
yes | sed '100000q' | awk 'BEGIN{srand();u=6*log(10)}{printf"%e ",rand()*exp(rand()*u)}'| cut -c1 | sort | uniq-c
Hope this example does not scare you to much--it surely shows that old-fashioned UNIX command line can be as complicated (and powerful) as you want to make it. If you are interested why the frequency of digits varies, try the place from which I borrowed the example: http://www.newscientist.com/ns/19990731/letters4.html

7.3 Math Tools

dc
A command-line, arbitrary-precision "reverse Polish notation" (RPN) calculator.

dc is based on the concept of a stack, which is central to the operations of modern digital computer. A computer stack is not unlike a stack of kitchen plates, the last to come on stack, is the first to go out (this is also known as LIFO="last-in, first-out"). This contrasts with a queue (another important concept) where the first in is the first out (FIFO).

You can perform operations only on the number(s) which is on the top of the stack. The two basic operations are: push and pop (put on the top of stack, and retrieve from the top of stack). Unary operations pop one value off the stack ("unary" means "requiring one operand"). Binary operations pop two values off the stack ("binary" means "requiring two operands"). Tertiary operations pop three values off the stack ("tertiary" means "requiring three operands"). In all cases, the result is always pushed back onto the top of stack.

RPN calculators (regular, hand-held) are very popular among technically oriented people and in academia. The RPN notation never requires parentheses.

History. The parentheses-free logic was developed by Polish mathematician Jan Lukasiewicz (1878-1956) before the WWII. Originally, the operator preceded the values. For computer applications, it's been modified so that the operator goes after the values, hence "reversed" in the name "reversed Polish notation".

To exercise some operations on stack, try this:
dc [start the arbitrary precision reverse Polish notation calculator]
1 [push "1" on the stack]
2 [push another number on the stack]
3 [push yet another number on the stack]
4 [push yet another number on the stack]
f [print the entire stack; you should see 1 2 3 4]
p [print the number on the top of the stack without affecting the stack; you should see 4]
+ [perform addition (binary operation), therefore pop two last values off the stack (4,3), and push the result (7) on the stack]
p [print the number on the top of the stack, i.e. the results of the last addition (7).].
p [print again the number on the top of the stack to see that the stack wasn't affected by printing (7)]
* [perform multiplication (binary operation), therefore pop two last values, and push the result (14)]
p [print the result of the multiplication (14)]
P [pop the last number off the stack (14)]
p [print the number on the top of the stack]
2000 [push a large integer on the stack]
k [set the precision to the value which is on the top of the stack, i.e. 2000]
1 [push another number on the stack]
f [print the content of the entire stack]
701 [push another number on the stack]
/ [divide last two numbers on the stack, i.e. "1/701" with 2000 decimal places of precision]
p [print the result of the last division]
q [quit the arbitrary precision reverse Polish notation calculator]

Please note that when using the reverse Polish notation (RPN) you never need parentheses. Try man dc to read about other capabilities of dc.

bc
An interactive arbitrary-precision calculator. Type "quit" to exit bc. Type "scale=20" (or so) before doing floating point divisions or you end up with the quotient instead of the floating-point result of the division.

kcalc
xcalc
(in X terminal) Standard, GUI calculators.

scilab
(in X terminal) A large and sophisticated system for numerical computing, somewhat resembling "matlab", and also with clampsy interface. Don't even try it unless you have rather sophisticated math needs else you won't appreciate it. It is included on RH7.0 "powertools" CD.

A silly example session showing some matrix algebra. My input is shown bold.
-->a=[1 1 1;2 3 4]
a =
! 1. 1. 1. !
! 2. 3. 4. !

-->b=[1 1; 2 2;3 3]
b =
! 1. 1. !
! 2. 2. !
! 3. 3. !

-->c=a*b
c =
! 6. 6. !
! 20. 20. !

-->d=inv(c)
d =

1.0E+14 *

! 11.258999 - 3.3776997 !
! - 11.258999 3.3776997 !

-->

head -c 8 /dev/random
cat /dev/random | od
cat /dev/urandom | memencode
(3 commands.) Examples on how to generate random characters on the Linux command line by reading from the device "random" or "urandom". The first command produces approximately 8 characters by reading from the device "random", which generates high quality (difficult to predict) random numbers. It will become slow once the "entropy" on your computer is exhausted (e.g., when producing lots of random characters). The solution then is to wait or type something on the keyboard, move your mouse, switch the terminal, make your hard drive to read or write, etc., to generate more random noise ("entropy"). The second command keeps producing random characters, but displays them as octal numbers (using the "octal dump", od, filter). It has to be interrupted with . The third command uses the device "urandom", which is faster then "random" when generating lots of random characters. But when the system enthropy is low, the randomness of its output from "urandom" might be compromised, yet it probably is still good for all but most demanding applications. The output is filtered to the mime format (the Internet mail-oriented 7-bit encoding standard) so it is all printable ASCII. The detailed description of the theorry and implementation of the Linux algorithm for generating the random numbers can be found in the source code file://usr/src/linux/drivers/char/random.c on your Linux system.

factor 10533858466222239345
Find all the prime factors of an integer number. Factors of an integer are numbers by which the integer is divisible without a remainder. For example, the factors for 6 are: 1, 2, 3, and 6.

R
A programming language / environment for advanced statistical computing. Type "quit()" to exit.

gnuplot
Utility for creating graphs and plots. Very good for non-interactive (batch) work, but not very simple for interactive use. A good introduction to gnuplot can be found at http://www.duke.edu/~hpgavin/gnuplot.html.

7.4 Miscellaneous

How do I run an MS Windows Application (using "wine")?

You don't want to install Linux to run MS Windows applications. Still, the Linux-based "wine" library lets me execute some MS Windows binaries, although with a rather severe speed penalty.

On my system (Wine installed), I can execute MS Solitaire by typing in the X-windows terminal:

wine /mnt/dos_hda1/windows/sol.exe

The /mnt/dos_hda1 is the mount point of the harddrive partition that contains MS Windows, and it is mounted.

If you don't have wine installed, put your Mandrake cd into the cdrom, mount it, and then do something like this (as root):
cd /mnt/cdrom/Mandrake/RPMS/
rpm -ihv wine-991212-1mdk.i586.rpm

Mandrake packages are RedHat-compatible so you can use the Mandrake CD to install software that RedHat lacks.

Can I have a RAID if my computer has two or more IDE (or other) harddrives?

RAID = "redundant array of inexpensive drives". RAID can be used for a "on-the-fly" mirroring of one drive to another so as to protect your data and keep your system functioning in case of a disk failure. Linux comes with a set of RAID tools that let you custom-design a RAID system to suit your particular needs.The pieces of RAID on Linux are:
mkraid - initializes/upgrades RAID device arrays
raid0run - starts up old (superblock-less) RAID0/LINEAR arrays
raidstart - command set to manage md devices
raidstop - command set to manage md devices
raidtab - configuration file for md (RAID) devices

RAID operates by joining two or more disks into a single logical device. There are several layers of RAID:

RAID 0 layer ("striping") just joins two or more disks into a single logical device, without giving any redundancy. It is often used to join RAID 1 or RAID 5 layers. RAID 0 + RAID 1 is called RAID 10. RAID 0 + RAID 5 is called RAID 50.

RAID 1 (mirroring) combines two disks, each containing the same data.

RAID 4 combines three or more disks, with one of the disks dedicated to parity. If any disk fails, the whole logincal device remains available, but with degraded performance. It is not used very often because of the performance.

RAID 5 combines three or more disks, with parity distributed accross the disks. Functionality similar to RAID 4 but apparently better performance.

Try http://www.osfaq.com/vol1/linux_softraid.htm if you would like set up a raid on your computer.

Network traffic shaping using shapecfg

Nice info can be found at: http://oreilly.linux.com/pub/a/linux/2000/08/24/LinuxAdmin.html
Unlikely I will really ever need traffic shaping on my home network, yet it makes an interesting exercise for the curious.