Ksh file name substitution

Created by Alin Brindusescu on 2011-02-23 04:22:55
? match any single character
* match zero or more characters, including null
[abc] match any character or characters between the brackets
[x-z] match any character or characters in the range x to z
[a-ce-g] match any character or characters in the range a to c, e to g
[!abc] match any character or characters not between the brackets
[!x-z] match any character or characters not in the range x to z
. strings starting with . must be explicitly matched
?(pattern-list) match zero or one occurrence of any pattern
*(pattern-list) match zero or more occurrences of any pattern
+(pattern-list) match one or more occurrence of any pattern
@(pattern-list) match exactly one occurrence of any pattern
!(pattern-list) match anything except any pattern
| multiple patterns must be separated with a | character


See more in ENG

Executing a sequence of commands in unix shell

Created by Alin Brindusescu on 2011-02-21 03:36:59

Last week I was arguing with a work colleague about how a sequence of commands is executed. And since I haven't upgrade my memory to a 64 bits system I had a look on the internet to see if I'm right or not and then update this here as an extension to my memory.

So what is the difference between:

 
# ./configure; make; make install
 
and 
 
# ./configure && make && make install
 
and when we have to use the first or the second one.

When using ; the commands are executed one after each other without taking into account the result of the run of the previous one. So if the execution of configure fail make and make install will still be executed. So, use ; into a sequence of commands if you want all commands to be executed. For our case this dosen't make to much sense, because if configure fails then we have no reason to run make.
When using && the commands are executed one after each other but if one of the commands fail then the next on the sequence will not be executed anymore. So if the execution of configure fail then make and make install will not be executed. So, use && into a sequence of commands if there is some logical flow to be considered. For our case this make sense, because if configure fails then we have no reason to run make, or if make fails then we have no reason to run make install even if the configure ended with success.

There is one more operand ||, using this operand will cause the execution of the right side command only if the first command fails. For example

 
# ./configure || echo "Configure Fail"
 
here the echo command is executed only if the return status of configure is non zero.

All this 3 can be combined in one line but some rules should be considered otherwise the results would be unexpected.
In the following example:

 
# ./configure || echo "Configure Fail" ; cat error_log
 
the cat command is executed even if the configure command it dose not fail.
Now let's try to add the cat to our configure make sequence:
 
# ./configure && make && make install || cat error_log
 
In this sequence if configure fails then cat is executed, if configure is successful and make fails cat will still be executed and so on. If all the commands return 0 then cat will not be executed. I found this sequence very useful
The commands and the logical operators can be grouped to form more complex execution flows using {} or ().
When using {} to group the commands the commands will be executed in the current shell and when grouping with () the commands will be executed in a subshell.



See more in ENG ROM

Getting the content of a string without the last column

Created by Alin Brindusescu on 2011-01-24 07:17:41

This days I need a Linux command line to extract the a part of a string without the last column.
My problem: I had, in a file, some strings like this:

 
# less test.txt
String_With_4_Columns
String_With_More_Than_4_Columns
With_Less
 
and I had to find a simple way to remove the last word after "_". This will not work with cut because the number of columns is variable so I can't specify what columns I want. The solution I had for this problem was to use awk:
 
# awk -F_ '{ print substr($0, 0, length($0) - (length($NF)==length($0)?0:length($NF) + 1)) }' test.txt
String_With_4
String_With_More_Than_4
With
 
See more in ENG ROM

Mic tutorial wget

Created by Alin Brindusescu on 2011-01-17 17:58:25

Cum nu sint un mare fan al programelor de download pear-to-pear, wget este prietenul meu. Nu de multe ori wget m-a ajutat sa imi simplific sau programez descarcarea fisierelor de pe web.

Wget poate fi folosit pentru a descarca fisiere HTTP, HTTPS si FTP. Ruleaza in linie de comanda si tocmai in asta sta puterea sa, in lipse de interactivitate cu utilizatorul, ceea ce ne permite sa il utilizam in scripturi shell sau in crontab.
Wget poate fi obtinut de pe pagina officiala GNU Wget unde este disponibila si o documentatie amanuntita.

Cel mai simplu mod de utilizare:
# wget URL
Cu ajutorul acestei comenzi (in cazul in care nu aveti un script de configurare, .wgetrc, pentru wget in directorul home) veti descarca fisierul index.htm(l) de la adresa URL trimisa ca si parametru. Apelind wget fara nici un alt parametru (in lipsa fisierului de configurare .wgetrc) se va descarca doar fisierul index de la adresa URL specificata fara imaginile continute in pagina respectiva

Optiunile pe care le-am folosit cel mai des:
OptiuneDescriere Exemple
-qDescarca fisierele in mod silentios, nu se afiseaza nici un fel de mesaje# wget -q www.brindusescu.org
Nu va afisa nici un fel de mesaj. Aceasta optiune este foarte utila mai ales atunci cind wget este apelat dintr-un script.
-nvAfiseaza mai putine informatii# wget -nv www.brindusescu.org
Va afisa doar: 00:23:39 URL:http://www.brindusescu.org/ [23360] -> \"index.html\" [1]
-o log_fileCreeaza un fisier de log# wget -o brind_down_log www.brindusescu.org
Daca totusi doriti sa stiti ce s-a intimplat exact pe parcursul procesului de descarcare dar nu doriti ca informatia sa fie afisata la stdout puteti folosi aceasta optiune. Toate mesajele care ar fi fost afisate se vor gasi in fisierul log specificat dupa optiunea -o
-A tip_fisierPermite \\r\\nspecificarea tipurilor de fisier care se doreste a fi descarcate# wget -A pdf,ps www.brindusescu.org/download/
Va descarca doar fisierele cu extensia .pdf si .ps de la adresa URL specificata
-R tip_fisierPermite specificarea tipurilor de fisier care se doreste a fi ignorate in procesul de descarcare# wget -r -A swf,wav www.brindusescu.org/download/
Nu va descarca fisierele cu extensia .swf si .wav daca acest tip de fisiere exista la adresa URL specificata
-rRecursiv Aceasta optiune este in general folosita impreuna cu -l.
-lAdincimea pina la care sa urmareasca legaturile atunci cind descarca fisiere# wget -r -l 2 www.brindusescu.org
Va descarca toate fisierele din directorul radacina (unde se afla si index.hmtl) si fisierele care sint specificate ca si legaturif fisierele descarcate din directorul radacina
--limit-rate=rateLimiteaza rata (viteza) de transfer# wget -r -l 2 --limit-rate=30k www.brindusescu.org
Va limita rata de transfer la 30 de kilo bytes pe secunda. Daca sinteti atit de norocosi sa aveti o conexiune de citiva MBps si doriti sa o limitati la valoarea de 1 MBps puteti folosi --limit-rate=1m
-QLimiteaza cantitatea descarcata# wget -r -l 2 -Q10m www.brindusescu.org
Va opri descarcarea fisierelor dupa ce au fost descarcati 10 Mega bytes.
-i input_fileDescarca fisierele de la adresele URL specificate in input_file# wget -i citeva_carti www.brindusescu.org
Va descarca fisierele specificate in fisirul trimis ca si parametru.
-cContinuua un fisier ca carui descarcare a fost intrerupta\\r\\n# wget www.brindusescu.org/download/un_fisier_mare.pdf\\r\\n
sa presupunem ca descarcarea se intrerupe dintr-un motiv oarecare si fisierul nu a fost complet descarcat.
# wget -c www.brindusescu.org/download/un_fisier_mare.pdf
Daca descarcare fisierului a fost intrerupta inainte de finalizare, cind se va apela a doua oara wget cu optiunea -c se va relua descarcarea fisierului fara a mai descarca inca o data partea descarcata in prima sesiune
-NDescarca doar fisierele modificate de la ultima operatie de descarcare# wget www.brindusescu.org/some_test.dat
dupa ceva timp (o ora, zi, saptamina) apelam wget cu optiunea -N
# wget -N www.brindusescu.org/some_test.dat
Daca fisierul some_test.dat a fost modificat (e suficient ca timestamp-ul sa fie modificat) va fi descarcat din nou, daca timestamp-ul nu a fost modificat fisierul nu va fi descarcat din nou.
--http-user=
user
--http-passwd=
password
Permite specificare unui nume de utilizator si parole daca adresa specificata necesita autentificare# wget --http-user=testUsr --http-passwd= testPass -r -N www.brindusescu.org/test/auth_test.dat
In cazul in care adresa specificata va initia o cerere de autentificare se testUsr/testPass vor fi folosite ca si date de indentificare

Daca aveti optiuni pe care le folositi tot timpul va puteti creea un fisier de configurare in directorul home, pentru a fi recunoscut si procesat automat de fiecare data cind apelati wget aceasta trebuie sa poarte numele de .wgetrc.
Wget va modifica fisierele html pe care le descarca inlocuind calea spre fisierele descarcate cu o cale locale (unde a descarcat fisierele), pentru fisierele care nu au fost descarcate va pastra calea origniala.

See more in ROM

Bash shortcuts

Created by Alin Brindusescu on 2011-01-16 14:50:49

Some bash shortcuts I found useful in bash shell. Note: not all of them are working when you use them over a ssh connection.
CTRL + A - move the prompt to the begining of the current line
CTRL + E - move the prompt to the end of the current line
ALT + F - move the prompt one word to the right
ALT + B - move the prompt one word to the left
CTRL + L - clear the the terminal (equivalent with clear comanda)
CTRL + R - search history
CTRL + U - delete the content of the curent line until the prompt
CTRL + K - delete the content of the current line from the prompt till the end of line
CTRL + W - delete the word found in front of the prompt
ALT + D - delete the word after the prompt
CTRL + T - swap the 2 letters before the prompt
ESC + T or ALT + T - swap the 2 words before the prompt
CTRL + K - cut the text before the prompt
CTRL + Y - paste the text cuted with CTRL + K

See more in ENG ROM

Search

Ads