Programming :: Writing Shell Snip To Import CSV Data Into Bash Array
Jan 5, 2011
I have been trying to write a simple snip of bash shell code to import from 1 to 100 records into a Bash array.
I have a CSV file that is structured like:
record1,item1,item2,item3,item4
record2,item1,item2,item3,item4
record3,item1,item2,item3,item4
record4,item1,item2,item3,item4
And would like to get this data into corresponding arrays as such:
$record1[item1-4]
$record2[item1-4]
$record3[item1-4]
$record4[item1-4]
I am about to write my first shell script for backing up my server.
These are the steps I have identified so far.
Copy files to be backed up to /srv/backup/ run mysqldump and copy the file to srv/backup/databases run duplicity to backup /srv/backup/* to another folder on my machine
I am writing a bash shell scrip that will be run everyday, and will carry out the three tasks mentioned above.
Note: point 3 (backing up to a local folder) is only a temporary measure - to allow me to understand what I'm doing, since all the tools I am using, are new to me. Once I can backup and restore correctly, I will use duplicity to compress and encrypt the files and upload them offsite.
If my understanding of duplicity is correct (according to the documentation here), the first time I run the script, a FULL backup will be done. Every subsequent backup will then be incremental. I will then force a FULL back on say a weekend.
First things first though - I have a few questions:
I would like to use backup rotation for the 'scheme' described above - I would like some recommendations on what kind/type of rotation to use. Once I have implemented the backup rotation, how can I restore from a particular day back in time (assuming the backup exists of course). ?
I've used ssh for a long time, but recently I set up a new server on my LAN. Often when I try to connect to it I get a message like so:
Code: Warning: the ECDSA host key for '<snip>' differs from the key for the IP address '<snip>' Offending key for IP in /home/<snip>/.ssh/known_hosts:14 Matching host key in /home/<snip>/.ssh/known_hosts:12 Are you sure you want to continue connecting (yes/no)? What is weird is that, in the known_hosts file, the entry for the ip address (line 14) is a "ssh-rsa" type, but the entry for the hostname is a "ecdsa-sha2-nistp256", even though they both connect to the same server. What is going on here?
I looked on the net for such function or example and didin't find anything, thus after having made one i guess it would be legitimate to drop it to see what others thinks of it.
#!/bin/bash addelementtoarray() { local arrayname=$1
I need help creating a script that makes a log file in wich to save information about every user that uses the ftp command (information like username and date) and the server to wich he is trying to connect.
I have assignment to write bash script and I have to submit it after tommorow . I do not have experience to finish it by myself.I am really need help if any one can contact me. on my email I will be thanks him/ her.
Here the description of the issue I am having.I am writing a bash test script which reads lines from a file, builds ISO messages, sends them to a server, reads the response with response code and reports the result of the test to a file or on the screen.The message that I need to send is 94 characters long.Here's the portion of a code that I initially wrote:
~ Open socket. exec 3<>/dev/tcp/172.26.0.25/9991 #~ Send msg.
Is there some type of functional way to read things in the Python shell interpreter similar to less or more in the bash (and other) command line shells?
Example:
Code:
>>> import subprocess >>> help(subprocess) ... [pages of stuff to read] ...
I'm hoping so as I hate scrolling and love how less works with simple keystrokes for page-up/page-down/searching etc.
I am trying to create a shell script to automate a process that takes inputs from two sets of data. The variable SHELL in my system echoes as /bin/tcsh, but I see that my /bin directory has other possibilities, such as bash. So, if changing the shell could simplify my problem, then I would go with that.
I have a list of files containing data about objects, say obj1.dat, obj2.dat, .... objn.dat, where n is around 20 (it varies). These objects belong to a few different categories, and there are files detailing the properties of the categories, say cat1.prop, cat2.prop, ... catm.prop, where m is smaller than n (more than one object in each category).
I know that shell scripts can handle 2-d arrays, although I am not sure how. The thing here is that category 1 could have 5 objects and category 3 could have 7 objects. So, this could produce a 2-d array, but not a rectangular one. I need to process these files taking into acount the data about the object and the properties of the category the object belongs to. Here is what I tried to do:
# First I enter a list of category names and m lists of objects in each category category='cat1 cat2 cat3 ... catm' cat1-obj='obj1 obj2 obj3' cat2-obj='obj4 obj5' cat3-obj='obj6 obj7 obj8 obj9 obj10' . . . catm-obj='objx objy objz objn'
# The lines above show the irregular 2-d array. Then I try to trick the script into parsing through this irregular 2-d array: for i in $category do data='$i -obj' for j in $data do mycommand $j.dat $i.prop > $j-$i.out done done
The line "mycommand $j.dat ..." is just the line that processes the information. That has been tested to work fine. Feeding the information to this is what does not work. The lines " data='$i-obj' " and " for j in $data " do not have the desired effect. I have tried other ways, like data=`echo $i "-obj"` and things of the sort, by to no avail.
1. How do I create an array with 5 entries 2. Output one of the entries in the array randomly 3. Subtract the entry output from the array 4. Repeat steps 2 and 3 until there are no more entries in the array
Code: SERVERS=(SERVER1 SERVER2 SERVER3) SERVER1_SERV=(web ftp mail) SERVER2_SERV=(web transcoding) SERVER3_SERV=(web ftp mail) for SERVER in ${SERVERS[@]} do echo "Starting tranfer for server $SERVER" for SERVICE in ${$SERVER_$SERVICE[@]} do something_to_be_done fi done But when I run it I get ${$SERVER_$SERVICE[@]}: bad substitution
I have two arrays of data, called data1.dat and data2.dat. each contains 60 data. What I want to do is to compare the data in each file and write the counting into bins. It goes like this. First, take the first data in data1.dat file and compare with the 60 data in data2.dat file. If there is any data which is same with the data in data1.dat then it count in bin. The total bins are also 60. Next it goes to the second data in data1.dat and compare with all the 60 data in data2.dat. If there is any data same then it add in second bin. And it repeats to all the data in data1.dat
I have two files of data with different numbers of columns and rows. I want to read this two files in two arrays and then compare for example the second column of first data file with the third column of the other text file and if the difference between of two numbers is less than a threshold then the program print the information in the rows which fulfill this condition in the third text file. I have written below program but the problem is that it does not go through all rows of second file.
declare -a a declare -a b r=` awk '{n++} END {print n}' second.txt ` echo $r awk ' {
[Code]....
Actually I have two data files one of them contains 44406 and the other one has 12066 rows and I want to check whether the difference between the components of two specific columns is less than a threshod but I have simplified it here. I had written this code and then I have realized that this code just goes through the number of rows as the same as the first.txt file and ignores the rest. I could not find the problem yet.
I'm new to UNIX scripting; I�m stuck with the following I have an Oracle SQL script that takes three parameters
1- File Name 2- File Path 3- File creation date
Under UNIX I have a folder where files will be placed frequently and I need to upload those files to Oracle, what I need is a UNIX script that can do the following
Loop through Directory "/home/applmgr/snktmp" Picks only files Pass the file name to parameter &1
[code]....
Is the above possible? I already knows how to call the Oracle Script from UNIX Im only stuck on writing the UNIX part where it List the files attribute(name,path,date) and store them to parameters ,Looping until the last file in the directory If the above is not possible,then how can I create the below from the command line
I'm trying my hand at arrays in bash for a backup script. Now I not sure if this is the correct thing to do and just look at website and amended but does'nt work. Could someone tell me where I'm going wrong
test=(1 2 3 4 5) for car in ${test[@]} do echo "Element : $car"
[code]....
if variable $car equals 1, new element is added "6" to an array. But i don't know why when i am printing all elements of this array (echo "Element : $car") this element ("6") is not mentioned, but if i make a command which check an amount of contained elements by array it will be 6 elements.
I have the following function that does not iterate through the array I want to be able to do some manipulation on each element in the array[@].it appears the below array has only one item in the array whereas i want the array to have 3 items hence the loop three times printing the message Any ideas why this is not happening ?
So I can select the data from the array like so: PHP Code: echo $data[col_name][row_num];
However when I dump the array I dont get the data I am expecting: Code: running select * from users: meta [company_name]meta [company_name] DATA [Array ( [company_name] => Array ( [0] => My company [1] => My details ) ) ]
...and returning the index of the found element in its array.
I have:
for ((i=0; i < ${#array1[@]}; i++)); do # Read each line of the file test if [[ $(eval "sed -n '$(($i+1))'p test") == *${array2[0]}* ]] stuff
I want to find the index of the found substring in array2 and only if it isn't found, move on to the next element of array2. I don't know the size of array2 so that [0] has just got to go.
I need to know how to assign a result from a select. I am clueless on the sytax. I am trying this in bash. Maybe I am not assigning the array right. It gives me the whole row in the echo instead of just field a. How do I get fieldA = a in the select. Note script was stripped for security on database info but the syntax is same.
Code:
#!/bin/sh results="$(mysql --user ${DB_USER} -p${DB_PWD} ${DB_NAME} -Bse 'select a,b,c,d from tblMytable')" for rows in "${results[@]}" do fieldA=${rows[0]}; echo ${fieldA}; done
