General :: Column Manipulation In Bash Script
Mar 19, 2010
I have a text file and I need to replace the 3rd column of that file from row 3 to the end of the file with a column which I have stored in the different text file. For e.g the original file is like given below:
a.txt nobla 6 gadf 72.500 1.600 1.800 .850 5.250 8.540
A# rad ang ht prf bk sd dia type blade
1 0.3081 9.00 1.9235 -17.50 18.00 -3.00 0.6250 1613 1
2 0.6509 194.00 2.0316 -17.50 18.00 -3.00 0.6250 1613 4
3 1.0128 8.00 2.1457 -17.50 18.00 -3.00 0.6250 1616 1
4 1.3748 192.00 2.2598 -17.50 18.00 -3.00 0.6250 1616 4
5 1.6986 7.00 2.3619 -17.50 18.00 -3.00 0.6250 1616 1
6 1.9347 120.00 2.4364 -17.51 18.00 -3.00 0.6250 1616 5
7 2.1327 190.00 2.4988 -17.34 18.00 -3.00 0.6250 1616 4
So lets say I wanna replace column 3 from row 3 to row 7 with a data from another file which is given below:
54.00
239.00
53.00
237.00
52.00
165.00
235.0
So the final output file should be like this:
a.txt nobla 6 gadf 72.500 1.600 1.800 .850 5.250 8.540
A# rad ang ht prf bk sd dia type blade
1 0.3081 54.00 1.9235 -17.50 18.00 -3.00 0.6250 1613 1
2 0.6509 239.00 2.0316 -17.50 18.00 -3.00 0.6250 1613 4
3 1.0128 53.00 2.1457 -17.50 18.00 -3.00 0.6250 1616 1
4 1.3748 237.00 2.2598 -17.50 18.00 -3.00 0.6250 1616 4
5 1.6986 52.00 2.3619 -17.50 18.00 -3.00 0.6250 1616 1
6 1.9347 165.00 2.4364 -17.51 18.00 -3.00 0.6250 1616 5
7 2.1327 235.00 2.4988 -17.34 18.00 -3.00 0.6250 1616 4
And I will post the code whatever i have tried soon. I started with awk and cut commands..but never got it to work and also tried PASTE command too.
View 3 Replies
ADVERTISEMENT
Mar 7, 2010
i use this script to get the time and date of back and fourth transactions for a particular execution id. I use a substr command on the 5th column to to cut the milli seconds off the time value. - otherwise the times would look like 08:30:04.235
grep <executionID> <auditfile> | awk '{ print $1, $2, $3, $4, substr($5,1,8}
FIX -> Mon 3/1/2010 08:30:04
FIX <- Mon 3/1/2010 08:32:36
FIX <- Mon 3/1/2010 08:35:08
[code].....
anyhow - i append two sed commands to further clarify the direction of the message.
awk '{ print $1, $2, $3, $4, substr($5,1,8} | sed -e 's/->/ ->IN/g' | sed -e 's/<-/<-OUT/g'
FIX -> IN Mon 3/1/2010 08:30:04
FIX <- OUT Mon 3/1/2010 08:32:36
[code]....
I tried using an awk gsub () command within the string instead of the two seds, but it did not work:
awk '{ print gsub(/<regex>/, <replace with>,$1), $2, $3, $4, substr($5,1,8}
the sed works ok, but it would be cooler to make the replacement within the awk command:
gsub(/->/,-> IN, $1)
Is there a way where i could replace the value of the $1 column in the awk print string?
View 1 Replies
View Related
Sep 2, 2010
I have been using comm to compare two simple column lists, and suppress items that were contained in the second list (suppression list). This was extremely simple and basic, however now list1 has two columns, and I must compare the second column in list1 with my suppression list.
Basically I need to compare my user list and suppression list to suppress any users that exist in the suppression list, then remove the second column (md5).I wasn't sure the fastest way to make comparisons if there was a similar command like comm, or if I needed to create an array of users and see if any of them matched the suppression list one by one. This seemed like it would be pretty process intensive. Anyone have any less cumbersome ideas?
View 6 Replies
View Related
Nov 7, 2010
I would like the ability to manipulate a jpg (bmp, gif and raw would be nice as well) while in BASH. To be more specific, I would like to either add a water mark or a little section as a footer on the bottom of an image.
I will be using this as part of a shell script to alter files in a directory on my webserver.
View 2 Replies
View Related
Jul 5, 2010
I have a bash script that inserts some text onto every image at a certain place within a directory.
Heres what I have (from a German friend who appears online once in a blue moon), this is the line that resizes to a maximum of 800 either width or height and puts in the text 'text goes here'.
convert ${bild} -resize "800x800>" -strip -family Arial -pointsize 16 -draw "gravity southeast fill rgba(255,0,0, 1.0) text 10,10 'text goes here'" ${dstdir}/${bild}
Now, I would like to know how to place a PNG image (a watermark, so to speak) over all images within a directory in a certain place, so how would I go about modifying this line to place an image instead of the text?
View 9 Replies
View Related
Nov 26, 2010
I have a little problem about string in Snort alerting. I understood about Snort alerting saved in /var/log/snort/alert and Snort will add a new entry if there was a attack from anywhere. Then here's my problem. Because it has a lot of file on it, all I want to do is parse that string in snort alert then make into simply log files with it. I'm getting confused with Snort alert and parse that file.
Here's the simple algorithm; Snort get the alert <- parse the alert with my parameter which I've configured with bash (ip address, dest, kind of attack and time) <- then sent that parse alert into new text (let's called snortsent.txt) <- after ten alerts then clear the text then waiting again until the Snort alert go on -> back to snort alert. Here's the sample of my snort alert: (/var/log/snort/alert)
[Code]....
View 16 Replies
View Related
Jan 6, 2010
I am trying to do this:
1. Read csv delimited file line1 and store all values in array
2. Use the values stored in the array and replace values in other text file with them
3. read line2 in the cvs file and repeat the process
4. Do above for all lines in the cvs file
for example:
file1.cvs content:
text1,text2,text3,"text 4"
a1,a2,3,"a 4"
file.txt content:
some text $array1$ some text
some text $array2$ some text
1. read line 1 - text1,text2,text3,"text 4" put each value in array X[] lines that contain spaces in cvs will have double quotes
2. read x[1] and replace value $array1$ (in file.txt) with x[1]read x[2] and replace value $array2$ (in file.txt) with x[2] and so on
Can above be accomplished in BASH and how?
View 1 Replies
View Related
Oct 10, 2009
I need to do some text file manipulation which I think should be done with standard commands in BASH. I'm looking at comma seperated text files (stock market data). It comes in the form of date, stock code, open, high, low, close, volume. What I need to do first is move all data with same stock code sequentially into individual files.
While doing this since the stock code will now be the file name I need to remove the stock code. Next I need to filter out overlapping data from different files with the same date. ie. where two files contain the same date on the one line only one line will be added to the combined file. I think there must be a tutorial out there for basic text manipulation like this, I just haven't found it yet.
View 11 Replies
View Related
Jul 19, 2010
How would i calculate following values.Initial file
10 3
20 4
How would i calculate 3rd column which should be addition of value in 1 and 2nd column.File after calculation
10 3 13
20 4 24
View 14 Replies
View Related
Sep 4, 2010
Code:
#!/bin/bash
ls -lhGg | while read line; do echo "$line"; done | awk ' { print $3" "$6 } '
what i want to do is be able to print column 3 and every column greater then 5. Has to be to the end of the line, since different filenames can have different amounts of words in them and the blank space is the separator. my current code works just fine if the file has no blank space.
View 8 Replies
View Related
Jul 20, 2011
I have a performance report that provides all the information I need to report the following: total transactions per day, average transactions per second, and peak transactions per second. It just doesn't provide any of it in a very accessible manner, so I want to parse on it and just capture the bits I care about. Ideally, I'd like the output to look something like this:
Code:
Date Total Avg Peak
07/11/11 12,328,033 24.05 64
07/12/11 9,328,429 21.98 56
The problem is the format of the input file, which is somewhat complicated. The report gives a summary of all transactions within any given second, and then totals at the end of each day, with page breaks in the middle, like so:
[Code]...
So first, the easy part that takes me to the daily summary, which gives me the date, the total transactions, and I can divide the total by 86400 to get the average per second, too. No problem. It's the last part that's got me stumped... the daily peak. I can't just do a while loop on the date, because it's missing from most of the records. And it also means I can't use positional parameters, because depending on the page break, the total will move between $2 or $3. And I need the date as a conditional to find the daily peak, because this output will have many days' worth of data.
Any ideas? Some kind of awk or sed command to insert the date wherever it's missing (I'm not particularly good at either utility)? Is there a method to parse these things based on column location that I'm not aware of?
View 12 Replies
View Related
Jul 15, 2010
I have a multicolumn datas, like
a1 b1 ... f1
a2 b2 ... f2
. . ... .
[code]...
I would like to make a file with all these data in one column, like
a1
a2
.
.
[code]....
Can it be done with awk or some other command? Also, is it possible then do add another column in front of this one with numbers of the lines (for every previous column), like
1 a1
2 a2
. .
. .
[code].....
View 14 Replies
View Related
Dec 7, 2010
Let me *try* and explain what I'm trying to do, and keep in mind aside from a little command line stuff I'm a beginner to any of what I'm asking about.
I have a directory structure something like this:
Code:
/usr/local/chatlogs/
webserver1/
1.chatlog
2.chatlog
[Code]....
So that whatever was captured in the () in the first part of the statement would be used in the 1 in the back part of the statement for every n.chatlog that might be in any of the /webserver directories at that time.
View 2 Replies
View Related
May 15, 2010
What is the best way to copy a file from linux or unbuntu, to microsoft? Is there a way to do so?
View 4 Replies
View Related
Jun 1, 2010
I have a string like $(SOMETHING) and want to remove '(' and ')' in a script file. How can I do that? I read that sed command is a filtering utility but don't know how to use it.
View 12 Replies
View Related
May 7, 2010
I have a number of text files (26 per database x 100+ databases) which need 'correcting' in order to import into postgresql. I think that I have identified all the problem characters and I need to automate the process as much as possible. I have a script to convert the characters and I do them one by one (not effecient but easier to understand).
[Code].....
View 2 Replies
View Related
Sep 17, 2010
Does anyone know a free, command line, open source audio manipulation software for Linux? The primary usage is to distort/scramble a recorded voice. I've tried SoX but it seems to lack features for my task.
View 1 Replies
View Related
Jan 24, 2010
find and use the file-manipulation tools for the command line.
View 2 Replies
View Related
Apr 9, 2010
What is file-manipulation tool used for on the Linux command line?
View 7 Replies
View Related
May 14, 2010
How do I find the file-manipulation tools on the command line?
View 5 Replies
View Related
Aug 27, 2010
What I have works, but wondering what is the 'right' way to replace the digits with the letters given in this loop? somehow use a case or multiple sed? i thought of a multiple sed or a case but couldn't get it to work
Code:
# ...
bcv=$(echo $line | awk -F" " '{ print $1 }' | sed 's/1/q/g;s/2/w/g;s/3/e/g') # and so on
Code:
while read line
do
bcv=$(echo $line | awk -F" " '{ print $1 }')
if [ $bcv == "" ]
[code].....
View 12 Replies
View Related
Jan 18, 2011
I have a two column file and I need to create a new column in between the first and second but the new column adds a value to the first. E.g.code...
I thought I had figured out how to do it with the following but it just hangs:
awk -F '{print $0,$0+25,$1}' file_in > file_out
Also tried the following to no avail:
awk -F,-v OFS,'{print $0,$0+25,$1}' file_in > file_out
I can add the new column with the added value to the last column easy ( awk '{print $0,$0+25}' file_in > file_out).
View 8 Replies
View Related
Feb 23, 2011
I have not defined a user vimrc, OS is redhat 4.6 and after a search and replace the first column in the editor is highlighted yellow and it stays that way as I close and open vim. This is the /etc/vimrc that came on the system. anyone see a bug or a reason it would do that?
if v:lang =~ "utf8$" || v:lang =~ "UTF-8$"
set fileencodings=utf-8,latin1
endif
set nocompatible " Use Vim defaults (much better!)
[Code]....
View 2 Replies
View Related
Oct 20, 2010
1. when we run od command it displays octal values. But the first column will be always 000000. after that the actual file contents are displayed. Can anyone tell the meaning of that.
2. When we run ls -l command, in the first line of the output, we can see some integer value. What is the significance of that value.
View 4 Replies
View Related
Apr 26, 2010
I have some series of files, which actually named bellow
<day_month_yr>_<hh:mm:ss>.<host_name>.<IP.ip.ip.ip>.<log_name>.txt
I want to show this file name below the column may be
Date Time Host_Name IP_address Login_Name
----- ---- --------- ---------- ----------
<day_month_yr> <hh:mm:ss> host_name x.x.x.x log_name
...
View 5 Replies
View Related
Jul 12, 2010
Im trying to use awk to do matching in only a specific column
example.txt:
www.google.com www.example.com
www.google.com/search www.example2.com
i used:
awk '{ (if $1 == "http://www.google.com") print $2}' example.txt
this awk statement only returns the first line, and i cant seems to make it perform in a way to match based on keywords like GREP. Is there any way to make display the other lines which contains "google" also?
View 1 Replies
View Related
Feb 22, 2010
It is very important for my research work. For example mydata.txt:
id type x y z
1 6 0.474611 0.227223 0.583947
2 4 0.422894 0.22726 0.536791
3 5 0.448963 0.200148 0.560336
4 3 0.386478 0.207721 0.515293
5 6 0.371617 0.22361 0.582206
6 4 0.32123 0.222999 0.534782
How to change second column (type) whose values are 4 and 3 to value 1, so that mydata.text file become:
id type x y z
1 6 0.474611 0.227223 0.583947
2 1 0.422894 0.22726 0.536791
3 5 0.448963 0.200148 0.560336
4 1 0.386478 0.207721 0.515293
5 6 0.371617 0.22361 0.582206
6 1 0.32123 0.222999 0.534782
View 6 Replies
View Related
May 20, 2011
I have a basic question of awk.
Code:
var=/test/build/create/sls
echo $var | awk '{ FS = ":" ; print $NF }'
/test/build/create/sls
I am trying to extract last column from the above awk one liner.
View 4 Replies
View Related
Jul 28, 2011
I want to use awk to modify file like this
origin:
A[]A[][]A[]A
modified:
A[]a[][]A[]A
but when I use
awk '{$2="a"; print $0}' inputfile
the output is
A[]a[]A[]A
where the [] means space. This is not what i want. I guess that is because the OFS is one space by default, but i really don't know how to solve this.
View 11 Replies
View Related
Jun 21, 2011
I need to replace a value in a file. For example the content of data.txt file is:
1 1 23
2 1 42
3 2 52
4 2 62
5 1 77
6 1 88
7 2 99
8 1 100
Could I substitute 2 in second column with 3 using awk and or sed or other command so that the data will be change as follow?
1 1 23
2 1 42
3 3 52
4 3 62
5 1 77
6 1 88
7 3 99
8 1 100
View 3 Replies
View Related
