Programming :: Bash Script - Merge Files Comparing Columns
Apr 13, 2011
I have 2 very long files which are quite similar:
file1.csv:
Code:
file2.csv:
Code:
I need this output:
Code:
explanation:
If the fields 3 (serie) and 4 (modello) are identical, the lines of the 2 files should be "added":
Code:
If the fields 3 (serie) and 4 (modello) are different, just print the line of both files:
Code:
Is there a way to do this without reading the (huge) files with a "while read line" loop?
View 14 Replies
ADVERTISEMENT
Dec 20, 2010
I want to merge columns (selectively) from several files and create a new file with the merge output. I saw some suggestions to use pr/paste to join the columns and then awk to pick-up the columns.
Code:
pr -m -t -s file1 file2 | gawk '{print $4,$5,$6,$1}'
But I have hundreds of files and I cannot manually pick up columns using awk as given in
[code]....
View 14 Replies
View Related
Sep 2, 2010
I want to compare two files and display values unique to file1. I tried using comm but it did not give me useful outputs.
comm -2 -3 file1 file2 does not work
Similar threads provide matched content. What I am looking for is unique content.
View 9 Replies
View Related
May 7, 2011
I have two files
Code:
we have to consider here $5 , $6 , $7 for our search
file2.txt
Code:
Here we should take only $2 for comparison. As you can most of the $2 field records has value and some do not have value.
Question:I want to take the fields $5 , $6 , $7 from file 1 and compare it with $2 field from file 2. and the rsult should be like this:
Code:
The final output will look like this
Actual file1.txt (before running the code)
Code:
FIle1.txt after running the above said condition
Code:
So the field $5 , $6 , $7 should get replaced from the matched valued of $1(file1)
View 3 Replies
View Related
Nov 30, 2010
I have two text files i want to compare the differances between but i dont wnat all of them, there is only about 30lines of relvent text i want to compare.
View 10 Replies
View Related
Oct 9, 2009
I would like to ask opinion from perl experts.
I want to compare 2 files and show the differences in a text file.
For example, if i open File A an B in notepad
File A:
File B:
Quote:
line3 is missing in File B
So if I did a File compare (line by line), the differences will be in line3, line4, line5, line 6.
But I dont want it to be like that.
I want it to be like this
Quote:
Can Text:: Diff able to perform the work.
View 1 Replies
View Related
May 26, 2011
I've been hitting my head against a wall for awhile with this one:As the last part of some data analysis I performing I would to construct a matrix from a series of different files. These files have the format:
file 1 file 2 file 3
AAAAA .1 AAAAA .1 BBBBB .1
BBBBB .2 BBBBB .1 CCCCC .9
[code]...
View 6 Replies
View Related
Jun 9, 2009
I have been messing with diff and grep for 2 days now without result
I am trying to match a file consisting of words to many separate other wordfiles in a specific directory. one by one.
What i want the script to do is to report how many matching words my main file has with every file in the directory, each in turn
setup:
Each of em are plain text files with 1 word per line
Output should be something like:
SCRIPT REPORT:
View 8 Replies
View Related
Jun 2, 2010
Is there a way to compare an array in a while conditions?
I have one array that contains the results of some search and if the script has found all the items, then it should stop, so my idea is to have a while loop � la:
Code:
View 4 Replies
View Related
Jul 6, 2010
I'm a bit new to Python programming and hoped that someone might be able to help with a problem I'm having. What I essentially want to do is to combine two text files line for line. I know how to do this in a bash script so to give you a better idea here's the code for that:
Code:
This is basically for adding on values to the end of a CSV file that uses ';' as the delimiter. So say file1 said:
And file2 said:
Then running this command would create merged_file1_and_file2 which would be:
The code I'm using at the moment is:
Code:
As I'm sure any experienced python programmer will see, this prints out the first line of the file "csvraw" and then all of the lines of "stamps" and then the remainder of "csvraw".
What I'd like to do is something like: (pseudo code, I know it's not python ;-))
Code:
Is this possible? I've tried googling and my Python Pocket Reference hasn't been much help. I've looked at pickling but that doesn't seem appropriate.
View 1 Replies
View Related
Jul 29, 2011
I would like to transform rows into columns from a file in bash.
View 4 Replies
View Related
Sep 28, 2009
as a result of a find command, i have
852065 72: /bin/gunzip
852065 72: /bin/gzip
852065 72: /bin/zcat
(the first column is inode number and the second is size if you're curious)
I want to be able to format it in a way such that:
852065 72:
/bin/gunzip
/bin/gzip
/bin/zcat
I know I can get the bottom half using awk- but I can't figure out how to extract the first set of numbers and get them to appear only once
View 9 Replies
View Related
Mar 12, 2011
comparing multiple items or conditions using 'if' statements? I want to do something if one or more conditions is true, for example:
If a = 1 or b = 1.
I've tried:
[Code]...
View 5 Replies
View Related
Jul 27, 2011
I've searched everywhere and I can't come up with a good solution. For each line I need to find the average, min, and max. I've seen plenty of solutions where the number of columns is fixed, unfortunately for me these lines can get pretty large. My thought was to read each line individually into an array, loop through the array and find the avg, min, and max that way but i haven't had much luck. I can read each line using a while loop but I'm having trouble with the array part, or perhaps that's not the best solution?
View 14 Replies
View Related
Mar 31, 2011
I need to rename the resulted searched files from a loopI have the following code:
find . -name DOC* | while read i
do
find $i -type f -name '*.txt'
done
basically, I am searching for all txt files inside any folder starting with DOC name.this code is working fine with me.I need to rename those .txt files to .txtOLDOS: Ubuntu 10.4Bash shell
View 10 Replies
View Related
Mar 20, 2010
I am fairly new to Linux and was needing some help on a comparing more than 2 files. I am try to come up with something that would compare at least 10+ different files to a master file and give me an output of what is missing.
Example would be: a.txt, b.txt, c.txt, d.txt compare each of them to the master.txt file, than output the missing text for each file into new file.
I came across comm and diff commands, am I looking in the right place or is there a much easier way of doing this?
View 2 Replies
View Related
Jul 12, 2011
I have a file1:
Code:
$ cat PF(1).out
Tmp39 PF10271.3 423 ENSP00000326063 488 1.2e-201 41-478
Tmp39 PF10271.3 423 ENSP00000338165 492 1.9e-200 46-479
[code].....
View 14 Replies
View Related
Jul 7, 2010
What's the command for comparing files? I think I've used it before, but can't remember it.
View 3 Replies
View Related
Sep 19, 2010
If I have a variable, say xvar, which can take both string and integer value and I want to perform an operation in following 2 conditions:1. Either xvar is null2. xvar equals 2[ -z "$xvar" ] || [ $xvar -eq 2 ] && <some-code>Doesn't seem to work if xvar takes string valuesI know that since I have no restriction on xvar, I can get away with string comparison in second test too, ie[ -z "$xvar" ] || [ "$xvar" = "2" ] && <some-code>
But, 'Sams teach yourself shell script in 24 hrs' says that [ expr1 -eq expr2 ], if either is string, it assumes 0 valueIs it true
View 5 Replies
View Related
Nov 5, 2010
Is there a way, besides writing a PERL program, to read each line one by one in file A and tell if this line also exists in file B? Can this be done via a shell script?
View 6 Replies
View Related
Sep 18, 2010
I wish to compare all the files in a directory....comparing is by content. And same files should be printed...
echo "program : $0"
cd $1
for a in *
do
[code]....
This logic did not give the correct result...
View 5 Replies
View Related
Jul 26, 2010
Iam having the following two linux files.
[Code]...
Now i want the following out puts
1. similar nos in both the file 1 and file 2 > output= File 3;
2. In file 1, but not in file 2 > out put= file 4;
3. In file 2, but not in file 1 > output = file 5;
The command sdiff is giving output with symbols > < | etc, and the such output file is not clear and ready to print. I want to print directly the output files. AND ALSO TELL ME WHERE I HAVE TO WRITE AWK PROGRAMS AND HOW TO RUN IT.
View 5 Replies
View Related
Feb 22, 2011
I am using ffmpeg for merge wav files to a mov video. My doing is below
1. First extract audio (wav file) from video
2. Create wav file from mp3 track 1
3. Create wav file from mp3 track 2
4 Merge extract audio from video with track 1 and track2.
Now finally create a new video with original video's video stream and merged audio stream.
Process is working. However final video is 3-4 times greater than original one. I want that final video should be near about size of original video. As I understand, all three wav files (created from ) make video larger.
Example commands i using is as below:
View 6 Replies
View Related
Apr 1, 2011
have a simple script for comparing two directories. I want to list all differences between this directories.
here is my function for compare:
function comp
{
for i in $1/*; do
if [ ! -e "$2/${i##*/}" ]; then
echo $i
[Code]....
in my script these two files are equal(my script ignored last time modified)
View 5 Replies
View Related
Mar 25, 2011
I want to compare 2 IP addresses, so that I may compare which is more/less "specific" or "restricted" than the other. So is there any function/library that may help in doing this comparison in C (on Ubuntu 10.10)?
View 1 Replies
View Related
Oct 11, 2010
I need a script which can format the below text file which contains comments
Code:
Code:
Output should be:
Code:
Code:
Script should compare the column name and paste the output in above said manner.
View 13 Replies
View Related
May 16, 2010
I have two arrays of data, called data1.dat and data2.dat. each contains 60 data. What I want to do is to compare the data in each file and write the counting into bins. It goes like this. First, take the first data in data1.dat file and compare with the 60 data in data2.dat file. If there is any data which is same with the data in data1.dat then it count in bin. The total bins are also 60. Next it goes to the second data in data1.dat and compare with all the 60 data in data2.dat. If there is any data same then it add in second bin. And it repeats to all the data in data1.dat
View 3 Replies
View Related
Oct 24, 2010
How can I add columns to the right of GtkTreeView? How can I add the menu to the right of the window? How can I change the position of the icon in the GnomeMessageBox to the right of the dialog? And how can I change the arrange of the buttons from right to left in GnomeMessageBox? and position of the icon on the buttons in the GnomeMessageBox?
View 5 Replies
View Related
Jul 1, 2010
I have a file that contains a couple of email addresses and I want to extract the usernames ( Letters before @ symbol ). How can I do that using sed/awk.
I know cut will work, but the current environment doesn't allow me to use cut command. I can use either awk or sed.
View 4 Replies
View Related
Jun 13, 2010
I've got an interesting challenge for the shell scripting wizards here. I've got a mySQL dump of three files for my amarok database with the intention of copying some files to my media server (cover art) so that I can keep the server the server and not rely on my local machine.
Step 1: Identify any cover art files on my local machine.
I did this with:
Code:
mysql -u amarok -p amarok -e "SELECT * FROM images WHERE path like '%.kde%'" > cover_art.txt
Output looks like this:
[Code]....
What I have here now is the ENTIRE album list in my collection -- and something to compare the IDs in Step 1 against. I'm going to stop here and will update the thread as I get past this stumbling block. "ID" in cover_art.txt = "image" in albums.txt... straightforward enough, right?
So the question is this: how do I create a simple shell script that will loop through the IDs in cover_art.txt (i.e. characters 0 -> 4 -- it will always be a 4 digit ID) and then search for that ID in the Albums.txt file.
View 3 Replies
View Related