Programming :: Bash Script - Merge Files Comparing Columns

Apr 13, 2011

I have 2 very long files which are quite similar:

file1.csv:

Code:

file2.csv:

Code:

I need this output:


Code:

explanation:

If the fields 3 (serie) and 4 (modello) are identical, the lines of the 2 files should be "added":

Code:

If the fields 3 (serie) and 4 (modello) are different, just print the line of both files:

Code:

Is there a way to do this without reading the (huge) files with a "while read line" loop?

View 14 Replies


ADVERTISEMENT

General :: Merge Columns From Multiple Files?

Dec 20, 2010

I want to merge columns (selectively) from several files and create a new file with the merge output. I saw some suggestions to use pr/paste to join the columns and then awk to pick-up the columns.

Code:
pr -m -t -s file1 file2 | gawk '{print $4,$5,$6,$1}'
But I have hundreds of files and I cannot manually pick up columns using awk as given in

[code]....

View 14 Replies View Related

General :: Comparing Files In Bash Script

Sep 2, 2010

I want to compare two files and display values unique to file1. I tried using comm but it did not give me useful outputs.

comm -2 -3 file1 file2 does not work

Similar threads provide matched content. What I am looking for is unique content.

View 9 Replies View Related

Programming :: Comparing Two Files ?

May 7, 2011

I have two files

Code:

we have to consider here $5 , $6 , $7 for our search

file2.txt

Code:

Here we should take only $2 for comparison. As you can most of the $2 field records has value and some do not have value.

Question:I want to take the fields $5 , $6 , $7 from file 1 and compare it with $2 field from file 2. and the rsult should be like this:

Code:

The final output will look like this

Actual file1.txt (before running the code)

Code:

FIle1.txt after running the above said condition

Code:

So the field $5 , $6 , $7 should get replaced from the matched valued of $1(file1)

View 3 Replies View Related

Programming :: Differances - Comparing Two Files ?

Nov 30, 2010

I have two text files i want to compare the differances between but i dont wnat all of them, there is only about 30lines of relvent text i want to compare.

View 10 Replies View Related

Programming :: Comparing Two Files In Perl ?

Oct 9, 2009

I would like to ask opinion from perl experts.

I want to compare 2 files and show the differences in a text file.

For example, if i open File A an B in notepad

File A:

File B:

Quote:

line3 is missing in File B

So if I did a File compare (line by line), the differences will be in line3, line4, line5, line 6.

But I dont want it to be like that.

I want it to be like this

Quote:

Can Text:: Diff able to perform the work.

View 1 Replies View Related

Programming :: AWK (or TCL/TK): Matching Rows And Columns Between Multiple Files?

May 26, 2011

I've been hitting my head against a wall for awhile with this one:As the last part of some data analysis I performing I would to construct a matrix from a series of different files. These files have the format:

file 1 file 2 file 3
AAAAA .1 AAAAA .1 BBBBB .1
BBBBB .2 BBBBB .1 CCCCC .9

[code]...

View 6 Replies View Related

Programming :: Bash File Comparing - Report How Many Matching Words My Main File ?

Jun 9, 2009

I have been messing with diff and grep for 2 days now without result

I am trying to match a file consisting of words to many separate other wordfiles in a specific directory. one by one.

What i want the script to do is to report how many matching words my main file has with every file in the directory, each in turn

setup:

Each of em are plain text files with 1 word per line

Output should be something like:

SCRIPT REPORT:

View 8 Replies View Related

General :: Comparing Arrays In Bash ?

Jun 2, 2010

Is there a way to compare an array in a while conditions?

I have one array that contains the results of some search and if the script has found all the items, then it should stop, so my idea is to have a while loop � la:


Code:

View 4 Replies View Related

Programming :: Using Python To Merge Two Text Files Line For Line

Jul 6, 2010

I'm a bit new to Python programming and hoped that someone might be able to help with a problem I'm having. What I essentially want to do is to combine two text files line for line. I know how to do this in a bash script so to give you a better idea here's the code for that:

Code:

This is basically for adding on values to the end of a CSV file that uses ';' as the delimiter. So say file1 said:

And file2 said:

Then running this command would create merged_file1_and_file2 which would be:

The code I'm using at the moment is:

Code:

As I'm sure any experienced python programmer will see, this prints out the first line of the file "csvraw" and then all of the lines of "stamps" and then the remainder of "csvraw".

What I'd like to do is something like: (pseudo code, I know it's not python ;-))

Code:

Is this possible? I've tried googling and my Python Pocket Reference hasn't been much help. I've looked at pickling but that doesn't seem appropriate.

View 1 Replies View Related

General :: Transform Rows Into Columns In Bash

Jul 29, 2011

I would like to transform rows into columns from a file in bash.

View 4 Replies View Related

General :: Parsing Columns In Bash / Extract The First Set Of Numbers And Get Them To Appear Only Once?

Sep 28, 2009

as a result of a find command, i have

852065 72: /bin/gunzip
852065 72: /bin/gzip
852065 72: /bin/zcat

(the first column is inode number and the second is size if you're curious)

I want to be able to format it in a way such that:
852065 72:
/bin/gunzip
/bin/gzip
/bin/zcat

I know I can get the bottom half using awk- but I can't figure out how to extract the first set of numbers and get them to appear only once

View 9 Replies View Related

General :: Bash Scripts - Comparing Multiple Items Or Conditions Using 'if' Statements?

Mar 12, 2011

comparing multiple items or conditions using 'if' statements? I want to do something if one or more conditions is true, for example:

If a = 1 or b = 1.

I've tried:

[Code]...

View 5 Replies View Related

General :: Bash Script To Read Csv File With Multiple Length Columns

Jul 27, 2011

I've searched everywhere and I can't come up with a good solution. For each line I need to find the average, min, and max. I've seen plenty of solutions where the number of columns is fixed, unfortunately for me these lines can get pretty large. My thought was to read each line individually into an array, loop through the array and find the avg, min, and max that way but i haven't had much luck. I can read each line using a while loop but I'm having trouble with the array part, or perhaps that's not the best solution?

View 14 Replies View Related

Programming :: Bash Programming - Rename Files In A Loop?

Mar 31, 2011

I need to rename the resulted searched files from a loopI have the following code:

find . -name DOC* | while read i
do
find $i -type f -name '*.txt'
done

basically, I am searching for all txt files inside any folder starting with DOC name.this code is working fine with me.I need to rename those .txt files to .txtOLDOS: Ubuntu 10.4Bash shell

View 10 Replies View Related

General :: Comparing Multiple Of Files ?

Mar 20, 2010

I am fairly new to Linux and was needing some help on a comparing more than 2 files. I am try to come up with something that would compare at least 10+ different files to a master file and give me an output of what is missing.

Example would be: a.txt, b.txt, c.txt, d.txt compare each of them to the master.txt file, than output the missing text for each file into new file.

I came across comm and diff commands, am I looking in the right place or is there a much easier way of doing this?

View 2 Replies View Related

General :: Comparing Two Fields In Two Files Using Awk?

Jul 12, 2011

I have a file1:

Code:

$ cat PF(1).out
Tmp39 PF10271.3 423 ENSP00000326063 488 1.2e-201 41-478
Tmp39 PF10271.3 423 ENSP00000338165 492 1.9e-200 46-479

[code].....

View 14 Replies View Related

Software :: Command For Comparing Files?

Jul 7, 2010

What's the command for comparing files? I think I've used it before, but can't remember it.

View 3 Replies View Related

Programming :: Comparing String With Integer?

Sep 19, 2010

If I have a variable, say xvar, which can take both string and integer value and I want to perform an operation in following 2 conditions:1. Either xvar is null2. xvar equals 2[ -z "$xvar" ] || [ $xvar -eq 2 ] && <some-code>Doesn't seem to work if xvar takes string valuesI know that since I have no restriction on xvar, I can get away with string comparison in second test too, ie[ -z "$xvar" ] || [ "$xvar" = "2" ] && <some-code>
But, 'Sams teach yourself shell script in 24 hrs' says that [ expr1 -eq expr2 ], if either is string, it assumes 0 valueIs it true

View 5 Replies View Related

General :: Comparing Lines In 2 Text Files?

Nov 5, 2010

Is there a way, besides writing a PERL program, to read each line one by one in file A and tell if this line also exists in file B? Can this be done via a shell script?

View 6 Replies View Related

General :: Comparing All The Files In A Directory By Content?

Sep 18, 2010

I wish to compare all the files in a directory....comparing is by content. And same files should be printed...

echo "program : $0"
cd $1
for a in *
do

[code]....

This logic did not give the correct result...

View 5 Replies View Related

General :: Comparing Two Files For Differences And Similarities

Jul 26, 2010

Iam having the following two linux files.

[Code]...

Now i want the following out puts

1. similar nos in both the file 1 and file 2 > output= File 3;
2. In file 1, but not in file 2 > out put= file 4;
3. In file 2, but not in file 1 > output = file 5;

The command sdiff is giving output with symbols > < | etc, and the such output file is not clear and ready to print. I want to print directly the output files. AND ALSO TELL ME WHERE I HAVE TO WRITE AWK PROGRAMS AND HOW TO RUN IT.

View 5 Replies View Related

Ubuntu Multimedia :: Ffmpeg Audio Merge With Video Merge

Feb 22, 2011

I am using ffmpeg for merge wav files to a mov video. My doing is below

1. First extract audio (wav file) from video
2. Create wav file from mp3 track 1
3. Create wav file from mp3 track 2
4 Merge extract audio from video with track 1 and track2.
Now finally create a new video with original video's video stream and merged audio stream.

Process is working. However final video is 3-4 times greater than original one. I want that final video should be near about size of original video. As I understand, all three wav files (created from ) make video larger.

Example commands i using is as below:

View 6 Replies View Related

Programming :: Simple Script For Comparing Two Directories?

Apr 1, 2011

have a simple script for comparing two directories. I want to list all differences between this directories.

here is my function for compare:

function comp
{
for i in $1/*; do
if [ ! -e "$2/${i##*/}" ]; then
echo $i

[Code]....

in my script these two files are equal(my script ignored last time modified)

View 5 Replies View Related

Programming :: Comparing Two IP Addresses - Specific / Restricted

Mar 25, 2011

I want to compare 2 IP addresses, so that I may compare which is more/less "specific" or "restricted" than the other. So is there any function/library that may help in doing this comparison in C (on Ubuntu 10.10)?

View 1 Replies View Related

Programming :: Comparing And Formatting The Text File

Oct 11, 2010

I need a script which can format the below text file which contains comments

Code:

Code:

Output should be:

Code:

Code:

Script should compare the column name and paste the output in above said manner.

View 13 Replies View Related

Programming :: Comparing Data In Array In FORTRAN?

May 16, 2010

I have two arrays of data, called data1.dat and data2.dat. each contains 60 data. What I want to do is to compare the data in each file and write the counting into bins. It goes like this. First, take the first data in data1.dat file and compare with the 60 data in data2.dat file. If there is any data which is same with the data in data1.dat then it count in bin. The total bins are also 60. Next it goes to the second data in data1.dat and compare with all the 60 data in data2.dat. If there is any data same then it add in second bin. And it repeats to all the data in data1.dat

View 3 Replies View Related

Programming :: How To Add Columns To Right Of GtkTreeView

Oct 24, 2010

How can I add columns to the right of GtkTreeView? How can I add the menu to the right of the window? How can I change the position of the icon in the GnomeMessageBox to the right of the dialog? And how can I change the arrange of the buttons from right to left in GnomeMessageBox? and position of the icon on the buttons in the GnomeMessageBox?

View 5 Replies View Related

Programming :: Separate Columns Using Sed?

Jul 1, 2010

I have a file that contains a couple of email addresses and I want to extract the usernames ( Letters before @ symbol ). How can I do that using sed/awk.

I know cut will work, but the current environment doesn't allow me to use cut command. I can use either awk or sed.

View 4 Replies View Related

Ubuntu :: Comparing Multiple Files In Shell Script?

Jun 13, 2010

I've got an interesting challenge for the shell scripting wizards here. I've got a mySQL dump of three files for my amarok database with the intention of copying some files to my media server (cover art) so that I can keep the server the server and not rely on my local machine.

Step 1: Identify any cover art files on my local machine.

I did this with:

Code:
mysql -u amarok -p amarok -e "SELECT * FROM images WHERE path like '%.kde%'" > cover_art.txt
Output looks like this:

[Code]....

What I have here now is the ENTIRE album list in my collection -- and something to compare the IDs in Step 1 against. I'm going to stop here and will update the thread as I get past this stumbling block. "ID" in cover_art.txt = "image" in albums.txt... straightforward enough, right?

So the question is this: how do I create a simple shell script that will loop through the IDs in cover_art.txt (i.e. characters 0 -> 4 -- it will always be a 4 digit ID) and then search for that ID in the Albums.txt file.

View 3 Replies View Related







Copyrights 2005-15 www.BigResource.com, All rights reserved