Programming :: AWK - Combining Multiple Columns?
Feb 23, 2010
I have a folder with only 24 files named <number>.dat (i.e. 4.dat, 6.dat and so on) where <number> is between 0 and 256. Each file has just two columns of data and nothing else.
I'm trying to combine all the second columns ($2) together. I've been fiddling around with getline and so far have
awk '{ getline ln < "6.dat" ; print ln" "$2 }' 4.dat
which takes file 4.dat and adds $2 from 6.dat, but I want a single command to take each $2 from every file and add them to (for example) 4.dat (having $1 from 4.dat is no problem). A command that takes every file in the folder and grabs $2 and places them in a common file would be ideal. Frankly I can work around if you combine both columns from every file.
View 5 Replies
ADVERTISEMENT
May 26, 2011
I've been hitting my head against a wall for awhile with this one:As the last part of some data analysis I performing I would to construct a matrix from a series of different files. These files have the format:
file 1 file 2 file 3
AAAAA .1 AAAAA .1 BBBBB .1
BBBBB .2 BBBBB .1 CCCCC .9
[code]...
View 6 Replies
View Related
Nov 10, 2009
Each line of the file I am sorting is in the following format:
<url> <month> <day>
For example:
[URL]
I wrote the following to sort:
Code:
#!/usr/bin/perl
$in = shift;
chomp($in);
[code]....
The script worked fine for my small testing files, but failed in my input file. The input file is 18MB and containing more than 300,000 lines. The output will contains some lines like that:
url_one 10 1
url_two 10 1
url_three 10 3
url_four 10 1
Is that because my file is too big for perl to handle ?
View 10 Replies
View Related
Nov 7, 2010
I have multiple .xls documents that are formatted as such,
<?xml version="1.0"?>
<?mso-application Excel.Sheet?>
They are all given different filenames bob_smells.xls, nancy_smells_worse.xls etc. I was wondering using Linux (LUbuntu specifically), would it be possible to combine all of these worksheets into one document, with each of the seperate files becoming a tab / sheet within the one file (And the name reflecting the original name of the document)?
View 2 Replies
View Related
Apr 1, 2010
I have 4 interfaces, dvb0_0 - dvb0_3. Each one has a multicast stream coming in on it. The program I am using to decode these streams only accepts one interface though. How can I "combine" so that the program, listening on 1 ip can get all 4 streams? they are on groups 224.0.1.1-4
View 1 Replies
View Related
Dec 19, 2010
I was trying to do a school assignment in LaTeX. The assignment involves having the page split into three columns: the first column is for the quote, the second is for a reaction, and the third is for questions. These columns must be able to break in the middle of a row. I was trying to accomplish this using LaTeX.
First, I tried the longtable environment, but that would not allow me to pagebreak in the middle of a row. Then, I tried parcolumns, but for some reason, the second "row" had a huge space between the first two words. Does anyone know of an environment suitable for this kind of work?
View 4 Replies
View Related
Aug 18, 2010
I am having a souce file (say SOURCEFILE) as below with 5 columns and many rows.
pol name status loan address
123 aaa 21 100 RJY
234 bbb 31 200 RMY
345 ccc 21 300 SJY
456 ddd 41 400 pqr
i want to compare the file (say COMPAREFILE ) which is below :
123
345
I need the following output file (sayOUTPUTFILE) as below :
( ie for file COMPAREFILE i need column 2, column 4 ie name and loan from source file )
ie output file required is to be like below :
123 aaa 100
345 ccc 400
Till now iam using the following method.
1. cutting column 2nd column
2. again cutting 4th column
3. in third, step merging the 2nd and 4 th column.
method of writting single line command or any awk programme to do this in one step.
I need output file (say OUTPUTFILE) as below
View 1 Replies
View Related
Sep 22, 2010
I'd like to extract a single column from 5 different files and put them gether in an output file. I saw a similar question for 2 input files, and the line of code workd very well, the code is:awk 'NR==FNR{a[NR]=$2; next} {print a[FNR], $2}' file1 file2I added the file3, file4 and file5 at the end, but it doesn't work. Does anyone know what do I have to do?
View 5 Replies
View Related
Dec 20, 2010
I want to merge columns (selectively) from several files and create a new file with the merge output. I saw some suggestions to use pr/paste to join the columns and then awk to pick-up the columns.
Code:
pr -m -t -s file1 file2 | gawk '{print $4,$5,$6,$1}'
But I have hundreds of files and I cannot manually pick up columns using awk as given in
[code]....
View 14 Replies
View Related
Jul 27, 2011
I've searched everywhere and I can't come up with a good solution. For each line I need to find the average, min, and max. I've seen plenty of solutions where the number of columns is fixed, unfortunately for me these lines can get pretty large. My thought was to read each line individually into an array, loop through the array and find the avg, min, and max that way but i haven't had much luck. I can read each line using a while loop but I'm having trouble with the array part, or perhaps that's not the best solution?
View 14 Replies
View Related
Jan 5, 2011
I have a command which on the command line needs to look like this
rlam -if3 '!pvalue -H image1.jpg' > image2.jpg
Nevermind what rlam or pvalue do ... they are part of a program package I am using. The above command works on the command line, and also when written verbatim in a bash shell script.
My problem is: in the script I wish to replace image1.jpg with the content of a variable, e.g.
IM1=image1.jpg
How to I get the script to insert the value of $IM into the command when the pvalue part of it needs to be quoted?
View 1 Replies
View Related
Jun 17, 2010
I've been looking though different editors for one that has good printing support. Ideally it should be able to print C++ code with line numbers, syntax highlighting, multiple columns per page, customizable fonts and sizes and a print preview feature so that I can make sure it looks right before sending it to the printer. It appears that notepad++ had at least some of these features, but it is not available on linux. The best I could do so far is to copy/paste the output of 'cat -n foo.cpp' into oowriter and format it into two colums. I don't get synax highlighting though and I have to manually replace tabs with a few spaces as well as some excessive leading spaces before the line numbering.
View 1 Replies
View Related
Oct 24, 2010
How can I add columns to the right of GtkTreeView? How can I add the menu to the right of the window? How can I change the position of the icon in the GnomeMessageBox to the right of the dialog? And how can I change the arrange of the buttons from right to left in GnomeMessageBox? and position of the icon on the buttons in the GnomeMessageBox?
View 5 Replies
View Related
Jul 1, 2010
I have a file that contains a couple of email addresses and I want to extract the usernames ( Letters before @ symbol ). How can I do that using sed/awk.
I know cut will work, but the current environment doesn't allow me to use cut command. I can use either awk or sed.
View 4 Replies
View Related
Jun 9, 2010
Say I have a text file with10 columns. I need to reorder them based on a list of column numbers that will reorder them.
My problem is this:
If I want to cut out 5 columns (columns 1,2,3,9,10) in the order 1,10,2,9,3 then I have tried using:
Code: cut -f1,10,2,9,3 my_file.txt > reordered_file.txt But this just extracts the columns in order as if I used:
Code: cut -f1,2,3,9,10 my_file.txt > reordered_file.txt How can I cut these columns and place them into the new file in the order I specify?
While this might seem quite trivial, I will actually need to do this for a file containing ~14000 columns with ~12000 columns that I need to extract in a particular order.
View 2 Replies
View Related
Mar 1, 2011
Actually build a script which can monitor the "DateTimeHourMinute" in a data file.
I wanted to compare the "DateTimeHourMinute" from the 2nd column against "DateTimeHourMinute" on the 3rd column which located in the same line.
Below is the data sample:
View 7 Replies
View Related
Aug 10, 2009
I bet this is a Perl one-liner (or very simple python script).I have a tab separated files in which each row looks like:Unique_Eight_Character_Sequence [3 tabs] data1~moredata1~moredata1 [3 tabs] data2~ moredata2~ moredata2 ... dataN~.The output file should have each column converted into a row (with the unique character sequence copied in for the first column), and then each "~" replaced by a comma.
View 14 Replies
View Related
Nov 13, 2010
Is there any way to filter the output of a command based on the values on the output columns. For example i execute du -h on directory with many files. Now I want to filter the output based on the size (i.e. M or G or K ). The filtered o/p should contain only M(megabytes) or G(gigabytes) and also all columns.
20K ./fload/temp/20000101/Pam
15K ./fload/temp/20000101/NAVEX
29K ./fload/temp/20000101/Avenge
[code]....
View 3 Replies
View Related
Jun 16, 2010
I have a Perl script that has two arrays - they are related. I would like to print out the contents into two columns next to each other.
#!/usr/bin/perl
open(PINGFILE, </home/casper/pingdata.txt") or die " can not open file ";
my @totalfile=<PINGFILE>;
foreach $string(@totalfile) {
if ($string =~ m/(^1sping)(?=.*max))/) {
push(usecstring,"$string");
[Code]...
View 2 Replies
View Related
Jul 20, 2011
I have an array with 15 elements, and I want to break it down into three columns. When the array is split into a the three elements - however on the iteration, it does not conform to that structure.
Code:
#!/usr/bin/perl -w
use strict ;
[code]....
View 13 Replies
View Related
Jul 18, 2011
In linux terminal; how can we get the number of rows ad columns from linux kernel? I tried from environment variables(LINES,COLUMNS) but, I could not retrieve them as my editor program is a child process to linux terminal process.
View 7 Replies
View Related
Nov 27, 2010
How do I do this?The script below obviously dosn't work.
#!/bin/bash
for u in $(awk '{print $2}' user.txt)
for i in $(awk '{print $1}' user.txt)
[code]...
One text file with two columns in it, I need to iterate through the file and print the different columns each time
View 9 Replies
View Related
Dec 23, 2010
If I have a Query that returns columns where some of the values are NULL, is it possible to tell SQL to return all the NULL values as empty strings instead of the NULL datatype?
If the solution is vendor specific I am (unfortunately) using a Microsoft 2008 SQL server.
Example:
Code:
Select * from imaginary_table where id = 1
Might return:
Code:
Id: 1
Name:xyz
[Code].....
View 3 Replies
View Related
Apr 13, 2011
I have 2 very long files which are quite similar:
file1.csv:
Code:
file2.csv:
Code:
I need this output:
Code:
explanation:
If the fields 3 (serie) and 4 (modello) are identical, the lines of the 2 files should be "added":
Code:
If the fields 3 (serie) and 4 (modello) are different, just print the line of both files:
Code:
Is there a way to do this without reading the (huge) files with a "while read line" loop?
View 14 Replies
View Related
Jun 24, 2011
I'm writing a script and I have doubts on how to assign values to an already established variable. The value for the vatriable would be coming from a file with three columns. I'm using the awk command for this. Am I doing it correctly? which of the following two ways is the better one or if both are wrong which one should I use?
#!/bin/nsh
inputfile=$1
rolename=$2
[code]....
View 2 Replies
View Related
Apr 20, 2010
The lines beginning with greater-than symbols are the sequence descriptors and the lines immediately after each descriptor with A-Z characters, dashes, and question marks are the aligned DNA sequences. The sequences are always the same length within a file and never span/wrap across more than one line.I am trying to write a script to remove positions in the sequences that are only represented by a -, X, ?, or N (these represent gaps or missing data). Also, if there is exactly one non-gap/missing character in a position it is also useless (there is nothing to compare it to) so I would like to remove those positions as well.
Position 5 (from the left) was removed because it was all gap/missing characters. Position 9 was removed because only one character was a non-gap/missing character. Position 10 was retained because there were 2 non-gap/missing characters.I'm really not sure where to start here. My first concern is I can't figure out how to tell awk to treat each character in lines not containing a greater-than symbol as a separate field. After that, I'm thinking I should use set up a counter to count the number of lines with gap/missing characters comparing that to the total number of lines not containing greater-than signs?
View 14 Replies
View Related
Mar 31, 2009
I have been running one computer with Fedora 9 and one with Windows XP Pro. to conserve space, I have moved the Fedora hard drive into the Windows computer (newer). Both are sata drives and each boots completely on its on. I would really like to make it a dual boot system so I can choose which drive to boot without having to open the case and swap the cables around. The Linux disk is readable by the windows OS but not the other way.
Everything I have read about dual-boot is how to do it at installation time. Is there anyway to do it, without reinstalling one of the OS.
View 5 Replies
View Related
Jan 26, 2010
Is there any software that will allow me to combine like DVD Shrink does? Without having to convert them muliple times. In DVD Shrink, you open one titleset and choose the movie you want and then open the second titleset and then do the same, and have DVD Shrink combine and shrink them into another titleset, then I make a menu with Titlewriter.
View 4 Replies
View Related
Apr 4, 2010
Input file (already sorted on the key field):
204 Daniel Barton
206 Mary Halston
206 David Coleman
206 Carson Coleman
207 Roland Highsmith
207 Annie Highsmith
209 Connie Ward
Desired output file:
204 Daniel Barton
206 Mary Halston David Coleman Carson Coleman
207 Roland Highsmith Annie Highsmith
209 Connie Ward
View 1 Replies
View Related
Jun 24, 2009
I've browsed the internet, but haven't found a satisfying answer. Hope this forum can help me in the right direction.I'm exploring the possibilities of creating a program that manipulates (writes to) two different video cards.I program in C and hope to avoid high level libraries like Xlib to keep the mem footprint as low and the performance as high as possible. Windowing is not important no desktop environment will be used. Be able to write two pixels (one to each screen) to would do.
How would I go about?To give an idea of the solution I'm looking for:One screen will show thumbnails of images on the hard drive. Clicking an thumbnail on display A will show the selection full screen on display B. On a laptop; preferably thumbnails on the laptop screen and the selected image on the external monitor.Is this possible with just low-level libs or do I need X or some third-party software to make life easy?
View 14 Replies
View Related