General :: AWK: Rm Duplicates Based On Multi Fields?
Feb 7, 2011
I'm trying to use awk to remove rows that are duplicates based on 3 fields, and I want to keep the on that has the higher value in another field. I'm working in C-Shell. For example the below is greped out of a larger data set to use in here as example:
Input (Field separator is a comma:
Code:
4180,-6999,MA,BARNSTABLE,BOURNE,1,1.7,1700,PM,1/26
4180,-6999,MA,BARNSTABLE,BOURNE,1,3.5,2025,PM,1/26
4180,-6999,MA,BARNSTABLE,BOURNE,1,1.0,1511,PM,1/26
4180,-6999,MA,BARNSTABLE,BOURNE,1,5.7,0540,AM,1/27
[Cpde]....
View 1 Replies
ADVERTISEMENT
Mar 11, 2011
I have a large file and want to keep lines which are duplicates, but the test for duplicates is performed only on the first blank-delimited word.
View 6 Replies
View Related
May 12, 2010
I've got a set of objects (all of the same type). I'm trying to think of a good way to divide it into equivalence classes, with equivalence of two objects defined as meaning a specified set of attributes are equal for both objects. More concretely, I've got:
- a Java class with around 50 fields
- a bunch of instances of the class
I want:
- to divide the instances into a few sets
- in each set, each instance has field 1 - field 5 equal to fields 1-5 of the other instances in the set.
The method I've come up with is to generate a hashcode for each instance based on the hashcodes of fields 1-5*, and map the hashcode to one of my sets. Ignoring problems with potential hashcode collisions (which I'm expecting to be too rare to worry about for now), does that sound reasonable? It seems simple enough, but I'm wondering if there's a simpler method I haven't thought of.
* I'll generate the hashcode using a method based on Eclipse's generic hashcode method, which looks like this:
Code:
public int hashCode() {
final int prime = 31;
int result = 1;
[code]....
View 5 Replies
View Related
May 15, 2010
Does anyone know of an alsa based software multi-band (10 or more) graphic audio frequency equalizer that works with Suse 11.x? To be clear, I don't mean an equalizer within an audio or video player. One that can be used with any sound application, that works between the output of the player and the output plugs on a motherboard or sound card.
View 9 Replies
View Related
Jul 1, 2011
I have a text file which is a list of all my contacts. So far i have only found software and commands which remove duplicates but i would like to remove all duplicates AND their original entries too so only contacts which have no duplicates are left.
View 10 Replies
View Related
Jul 12, 2011
I have a file1:
Code:
$ cat PF(1).out
Tmp39 PF10271.3 423 ENSP00000326063 488 1.2e-201 41-478
Tmp39 PF10271.3 423 ENSP00000338165 492 1.9e-200 46-479
[code].....
View 14 Replies
View Related
Jun 7, 2010
I notice that on DistroWatch, the descriptions include "debian based", "arch based", etc. For the newbie, is there any practical difference between the different bases, or is it a Coke vs Pepsi type question?
Is a distro based on Debian say, easier for a newbie to learn or work with than say one that is FreeBSD based?
View 9 Replies
View Related
Mar 28, 2010
I'm writing a script that picks out the time offset in a string, but am trying to figure out a way to print a field based on a match (likely needs to be regex).
Here are some sample fields:
09.900. Either add -12:-1: 0 to existing offset, or restart the slicer with no offset or with offset=00
09.900. Either add -12:-1:30 to existing offset, or restart the slicer with no offset or with offset=00
09.900. Either add -10: 0:-47 to existing offset, or restart the slicer with no offset or with offset=00
What I need would be "-12:-1: 0" from the first line, for example. I was just using "awk '{print $4}'" originally but because of the damn space in the 1st and 3rd lines by the 0, that doesn't work for me. How to grab those fields correctly for all cases?
View 3 Replies
View Related
Feb 22, 2010
I'm looking for a way to dynamically modify a fied in the SQL file.
Example,
INSERT INTO `TABLE` VALUES(1AADFDF,DFF33D,10023,SOMEDATA, SOMEDATA);
INSERT INTO `TABLE` VALUES(1AADFDF,DFF33D,10098,SOMEDATA, SOMEDATA);
[code]....
The third field in this SQL file is what needs to change to n+1
"10023" needs to change to "1"
"10098" needs to change to "2"
"10123" needs to change to "3"
"10983" needs to change to "4"
etc...
I tried with awk and sed, but since both require the usage of ' (single quote), my variable fails to get updated. I know i can use cut and recreate the sql statememnt, but i believe there should be easier way using sed or awk. Would anyone know of any creative way to get this to work in one-liner? This needs to be a part of the script preferable without making call to external file.
View 1 Replies
View Related
Jul 2, 2010
How can I find a list of files that are named duplicates i.e. have same name but in different case that exist in the same directory?
View 7 Replies
View Related
Oct 15, 2010
I am using 10.10 but am having a problem with duplicates being automatically downloaded into my inbox in evolution. This has not happened under 10.04 with the same settings. How torid my inbox of the dupes? It seems to have faded away after the "official release" but I still have a messed up inbox. I use gmail recent mode with pop settings.
View 14 Replies
View Related
Dec 13, 2010
For example: file a:Tom:blackLily:pinkfile b:Tom:bigKate:smallAnd, the result:join -t: a1 a ot:Tom:black:bigLily:pinkBut what I want is:Tom:black:bigLily::pink
View 2 Replies
View Related
Jan 10, 2011
I have a couple of MP3s that have duplicate fields in their ID3 tags. Let me show you what I mean: This is causing problems with some media players. Is there a tool that can automatically fix these MP3s in batch? I'd prefer a free Windows or Linux program. I'm not afraid to work on the command line if necessary.
View 2 Replies
View Related
Apr 5, 2010
When you use an IDE and it asks you if you want frame based or dialog based, what is the difference?
View 1 Replies
View Related
Feb 2, 2011
what distro should i use, rpm based or debian based?
View 2 Replies
View Related
Dec 2, 2010
I am trying to remove the space between the 2nd and 3rd fields but just cannot work out how to do it. I've considered sed, cut, awk...I know with sed you can replace all instances within a file but can you also specify to remove, say, the 2nd instance of a space in each line (this would be perfect for me)?
DAAE 60 402 Bejaia Algeria 1 36-43N 005-04E 36-43N 005-04E 2 2 P
DAAG 60 390 Dar-El-Beida Algeria 1 36-43N 003-15E 36-43N 003-15E 25 25 P
DAAJ 60 670 Djanet Algeria 1 24-33N 009-28E 24-33N 009-28E 1054 1054 P
[code]...
View 3 Replies
View Related
Feb 26, 2011
I have following in cpuinfo:
processor : 0
vendor_id : AuthenticAMD
cpu family : 15
[code]....
View 2 Replies
View Related
Jun 24, 2011
I have an ATI Radeon HD 3300 on-board video chipset, and an ATI Radeon HD 4350 PCI card. What I want is to have both displays available from one mouse/keyboard. I want to play media on one and have the other as my main desktop.The problem is that with Xinerama enabled, KDE desktop effects do not work (KDE says XComposite and XDamage are not available, even though I explicitly enabled them as extensions in the xorg.conf file), and performance is quite bad. Without Xinerama enabled, performance is great, desktop effects work great, but there's a lot of trouble with full-screen video, and the KWin window manager does not apply in the second display (although I can run a second instance of KWin on :0.1).
View 2 Replies
View Related
Mar 21, 2011
Anyone successfully using VNC client on a Mac to control a Debian server?I have the vncserver setup on the Debian machine properly. But I'm having problems connecting to it from both a PowerMac running Tiger and a MacBookPro running leopard.I can connect no problem from a machine running Slack12.2, have not setup port forwarding on my router to connect remotely yet.My Debian machine is running the latest stable release of squeeze with KDE4.I originally tried this with RealVNC Enterprise for OSX but I'm not gonna buy it so I need another alternative after the 30 day trial ends as they have no free version for OSX.
The situation is that I do freelance graphic design on the PowerMac with Cinema4D and Photoshop so I spend most of my time on that machine which is located in my home studio in my attic. Aside from the MacBook and a Dell desktop(family machine)all my other machines and network hardware are in the basement. So to go from the attic to the basement everytime I need to do something on another machine is not practical, and the only other machine I need to access on a regular basis is the Debian box in the basement, this makes the most sense.
I also have a 14 year old living in the house and he's fascinated by all this and will meddle in anything he gets the chance to so all the Linux machines and network hardware need to be behind lock and key.
View 6 Replies
View Related
Sep 17, 2010
Now that Ubuntu 10.04 has multi-touch capabilities built-in, if I do not have a multi-touch screen or surface device, can I get 2 USB mice and get 2 pointers on the screen? One for the right hand and one for the left hand as I am ambidextrous, and would find it very convenient to have 2 mice.
View 4 Replies
View Related
Sep 12, 2011
Im running 64bit centos 5.6 and using virt-manager.On one of my guest OS's, Windows 7, The max Physical CPUs is 2, you can have unlimited CPU Cores however. (like my machine i use for work has 1, 4 core processor).The issue im having is xen only allows you to set the vcpu arguemnet in your xen config file. How can i set it so that 1 CPU has several Cores just as windows would recognize this machine if i were installing directly to the hardware vs via a VM.Ive searched for 2 days staright trying to address this issue, very little progress, Does anyone know where a XEN support forum is? all i get is the citrix xen support forums.
here is the best info i have found on this, but i dont know how to change this for my CPU to work, when i enter this in my xen config it essentially ignores it and just takes the value of vcpu= so windows shows 2 CPUs each with ONLY one core. Id like 1 or 2 CPUS showing Several cores.The physical Hardware is 2x Xeon 5300 Quad Core CPUs.
> # Expose to the guest multi-core cpu instead of multiple processors
> # Example for intel, expose a 8-core processor :
> #cpuid=['1:edx=xxx1xxxxxxxxxxxxxxxxxxxxxxxxxxxx,[code]........
View 3 Replies
View Related
Mar 17, 2011
I have a tab delimited file with 2 columns.
2224 23943843
5494 39843843
8372 48938384
I have to loop through this file and copy the files with the first field to a new name that contains both the first and second field. To be more specific, file quote2224.pdf has to be copied to 23943843_quote2224.pdf file quote5494.pdf has to be copied to 39843843_quote5494.pdf I don't know how or if it's even possible to use a for loop and access both fields. I don't know how to copy files in awk.
View 1 Replies
View Related
Sep 1, 2010
I have a mytext file with month and year as two separate fields. likemytext fil
08 2010
09 2010
10 2010
........
........
........
I want to read the values of each field i.e., month and year into an awk script.
View 10 Replies
View Related
Aug 12, 2010
I've got some source code that loads and plays WAV files through the OSS drivers (and USB audio). The code works even under 2.6.34. Is there any benefit to rewriting the code using the ALSA system?
View 1 Replies
View Related
Jan 17, 2011
Open office Calc all the googles i can read all have removing the duplicates from the spreadsheet. I want to do the exact opposite. I want to keep the dupes and remove the others.
View 1 Replies
View Related
Apr 24, 2010
I am working on a set up, and I like to be able to toggle compiz on and off, or toggle dualscreen on and off. The thing is, I also have conky on my desktop as well as a terminal window embedded in my desktop (that requires compiz). So, when I turn comiz off, or resize my desktop, I want to be able to reposition conky/embedded-terminal and the terminals position is relative to my conky position and the size of my virtual desktop.
I can do this all fine, except that to reposition the terminal I need to kill it then reopen it. But if I kill gnome-terminal it kills ALL gnome-terminals instead of just my embedded one. How can I specifically close my embedded one and leave any others untouched? Lets say that the title of my embedded terminal is "trans777"Also, the trans777 titled gnome-terminal will be killed when compiz is not running.
View 1 Replies
View Related
Apr 16, 2010
I am running KDE 4.4.2 release '241'. I have no clue if that's the latest dev version, but I do know that each day I have to install a 150MB KDE update, so it probably is. Either way it's pretty annoying. The way opensuse handles repositories is different than Ubuntu's, so somehow I ended up creating duplicates too. So basically, I want to remove any duplicates and also stop receiving the daily KDE updates.You can ignore the google repositories. My repositories
[code]...
View 8 Replies
View Related
Oct 24, 2010
Although having used Ubuntu for a good couple of years, I'm still a total beginner when it comes to scripting. However, what I need to do should be fairly straightforward:
Importing images from my digital camera, both RAW "originals" and JPG "copies" end up in the same folder. I typically flip through the JPG:s in Image Viewer and remove those that I'm not interested in. Now, this leaves me with the tedious job of going though all the RAW files in the folder manually to get rid of those too! It sure would be wonderful to get Ubuntu to do the work for me...
The script would simply need to go though all the RAW files in a folder one by one, check for a corresponding JPG file - and if there isn't one, remove the RAW file. How could I accomplish that?
View 3 Replies
View Related
Sep 24, 2009
i am getting duplicate entries in dag rpm repository with different names in different cases !
# yum search fileinfo
php-pecl-Fileinfo.x86_64 : Fileinfo is a PHP extension that wraps the libmagic library
php-pecl-fileinfo.x86_64 : PECL package to get file information through libmagic[code]....
which is the correct rpm to install ?
View 8 Replies
View Related
Apr 27, 2010
I have a filename file-name-here-1.2.3-i486-2.tar I would like to know how to remove the last two fields leaving only: file-name-here-1.2.3
I can use:
x=file-name-here-1.2.3-i486-2.tar
echo ${a%-*}
to remove the last field (after -) but how do i remove the last two in one fell swoop?
View 2 Replies
View Related
