Ubuntu :: Script For Removing Duplicates?
Oct 24, 2010
Although having used Ubuntu for a good couple of years, I'm still a total beginner when it comes to scripting. However, what I need to do should be fairly straightforward:
Importing images from my digital camera, both RAW "originals" and JPG "copies" end up in the same folder. I typically flip through the JPG:s in Image Viewer and remove those that I'm not interested in. Now, this leaves me with the tedious job of going though all the RAW files in the folder manually to get rid of those too! It sure would be wonderful to get Ubuntu to do the work for me...
The script would simply need to go though all the RAW files in a folder one by one, check for a corresponding JPG file - and if there isn't one, remove the RAW file. How could I accomplish that?
View 3 Replies
ADVERTISEMENT
Oct 20, 2010
I have a column of datetime entries which I sorted to removed duplicate entries.
I have still lots of entries which are adjacent to each other by 1 second.
How would I go about removing any entries which have an offset of the previous or after entry by 1 second?
Code:
View 5 Replies
View Related
Nov 20, 2010
I'm writing a bash script to auto run on boot in Tinycore.
This is a watered down version.
Code:
I need it to either not add the time stamp if the awk finds a duplicate or write over the time with the new time if awk finds a duplicate.
BTW this is all pretty much cut-and-paste scripting so please feel free to comment if you know a more elegant way.
View 3 Replies
View Related
Feb 27, 2011
what i did was, remove evolution mail from synaptic, what i wanted to do was just remove the indicator applet from the task bar. i read a bunch of bad stuff about removing evolution from synaptic vs just removing the applet.
im worried. did i break anything or put my security at risk. after, i used a command (older) (sudo apt-get install ubuntu-desktop)to install ubuntu desktop. because i thought that it would fix evolution. then i went to synaptic and installed a package called evolution. i rechecked evolution in applications menu. however, i notice that i have both a checkable evolution and two evolution icons. nothing 'seems' broken. im not sure if it ever was. and evolution calender pops up as normal, as does the the installed plain evolution. they both seems to be an exact copy of the other.
all i really wanted to do was remove the indicator applet. did i make a serious mistake. since ive had ubuntu, ive reformatted a lot because i was worried i made a mistake of some kind. however now im into the more "make a mistake and fix it stage' as im pretty happy with my current desktop and have worked hard to customize it. the command, sudo apt-get remove indicator-messages removed the mail icon. i still am worried that i broke something, or put my security at risk. also, now i have two mail icons. evolution mail and calendar, and another just called evolution.
View 1 Replies
View Related
May 31, 2011
I know a bunch of commands and I am comfortable using the terminal, I even set a powerpc server but I can't figure out how to remove epiphany on this new computer I'm setting up. I didn't install anything with tasksel. I installed gnome and xorg afterwards... I load it up and 'startx' just fine. then I check around for the programs that were installed. I lik'em gimp, lot's of utilities. gedit. anyway I find epiphany, which I have already established that I dislike, I immediately go to the root terminal (another nice program that comes with gnome) and type apt-get remove epiphany-browser-data the output says it will be deleting gnome... however I have researched and found these are simpy meta packages that don't really matter.... however under the section that states all the packages that will be removed by autoremove there is a huge list... I doubt these packages are safe to remove. how to remove epiphany without removing a huge amount of probably needed software
View 14 Replies
View Related
Aug 25, 2010
These should be my last pleas for help with regard to Fedora 13. I've been unable to turn off the notifications that appear in the top right corner, despite a decent amount of searching on google. I can't remove any notifications package without removing a bunch of important software along with it. Also, F13 refuses to "Safely Remove" either of my external disks. I have to yank out the usb cord, touching wood each time.
View 5 Replies
View Related
Feb 8, 2010
I am hoping someone already has a script or knows of an app that will let me do this fairly easily - I have a fairly large folder structure that goes several levels deep, etc. In many cases there are duplicate file names that are not really different, e.g.,
/home/chris/folder/folder1/doc1.doc
/home/chris/folder/folder2/folder3/doc1.doc
I want to recursively go through /home/chris/folder and move everything to /home/chris/another_location/ without subfolders and renaming duplicates as appropriate, e.g.,
/home/chris/another_location/doc1.doc
/home/chris/another_location/doc1_1.doc
View 1 Replies
View Related
Oct 19, 2010
I have tried a combination of the following lines in .bashrc to try and control duplicates in the bash history file:
export HISTCONTROL=ignoreboth
export HISTCONTROL=erasedups
export HISTCONTROL=ignoredups
[code]....
View 5 Replies
View Related
Nov 17, 2010
I've tried using a script to run incremental backups. The idea was to use the update switch to just update the files in the .tar instead of creating new ones. However it seems to have created duplicate files in the tar instead of just updating them (refer to screenshot). Is this normal?
Here is the command in the script;
Code:
tar -uvpf /home/jonny/.BackUps/Updating/Documents.tar /home/jonny/Documents
Is there a way stop this but still have them update to the latest version?
View 1 Replies
View Related
Jan 17, 2011
Open office Calc all the googles i can read all have removing the duplicates from the spreadsheet. I want to do the exact opposite. I want to keep the dupes and remove the others.
View 1 Replies
View Related
Jan 29, 2010
Recently, I'm getting an annoying problem: When I scroll quickly the text smears or duplicates itself -- doesn't happen with all applications. I haven't had this problem until I started to use OpenGL compositing with my NVidia card.
View 3 Replies
View Related
Apr 16, 2010
I am running KDE 4.4.2 release '241'. I have no clue if that's the latest dev version, but I do know that each day I have to install a 150MB KDE update, so it probably is. Either way it's pretty annoying. The way opensuse handles repositories is different than Ubuntu's, so somehow I ended up creating duplicates too. So basically, I want to remove any duplicates and also stop receiving the daily KDE updates.You can ignore the google repositories. My repositories
[code]...
View 8 Replies
View Related
Jul 1, 2011
I have a text file which is a list of all my contacts. So far i have only found software and commands which remove duplicates but i would like to remove all duplicates AND their original entries too so only contacts which have no duplicates are left.
View 10 Replies
View Related
Mar 11, 2011
I have a large file and want to keep lines which are duplicates, but the test for duplicates is performed only on the first blank-delimited word.
View 6 Replies
View Related
Sep 24, 2009
i am getting duplicate entries in dag rpm repository with different names in different cases !
# yum search fileinfo
php-pecl-Fileinfo.x86_64 : Fileinfo is a PHP extension that wraps the libmagic library
php-pecl-fileinfo.x86_64 : PECL package to get file information through libmagic[code]....
which is the correct rpm to install ?
View 8 Replies
View Related
Sep 28, 2010
When I ping our Microsoft Windows terminal server "cluster" farm, I get ICMP warnings that there are duplicate packets. I am able to rdesktop to the cluster with no problems. We are trying to setup nagios to run on this Ubuntu configuration and nagios is reporting the following error:
"PING WARNING - DUPLICATES! Packet Loss=0%, RTA=.98ms.
FPing reports duplicates as well. Is there a setting in the Arp table that needs to be set differently because the "Cluster" MAC address isn't an actual hardware MAC but a virtual MAC address?
View 2 Replies
View Related
Jul 14, 2011
I found this email in my inbox. From time to time it's showing up here then Evolution starts downloading messages again and duplicates all my inbox. I think.
View 1 Replies
View Related
Jul 30, 2015
I'm having issues setting up rsyslog to receive syslog from another server and only log to one file. I'm receiving the syslog from the remote side, however its putting the entries into more than one log file.
I configured /etc/rsyslog.conf to enable udp, and I have implemented a filter to log only from that IP address, and then stop processing more rules, but it seems to continue on.
I have found that the remote syslog events are using local0 and local1. There are two custom rsyslog config files in /etc/rsyslog.d that handle those two facilities. If I use that same if statement at the beginning of those custom config files, I can get it to work. Seems like a hack though.
Not working:
I put my if statement before the include statement, thinking I could stop it from hitting the custom rules.
Code:
Select all# /etc/rsyslog.conf Configuration file for rsyslog v3.
#
# For more information see
# /usr/share/doc/rsyslog-doc/html/rsyslog_conf.html
#################
#### MODULES ####
#################
$ModLoad imuxsock # provides support for local system logging
$ModLoad imklog # provides kernel logging support (previously done by rklogd)
#$ModLoad immark # provides --MARK-- message capability
[Code] ....
This works:
A custom config file in /etc/rsyslog.d
Code: Select allif $fromhost-ip == '<my ip>' then /var/log/<my directory>/syslog.log
& ~
local0.* /var/log/<a log file for local0>.log
This is on a WD Mycloud device:
Code: Select allLinux WDMyCloud 3.2.26 #1 SMP Tue Jun 17 15:53:22 PDT 2014 wd-2.2-rel armv7l
The programs included with the Debian GNU/Linux system are free software; the exact distribution terms for each program are described in the individual files in /usr/share/doc/*/copyright.
View 1 Replies
View Related
Feb 7, 2011
I'm trying to use awk to remove rows that are duplicates based on 3 fields, and I want to keep the on that has the higher value in another field. I'm working in C-Shell. For example the below is greped out of a larger data set to use in here as example:
Input (Field separator is a comma:
Code:
4180,-6999,MA,BARNSTABLE,BOURNE,1,1.7,1700,PM,1/26
4180,-6999,MA,BARNSTABLE,BOURNE,1,3.5,2025,PM,1/26
4180,-6999,MA,BARNSTABLE,BOURNE,1,1.0,1511,PM,1/26
4180,-6999,MA,BARNSTABLE,BOURNE,1,5.7,0540,AM,1/27
[Cpde]....
View 1 Replies
View Related
Jun 18, 2010
when i ping a remote machine to 172.16.1.55 then i get this result...how can get normal packets......
64 bytes from 172.16.1.55: icmp_seq=1 ttl=128 time=0.468 ms (DUP!)
64 bytes from 172.16.1.55: icmp_seq=2 ttl=128 time=0.448 ms
64 bytes from 172.16.1.55: icmp_seq=2 ttl=128 time=0.469 ms (DUP!)
[code]....
View 1 Replies
View Related
Jul 18, 2011
I'm trying to trim down Linux so that it fits on an appliance, and noticed that some related files in /bin have the exact same size:
Code:
Are those file duplicates, or are they just hard-linked, ie. there's really only one file in the flash memory but it looks like there are more than one? How can make sure?
View 7 Replies
View Related
Aug 2, 2010
I have about 1TB of data to sort out and need a good GUI package to find and safely delete the duplicate files. I am running Ubuntu 10.04 amd_64. Would appriciate any thoughts on how to get it done.
View 8 Replies
View Related
Dec 26, 2010
I have a list of about 425 servers that are mostly redundant. I need to weed out the duplicate names so that I have a count of only the unique server hostnames. What is a good command to do this?
View 3 Replies
View Related
Oct 7, 2010
When I try to open something in gnome, say, a file format it does not recognise, I get the usual "open with..." dialog. There, I see many duplicates of wine programs, such as Windows Media Player, or Wine core exe, wine windows program loader, etc.For example, the "Windows Media Player" is listed at least 10 times in that dialog. How can I remove these duplicates?
View 2 Replies
View Related
Aug 13, 2011
I have just got rid of Ubuntu in favor of Fedora, after more than ten years using Debian and Ubuntu. Most differences are pretty minimal but one big difference I haven't been able to master yet is the realm of rpm and yum. My kernel is 2.6.40-4.fc15.x86_64. Today, when running yum update I was greeted with a number of errors.
Output of yum check:
Code:
Loaded plugins: fastestmirror, langpacks, presto, refresh-packagekit
glibc-2.14-5.x86_64 is a duplicate with glibc-2.14-4.x86_64
glibc-common-2.14-5.x86_64 is a duplicate with glibc-common-2.14-4.x86_64
1:perl-Module-Pluggable-3.90-160.fc15.noarch is a duplicate with 1:perl-Module-Pluggable-3.90-159.fc15.noarch
1:perl-Module-Pluggable-3.90-160.fc15.noarch has missing requires of perl = ('4', '5.12.4', '160.fc15')
1:perl-Pod-Escapes-1.04-160.fc15.noarch is a duplicate with 1:perl-Pod-Escapes-1.04-159.fc15.noarch
1:perl-Pod-Escapes-1.04-160.fc15.noarch has missing requires of perl = ('4', '5.12.4', '160.fc15')
1:perl-Pod-Simple-3.13-160.fc15.noarch is a duplicate with 1:perl-Pod-Simple-3.13-159.fc15.noarch
1:perl-Pod-Simple-3.13-160.fc15.noarch has missing requires of perl = ('4', '5.12.4', '160.fc15')
I have also attached the output of yum update, which I believe is more verbose on the sources, etc.
I'm afraid that if I just start forcing removal of packages I'll render my system useless.
View 6 Replies
View Related
Jul 2, 2010
How can I find a list of files that are named duplicates i.e. have same name but in different case that exist in the same directory?
View 7 Replies
View Related
Oct 15, 2010
I am using 10.10 but am having a problem with duplicates being automatically downloaded into my inbox in evolution. This has not happened under 10.04 with the same settings. How torid my inbox of the dupes? It seems to have faded away after the "official release" but I still have a messed up inbox. I use gmail recent mode with pop settings.
View 14 Replies
View Related
May 23, 2011
I would like to combine two arrays and delete duplicates that might occur. I tried to solve it like this:
Code:
combined=( "${results1[@]}" "${results2[@]}" )
printf "${combined[@]}" | sort -n | uniq
Example:
result1 (4 elements):
[Code]....
In my code printf seems to have a problem with elements that have the same letters but a space inbetween. For instance "new foo", "newfoo" are the same for printf
View 5 Replies
View Related
Apr 16, 2011
i've been trying to get rid of unnecessary files on my kids' computer and just noticed that /home/.Trash-0 contains a duplicate of the file tree of the /home directory, including all of the current users' files and subdirectories. is this something i should leave alone? the machine is running slack 13.0 with KDE, and this is something i never noticed before.
View 4 Replies
View Related
Jun 14, 2011
I am looking for an application to detect mp3 duplicates using the mp3 spectrograph (sound wave matching) or something similar because simply I have many mp3 files that are the same but with different bitrate,ID3 tags,filesize
View 1 Replies
View Related
