Anyone know of a good Linux application that will remove duplicate files interactively? I've recently spent a lot of time (read weeks) pruning my music collection, basically by hand. and now I'm moving on to my family photos. Most of the work with the music was done under Windows XP. As for the photos, I have a fantastic Windows application, D'Peg, that I had actually purchased some years ago. This app rocks for Windows. In my opinion it's so good that I would happily pay double the asking price. However, I'd prefer to use Linux if possible, so, what's out there, anything that is worth it's salt? Currently playing around with Picasa.
I recently upgraded my x86_64 system from FC8 to FC10 using Pre-Upgrade. (related blog link) It appears that the upgrade process installed a steaming pile of i386 packages that are duplicates of existing x86_64 packages. I now get update errors because of this package clash. I have searched the fora and the most progress I've been able to make so far is: I apparently had 8 unfinished yum transactions so I did yum-complete-transaction 8 times and have no more incomplete transactions.
The output of package-cleanup --dupes is not very helpful:
Yet I still get transaction errors when I run updates via synaptic. It checks dependencies and downloads everything and errs when testing changes. This is the error it gives at the moment:
Code:
Test Transaction Errors: file /etc/gconf/schemas/gweather.schemas from install of libgweather-2.24.2-1.fc10.x86_64 conflicts with file from package gnome-applets-1:2.20.1-1.fc8.i386
I have made a custom grub2 menu however, both the default and the custom show together. So my grub looks like the list below, the bolded entries are my custom ones. How do I get rid of the duplicates? I have tried apt-get remove and deleting old kernels.
ubuntu,linux ... ubuntu,linux recovery memtest memtest windows7 windows7 ubuntu linux ubuntu linux recover
I have a couple of MP3s that have duplicate fields in their ID3 tags. Let me show you what I mean: This is causing problems with some media players. Is there a tool that can automatically fix these MP3s in batch? I'd prefer a free Windows or Linux program. I'm not afraid to work on the command line if necessary.
I want to find and remove duplicate consecutive words from a text file. I've tried working with array but is very difficult..then i've tried using sed...somebody hint me with this sed : sed ':f;N;$!bf; s/(.*) 1/1 /g; s/(.*)1/1/g'. It works fine but if i have 3 consecutive identical words it only remove first one and the last two remain intact.
I used awk "'!x[$0]++' test.txt > file.new" ,but it deleted #1 also.I tried using uniq command but i didn't work. Can anyone Please let me know is there any way to do this using shell script.
I have a Nook. When I plug it into the USB port on my system, a window pops up asking what I want to do with the new device. I can open it and access media/nook and move files into and out of the directory.
There's a button to "Safely dismount Nook" before I unplug it. I use that. Apparently, however, sometimes it doesn't respond. Now I have .hal-mtab-lock in my /media folder, along with Nook Main Memory and Nook Main Memory (1) folders. I can't delete any of them.
How do I a) delete these folders, and b) make sure it actually unmounts the device in the future?
I recently upgraded to Ubuntu 11 and a few days later my ecryptfs filesystem began misbehaving in a weird way. In my home directory, many subdirectory names are duplicated verbatim. Here's an ls -F excerpt:
I can no longer access files in those directories (if I ls the directory, it appears empty; I can cd to it, but there's nothing inside). Not all of the directories are duplicated/damaged like this, but most are. A few non-directory files are also duplicated in this fashion, so for example:
These should be my last pleas for help with regard to Fedora 13. I've been unable to turn off the notifications that appear in the top right corner, despite a decent amount of searching on google. I can't remove any notifications package without removing a bunch of important software along with it. Also, F13 refuses to "Safely Remove" either of my external disks. I have to yank out the usb cord, touching wood each time.
Is there a program for linux which can show me a list of all duplicate music files in a directory? This will allow me to delete all duplicate files without searching for them manually.
i waas wondering if anyone knew of a script or program that removes duplicate words in a txt file. im making an install script and the install list has gotten a bit long so i want to ensure there are no duplicates in the file
I have a text file that is filled with references to duplicate files. I'm trying to create a text file for each duplicate file found that contains the paths to the duplicates. I would also like the text file names to be based on the size and file name.
Contained within each of these 67 text files is about 1 million urls. Yes. I have 67 text files that contain 1 million lines of urls each. I am sure I am swimming in duplicates. I tried opening one text file and clicking sort ----->remove duplicates. Now Gedit is not responding my processor is maxed out to 100% and I think I am finally ready to delve into some command line code. Can anyone give me idiot proof instructions on how to sort the duplicates out of each one of these 67 text files? How about no duplicates across all 67?
I am basically trying to remove duplicate words in my <title></title> tag after I got hit by Google Panda. I have around 750 .html files and it will be difficult for to me remove one by one. I am looking for a way to remove only from within <title> </title>
Example of a duplicate title I have:
Code:
<title>Pasta, Pasta Recipe and Pasta Guide</title>
I dont want to replace those words anywhere else in the file except for within the <title>
i have a big file of random numbers i generated at some point in time, after working with it with different things(how fun that was)... i want to remove duplicate lines and i'm not sure i'm doing this right
In debian/ubuntu I want to: a) Create a list of all the files in one directory tree b) Do the same for a second directory tree c) Compare the two lists such that, only the file NAMES are compared (i.e. just comparing the "file.txt" part so that "/home/folder/file.txt" == "/home/secondfolder/folder/file.txt) d) Output a list of all the duplicates How to do this using scripting languages or regex or something?
Trying to remove lines from a syslog text file that have duplicate strings
Mar 10 06:51:11[http-8080-1] INFO com.MYCOMPANY.webservices.userservice.web.UserServiceController [u:2533274802474744|360] Authorize [platformI$tformIdAndOs=2533274802474744|360, userRegion=America|360]
then a few lines down
Mar 10 06:52:03 [http-8080-1] INFO com.MYCOMPANY.webservices.userservice.web.UserServiceController [u:2533274802474744|360] Authorize [platformI$tformIdAndOs=2533274802474744|360, userRegion=America|360
got the same thing in terms of a u: number but the issue is I need to remove duplicates and just leave one and the file has multiple duplicates of different u: numbers and it's 14,000 lines long. can anyone tell me if I can use awk? sed? or sort for something like this to? removing lines that have a certain string in there that's a duplicate.
I want to take a graphics file and make 10 copies of it to the same directory, each with 001, 002, or some such designation at the end of each file name so they have discrete files names. Is this possible using cp?
Mandriva 2010 kde4.3.5 Dolphin 1.3I like to use split screen with file manager. Would like duplicate up and back buttons that always corraspond to each pane of the window so I need not worry about which pane has the focus when I go to use the bauttons.I am hoping to find the functionality anywhere anyhow. I have been using the default Daulphin file manager but don't care if I need to change to a different one.
I have a large text file containing over 180k lines and another text file containing about 1k. I would like to remove lines in the 180k-line file that exist in the 1k-line file.
I had a duplicate IP from another device on the network that would jut not let it go and was the same IP as my Fedora box that was working fine. After screwing around with the other device I finally just gave up and changed the IP on the Fedora box. Now I can't access the internet at all from the Fedora box. I looked in my routing tables in my router and the mac was showing as the mac of the other device. After some reboots here and there that is fix and the routers routing tables are now showing correctly. The Fedora box still can not resolve any domains or get online. Is there something somewhere inside the Fedora box that is still jacked up from the duplicate IP?
I have Fedora 11 installed-32bit-with xfce installed as the desktop. When I click on the fedora icon for the menu and select Preferences, there are 2 input methods listed even though I did not have any installed.Since there is no menu editor any more, does anybody know how to edit the menu so that I can get rid of these entries?
My program for check mail is kMail (version is 1.13.1), sometimes get duplicates mail from all my accounts, which on the gmail, rambler and other mail servers. How I can configure my kMail for remove this duplicates.
I couldn't find my previous posting about udev at booting, so I created a new posting with the same material. Subsequently my earlier post showed up again----or I managed to find it. I can't seem to find any information about how I go about deleting the earlier post. Supposedly we are supposed to be able to notify the managers of abusive posts. I thought I might use that to ask them to delete the earlier post, so I will try that..
I do realize, that there are a lot of other threads regarding this problem. I've spent almost the whole day today reading and trying them out, and at the end, I figured out, that none of them worked for me. So, I'll provide as much info as possible... code...
Unfortunately, none of this worked, and even after the first 2 methods I didn't have ANY sound at all! Please help me, I need this in order to have my speakers plugged into the monitor, so I can control the volume via the remote control. In order to dissolve any suspections to a defect hardware, I'd like to add, that this is perfectly working under Windows 7, 64-bit edition.
Was struggling with this for a little, no really clear answers around.Anyways, edit /etc/mstab, kill the entire line with the faulty drive.Unplug the drive/usb, restart, then do a
Code: sudo fsck -r /dev/whatever umount it, then remount and it should be fixed
I have a text file with many pairs of number, one pair in each line. Each 25 of these pairs are a solution to a math problem I've been working on, and each solution is separated from another by a line with "**********".The problem is that there are duplicate solutions. In order to know exactly how many solutions I found, I have to delete the duplicate ones. How can I do that?Just to make things clear, here are the first three solutions:
I tried upgrading from Fedora 8 to Fedora 10 using the DVD, but I ran into a few problems.After verifying the install media, it was no longer able to detect the DVD. Annoying, but I got around this by simply rebooting and then skipping the media check.The second problem happened after I tried to "upgrade an existing installation". It gives me the error "Multiple devices with label /boot detected. Devices must have unique labels in order for you system to work. Please correct this problem before upgrading".My system's working just fine, so I'm not sure why I'm getting this error. I have a RAID-1 setup, so would that be causing the error? How do I correct this problem without wiping out my system?
