General :: Messy Filesystem - Duplicate File Removal From Command Line
Jan 17, 2011
In debian/ubuntu I want to:
a) Create a list of all the files in one directory tree
b) Do the same for a second directory tree
c) Compare the two lists such that, only the file NAMES are compared (i.e. just comparing the "file.txt" part so that "/home/folder/file.txt" == "/home/secondfolder/folder/file.txt)
d) Output a list of all the duplicates
How to do this using scripting languages or regex or something?
I want to take a graphics file and make 10 copies of it to the same directory, each with 001, 002, or some such designation at the end of each file name so they have discrete files names. Is this possible using cp?
Was wondering if any perl guru's could help me with a quick log file adjustment. I have a text file that looks like so (tabs and newlines are revealed so you can see what separates the data):
There are maybe 100 lines of text in this file at any given time. I need to delete all duplicate lines only looking at the first bit of text prior to the first tab. It doesn't matter which one gets deleted as long as there are no two lines that begin with that same text at the beginning before the first tab. So in this example, either the fist line "1234" or the last line "1234" would need to be deleted. I already have code in my script that opens the files - I just need the code to read the text into an array and the part that would find matches based on the above criteria, and make the deletions.
If it would be easier, I can even do a system call and use SED (v4.1.5) and/or AWK (3.1.5) instead.
Is there a way to remove duplicate files from a specific folder through SSH? I've uploaded a lot of flash games on my server and I can see in the Webmin's file manager that I have many duplicates. Their names are different, of course.
I'm relatively experienced with UNIX and Linux, but this has me thrown for quite a loop, and it seemed like such a simple question. How would I go about finding the newest file in a file system? I thought something like:
Code:
ls -ltr `find /usr -type f`
would work, but I seem to be exceeding the argument maximum for ls:
ksh: 0403-029 There is not enough memory available now
I thought something involving xargs might work, but I really suck with that command.
I need to be able to convert HTML email messages saved as text files (.eml or .msg) to PDF documents, one PDF per email, retaining formatting and images.
Are there any Linux tools that will allow me to do this from the command line (so it can be scripted)?
Let's say i have a link to a file http://www.domain.com/dir/myfile.ext
Is there a command line tool that will allow me to download this file. I'm looking for something like: download <http address> ... is there anything that simple?
I want to download a file from the Linux command line. Basically I'm using ssh and I'm trying to download a file to my file system on my laptop. How can I do that from the command line?
I used awk "'!x[$0]++' test.txt > file.new" ,but it deleted #1 also.I tried using uniq command but i didn't work. Can anyone Please let me know is there any way to do this using shell script.
How can I create multipart rar file in Linux using the official console rar client?RAR 3.90 Copyright (c) 1993-2009 Alexander Roshal 16 Aug 2009Shareware version Type RAR -? for helpI want a multipart rar with each part size being 150 MB.
I have a jar, and I need to replace a class in it, at this moment, I can only open it with "archive manager" and then drag and drop the new compiled class into the jar, but I think this is really boring, if I can do with with just a command ?
I want to list all the files that don't have a copy with the same filename with -1 somewhere in it. So, in the example above, the results would be 3.png.
NB: the file and its copy with "-1" in it will be the same filesize, if that helps.
I've got a Debian Squeeze computer on which the graphics have packed up, but the terminal in single user mode work perfectly fine.
There are a few files on this Debian computer that I want to transfer off, to a networked computer, but I have no idea how to do this.
The destination computer is a freshly re-setup Mandriva install, without (as yet) samba. I don't think it's necessary though. The Mandriva install works fine, has graphics, etc, but can't see the Debian Squeeze computer on the network, possibly because it's in single user mode, thus prompting the problem of how to transfer the files, using only a command line.
In Linux, I'd like to know how to find the file(s) if any which as using a particular sector on the hard drive (ext2/3). There is a similar question here regarding Windows, however I need a Linux command line solution (this is a headless system).
I need to download a file from a website which has a URL formatted like:
[URL]
This redirects to a .zip file which has to be saved. There is also a need to authenticate based on username and password.
I tried to use wget, curl and lynx with no luck.
UPDATE:
wget doesnt work with redirection. It simply downloads the webpage instead of the zip file. curl gives the error "Maximum redirection exceeded > 50 " lynx also gives the same error.
I've spent hours trying to scan + shrink a multipage PDF documentlosing readability. This is the first time I've ever needed to do this! (I had to scan each page as ".jpg" in order to email and open on another computer, so I could not scan to PDF directly, which I think is why each page was so large; lower DPIs made the text too blurry.)I found this great tip on UbuntuGeek...but anyone can do this if GhostScript is installed:
I can do:mkdir messages and then: touch messages/hello.txt Is there a command that will do both - create the directory if it doesn't exist, and then the empty file? Something like: touch -p messages/hello.txt
