Programming :: Bash Script For Moving X Number Files From / Direct1 To / Direct2?
Jan 29, 2010
I have no Bash skills but Im badly in need of a script to move the first 20,000 or whatever number of files from a directory containing over 200,000 files to a new directory. The problem is that I cant access the directory because its so large so I want to break it into chunks, but keep the files in order if possible. If you know of a script to meet my needs, please post it. Otherwise Ill post my fumblings with Bash until I find the right way to fix my problem.
View 14 Replies
ADVERTISEMENT
Sep 18, 2010
I want to move files from a $SOURCEDIR to a $DESTBASE/$DESTDIR. Under $DESTBASE there are many directories, and I need to test beforehand if a file from $SOURCEDIR already exists in any of them.
This is obviously extremely slow, and the real use case involves dozens of dirs and thousands of files. Creating a temporary "index" file for the find command (instead of running it every iteration) speeds it up a little, but it's still very clumsy.
View 14 Replies
View Related
Feb 4, 2011
I would like to know how do I print the line # in a script. My requirement is, I have a script which is about ~5000 lines long. If there are any errors happen I just exit. And I would like to add the line # of the script where the error happened.
View 3 Replies
View Related
Jun 1, 2010
I have a bash script that is tracking ERROR connections (running in a cron every 15 minutes), sometimes the errors are true and are not connecting. But most of the time, the disconnects/reconnects are between 30 seconds are sending out false positives. I am looking for a time comparison code to pick out the 30 second disconnects and know that this is and "ok" error. Here is a section of the log with timestamps, errors, etc.
Tue 2010-06-01 22:01:30 ERROR [DataBufferSendService] 172.31.0.1:8016 caught exception: [NetworkAPI] Exception: [SocketStream] socket error
Tue 2010-06-01 22:01:32 ERROR [DataBufferSendService] 172.31.0.2:8016 caught exception: [NetworkAPI] Exception: [SocketStream] socket error
Tue 2010-06-01 22:02:00 INFO [Client] 172.31.0.1:8016 connecting to 172.31.0.1 on port 8016
Tue 2010-06-01 22:02:02 INFO [Client] 172.31.0.2:8016 connecting to 172.31.0.2 on port 8016
View 8 Replies
View Related
May 15, 2011
How can i get bc to subtract negative numbers?
View 10 Replies
View Related
Feb 4, 2011
I would like to know how do I print the line # in a script. My requirement is, I have a script which is about ~5000 lines long. If there are any errors happen I just exit. And I would like to add the line # of the script where the error happened.
View 2 Replies
View Related
Jul 1, 2010
Is there a way to count the number of errors- an exit variable $? from one function?
The output from the exit variable (either erroneous or correct (1) or (0)) is it possible to add the erroneous one ups?
View 3 Replies
View Related
Dec 5, 2010
I'm starting bash shell script and I'm looping without any solution.
I'm trying to find some files under a folder hierarchy and in case of errors moving these files to a destination folder under the same hierarchy recreating this hierarchy if not exists.
Finding all ._* files under /src and moving them to /dest recreating folder1 or the others which contains ._* files but without moving files which does not correspond to the pattern.
Code:
I tried find command and I'am getting all needed files
Code:
But I don't know how to use the output to get the parent folder of files which are found to
1- create folder with mkdir -p /dest/folder1 or /dest/folder1/folder4
2- move found files from /src/... to /dest/... with rm command
I'm working on a find command as this trying to do all in the same line but ... little lost
Code:
View 8 Replies
View Related
Aug 9, 2010
I have the following code in bash script:
Code:
#!/bin/bash
COUNT=1
# bash until loop
until [$COUNT -gt 2]; do
pq A$COUNT [Pemptus].pq &
let COUNT=COUNT+1
done
I did this because I'm that much of a Progress Quest geek that I wanted to have a huge group on the online server, so I decided to make a script that would open all the files for me rather than having me do it manually. I created some characters with the boring name of A1, A2, etc. When I ran the above script, it went into a continuous loop and I had to halt it, then run sudo killall pq.exe to eliminate the 500 or so Progress Quest windows that popped open. Anyway, what is wrong with my script that I can't seem to get it to stop loading files at an arbitrary number? I want to get this part finished before I make any more boring named characters.
View 1 Replies
View Related
Aug 11, 2010
I would like to parse an input file in which there are two columns per each row. We want to see how many lines are duplicated where we define duplicate to be having the same second field and different first field. For instance if the input file looks like the following:
79874 13131
79873 12309
79820 13131
[code]...
View 10 Replies
View Related
Mar 4, 2010
I have limited experience in terminal, but let me first explain what I am trying to do to see if there is some easier way to do it. Basically I want to change the skin in aMSN. I downloaded the new skin but am unable to unzip or move it without /root permissions. I don't know how to acquire this without being in terminal. So I figured there had to be some way to go into the terminal and use it to move the unzipped folder from the desktop to the aMSN skins folder.
View 3 Replies
View Related
Jan 3, 2011
find . -type f -name "*" -print0 | xargs -0 --replace=% mv % `pwd`
works fine at moving all files anywhere below the current directory to the current directory.
My question is can it be modified so it only moves all the files up one directory layer? Otherwise I shall have to cd 100 or so times and run it in each directory i want to compress...
I imagine the directory below which the file is stored is in the % somewhere it is just a case of extracting it and applying it to the mv command, yes?
View 8 Replies
View Related
Jul 6, 2010
I have a single directory of pairs of files, with the pairs sharing a string as the beginning of the filename:
SF1-27F1492R-clone01_T3_A18_001.ab1
SF1-27F1492R-clone01_T7_A20_002.ab1
SF1-27F1492R-clone02_T3_A19_003.ab1
SF1-27F1492R-clone02_T7_A21_004.ab1
...etc
I need to create a subdirectory for each pair then move the pair into the subdirectory.
I accomplished the first step using:
$find /foo -name '*T3*' -exec mkdir '{}.wrk' ;
I can use a regex to designate the pair and associate the directory, but how do I use regex in a path as the output of a move command?
View 7 Replies
View Related
Nov 3, 2010
I want to itemize addition for each type of transactions. The example below is for 3 days,10/13/2010, 10/14/2010, & 10/15/2010.
PROD =89.25+89.25+104.47= 282.97
TAX =9.37+9.37+10.37 = 29.71
ETAX = 1+1+1 = 3
[code]...
View 8 Replies
View Related
Mar 31, 2011
I need to rename the resulted searched files from a loopI have the following code:
find . -name DOC* | while read i
do
find $i -type f -name '*.txt'
done
basically, I am searching for all txt files inside any folder starting with DOC name.this code is working fine with me.I need to rename those .txt files to .txtOLDOS: Ubuntu 10.4Bash shell
View 10 Replies
View Related
Dec 8, 2010
I am an uploader to a various hosts, so this tiny script me a lot. I make a rar archive and split files with 100mb. I could get 3-4 or even 76 parts of rar files and it would take me some time to paste all these urls to remote upload function of filehosting sites. For example:
Code:
server:/home/cober/downloads/teevee# ls -al
total 358784
drwxrwxrwx 2 root root 4096 Dec 8 19:38 .
[code]....
View 1 Replies
View Related
Jul 13, 2011
I have a lot of filenames (strings) following the same convention
m02_+1+7_London_0000$01.cfg
m02_+1+8_London_0000$01.cfg
m02_+1+8_London_0000$01.cfg
[code].....
What I want to do is to create a script that will interpret the following string and save into variables part of its name
m02_+1+7_London_0000$01.cfg as
------X-Y--City---------
X=1
Y=7
City=London
[code]....
then I want to copy the files that go all the files with the same City and X and Y to the same subfolder City/MX.Y I will need some help start doing that. And I think the first would be to get part of the filenames strings into variables.
View 3 Replies
View Related
Mar 11, 2011
I am trying to find a nightly backup if it was successfully copied over, rename it and curl, but it's always passing the check even if the file is older than specified. From the command line it does as it should. Example is here;
Code:
find /backup -type f -mmin +4440 -exec echo "found" {} ;
- nothing returned (good). Then I change the time
[code].....
View 4 Replies
View Related
Jun 15, 2011
I'm making a small script for searching and doing some operations with photos, but I'm kinda stuck on this little function:
Code:
function findallformat {
prefix=""
if [ $1 = -pre ] then
[code]....
That function should find for every file with a certain type; and you can specify a prefix using a "-pre" followed by the prefix that you want to search. The format should be "stackable", so you can use as many types that you want, without repeating the same function on the code.
Example: findallformat -pre IMG_ .JPG .CR2 #That should search files that start with "IMG_" and finishes with .JPG and .CR2. My problem it's that, when I try to use it on the script, it says "bash: syntax error near `token' unexpected `}'"
View 4 Replies
View Related
Jan 26, 2009
I want to catch some outputs in two different log files in bash, file simple.log and all.log So far, the script is started like this:
Code:
xterm -e "(./myscript.sh | tee -a simple.log) >& all.log"
What I want is:
- In simple.log, I want all the stdout but WITHOUT errors.
- In all.log I want BOTH stdout and stderr.
So all is ok, and the output becomes:
all.log
Code:
starting copying files
mv: cannot move ... : permission denied
copying completed
simple log
Code:
starting copying files
copying completed
But, in the new xterm, I'm loosing stdout. There is no output.How can I have the files logged as they are but also have the stdout in the xterm.
View 6 Replies
View Related
Apr 1, 2010
I'm trying to rename a lot of files getting rid of the space on the names. For that purpose I wrote this very simple bash script, but for some reason is not working.
Code:
for i in "$(ls)"
do
j=$(echo "$i" | sed 's/ /_/g')
mv "$i" "$j"
done But what I get in return for each line is just one long file name with all the file names concatenated. I've tried with echo -e "$i" as well with no results. This has to be something really simple that I'm missing but I just can't see it.
View 11 Replies
View Related
Nov 6, 2010
I am trying to write a simple script to list all the files in a directory. The script I wrote was as below where the pdb_files is a directory and all the files which I want to list are in that folder.
Code:
files=`ls -F pdb_files/*THERMO*`
for inFiles in $files
do
echo $inFiles
[Code]....
View 5 Replies
View Related
Mar 21, 2010
I have around 600 empty text files that I need to add the name of this file as part of the data, I meanfiles from "file1.txt to "file599.txt, all of them empty, and I need to get the name inside the file, so, when I open the file show the name as part the data "file1".these files were created on my web site, I am thinking in a small script in bash
View 5 Replies
View Related
Jan 14, 2011
How can I iterate over all the files in the current directory to check for certain permissions? This is what I have:
Code: #!/bin/bash
for file in *.tar.gz
do if [ -r "$file" ]
then echo "$file is readable"
else echo "$file is NOT readable"
fi done
But this only checks that the current user has read permissions for each file. I want to check that the group "others" has read permissions for each file. How can I do this? Is there a built in function to check if a file has read permissions for the "others" group? Otherwise, I thought I might be able to use this:
Code: $ stat --format=%a file
744 And parse the output "744" and make sure the 3rd number is between 4 and 7 (since the octals 4-7 have read permissions for others).
View 2 Replies
View Related
Feb 10, 2010
I'm building my first BASH programs and I have a hard time. I can't do a search in a folder and filter only script files without extension, the problem is to differentiate the script files from the others. I tried with ls I tried with find and i don't find a way to make it work.
View 8 Replies
View Related
Feb 2, 2010
Never mind, I figured it out myself. Firstly, the old version of BASH I'm using doesn't support
Code:
for i in {1..27}
So I had to use
Code:
for i in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
Secondly, it was simply
Code:
#!/bin/bash
for i in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
do
[code]....
View 2 Replies
View Related
Jul 22, 2011
I have about 50 files that the script will operate on, they are all located in the same directory.I need a bash script that will operate on all files in a directory. The script needs to add two lines to the beginning of each file based on the file name, and one line to the end of the file.A file named myfile.h should add these two lines to beginning of the file:
Code:
#ifndef MYFILE_H
#define MYFILE_H
[code]....
View 5 Replies
View Related
Jan 31, 2009
i was wondering if there's a bash script to check the amount of files in a directory with an IF statement..
View 8 Replies
View Related
Jan 26, 2011
I have wrote a 1 line command that parses a file, locates the IP Address in the file and then trims the output the way I want it, and then sorts numerically and by uniqueness and then >> appends to output.txt
I can get all the IP's into 1 file "output.txt", but what I am really looking for is some type of way to create a text file, for each IP it finds labeled xxx.xxx.xxx.xxx.txt and also put that ip address into that file..
xxx.xxx.xxx.xxx = the ip address it finds
View 14 Replies
View Related
Nov 9, 2010
However, the ffmpeg command generates a temporary file blahblah.mpg.tmp of about 1GB per hour of transcoded video.My issue is that I can't seem to delete these files automatically from any bash script.Now from the command line, I can cd to the directory and just rm -f *.tmp and they get deleted. However, from my script, that same command doesn't remove those files. I thought maybe the file was in use, so I put a sleep command in for like an hour before the delete happens, but it still fails. I also put rm -f /mnt/mythtv/*.tmp in a root cronjob and it still doesn't delete the files.
If I just rm *.tmp I do get a prompt about "Are you sure you want to delete this write protected file?". But the -f switch seems to work fine as a normal user from the command line and just delete them.Does anyone have an idea how to troubleshoot this problem? The particular filesystem that the tmp files get generated on is on it's own xfs partition mounted as /mnt/mythtv.
View 8 Replies
View Related
