Programming :: Bash - Read File Without Whitespace?
Mar 14, 2011
I am struggling with Bash scripting at the moment (I can't seem how anyone can write scripts with this language!!!) I have a need at home to have a cron job execute daily to lookup my downloads.txt file, read each url (per line) and download content from that url. Then that entry needs to be removed (well I keep all urls in memory and clear the file afterwards). If an error occurred during the download process, then the url is written to a downloads.err file. I got all the above working except for properly reading the url from the text file without including newline characters. I am using the following to read:
while read url; do
--Do whatever here--
done < downloads.txt
How can I get it not to let the url variable have newline characters?
I have a script that reads part of a line, delimited between the first and second intended part by a colon. Then it "chops" the part after the colon, which are words offset by commas (counting them beforehand so as to catch every word in the string's second part), like this:
Code:
"COLORS.JPG:red,orange,yellow,green," (Returning) red
[code]....
single script that parses/breaks both parts of a line like this "COLORS.JPG:red,orange,yellow,green;blue,indigo,violet," so that the two parts, separated into single words (or two and three words, sometimes with spaces) can be used as single-line annotations and written to JPEG files using Exiv2. So far, I haven't been able to come up with a script that does this without one part of the total string(usually that part after the colon) becoming the first word in the second array. In other words, I look for this:
KEYWORDS:
[ ]red [ ]orange [ ]yellow
[code]....
Or vice-versa (ie, the second array winds up as a single-line "member" of the first). I think it's because I'm using a single while read loop to read the text file in which the filenames and substrings happen to be. If there's some way of reading a file once and going back to the beginning to read it again in another while loop, I haven't found it.
For example, I have a text file with data which lists numerical values from two separate individuals
Code: Person A 100 200 300 400 500 600 700 800 900 1000 1100 1200
Person B 1200 1100 1000 900 800 700 600 500 400 300 200 100
How would I go about reading the values for each Person, then being able to perform mathematical equations for each Person (finding the sum for example)?
At my wit's end I can't find anything that I understand well enough to use. This is for a Unix class, we are working with shell scripting. File1 has 5 in it and File2 has 100 in it.The teacher wants us to read the values then do the math. This is what I have so far:#!/bin/bashvar1='cat File1'var2='cat File2'var3=`echo "scale=4; $var1 / $var2" | bc`echo The final result is: $var3
So far so good. Now, I want to define two variables (e.g. e1,e2) in the bash file, so that their values would correspond to 00.00 and 30.00, as read from the input file. This one I have not found yet, thus asking for your advice. At the end, writing echo $e1 $e2, I should get 10.00 30.00 This is even harder to me: I want to replace the values emin,emax in a new file "modify.dat" which looks like that:
with the values e1 and e2 I have in my bash file. In other words, I want to call "modify.dat", find these two lines and replace the numeric values with the e1 and e2. At the end, my file should be like:
Code: #!/bin/bash trap "echo 'you got me'" SIGINT SIGTERM # to trap ctrl+c echo "Press ctrl+c during 5 sec loop" for ((i=0;i<5;i++)); do
[Code]...
How come code behaves normally and stops when ctrl+c signal is caught and resumes, but after I use at least one timeout read in the code it looks like, if signal is caught again it doesn't pause the execution but skips the loop. If you remove -t (timeout) option from the read, both loops look the same!
I'm trying to read content of file to variable and use this variable in for loop. The problem is, when I have c++ comment style in file - /*. Spaces in line are also interpreted as separated lines.
For example:
Code:
Changing $files to "$files" eliminate these problems but causes that whole content of variable is treated as one string (one execution of loop).
bash 3.1.17(2) I'm trying do write a shell script which must operate on each line of an ASCII text file. So, all the code must be inside a loop, and inside the loop, the first thing should be to read the next line from the file. I have the bash read command. But it reads from stdin. Any way to make read from a file?
I had a script once that had in it a while true; do and read lines or data from a file similarly to a while read; do. I have a script I'm working on now where I'm trying to "prove the negative" that certain substrings culled from one file do not appear in another file. I'll try to explain what I mean with an example. An item in one list might be
Code:
[URL]
But the other list may only have
Code:
harvey1996nicks:Six stores in north London harvey1997nicks:Three stores in Bristol harvey1999nicks:Eight stores in the lower Midlands
The problem is, other items in the first list do match items in the second. The script I'm drawing this from has already pulled those out and written them to one (or several) lists that I've been able to use with other programs (Exiv2 specifically). The furthest I've bothered to go in the direction of "proving the negative" is a slow "if/then/fi" loop that looks like this:
Code:
linecheck=$[linecheck+1] if [[ "$linecheck" == "$axe" ]] && [[ "$matchX" == "" ]]; then echo -e $file>>nomatches.txt; fi
linecheck is counting the lines of the second file -- the one with the designations or "labels" (the "so many stores in ..." from my example) -- as they are checked against the file names in the first (the "harvey199*nicks.jpg" etc of my example. axe is the number of lines in that first list (culled from a cat |wc -l at the beginning of the script). matchX is the variable the "prove the positive" script used for matches, one list to the other, which then got written line by line to a text file.
Greetings, I have installed a VMware host on Windows 7 64bit and followed with a 10.04LTS 64 bit guest install. Everything went smooth. I have used VMware before, but the other way around.. Linux host and Windows guest.
So this time I am installing the VMWare tools on the guest, ubuntu 10.04, but it seems to be failing. The vmware tools install ran fine the first time with no errors, but I was getting the follwoing error after logging into the Ubuntu guest afterwards.
Could Not Apply the Start Configuration for Monitors Error Line 1: Char 1 Whitespace or empty file
So I re installed the VMware tools once again, but still the same.
I am hoping that someone here has come across this before and hs a workaround. I don't think the vmware tools was installed correctly.
I have been looking for a script example of reading and writing to the parallel port's data, status, and control registers using bash. I see it done in pascal, tcl, etc. but nothing in bash.
test=(1 2 3 4 5) for car in ${test[@]} do echo "Element : $car"
[code]....
if variable $car equals 1, new element is added "6" to an array. But i don't know why when i am printing all elements of this array (echo "Element : $car") this element ("6") is not mentioned, but if i make a command which check an amount of contained elements by array it will be 6 elements.
As I'm starting to learn bash scripting I'm trying to automatize some tasks I usually perform. I have a notification mail I need to send several times a day. It has this structure:
Quote:
Dear user, blah blah blah blah
You need to contact the following people:
[code]...
To replace "user", I found this:
Code:
read -p "Please enter username: " username echo "Dear $username,"
Which probe to be very useful with other simple notifications like this. But I don't know how to manage the email addresses as they are usually more than one and could vary from 1 to 10. They should appear one above the other. I found this: "Here is a little work around. The only thing the user needs to do is hit enter without anything else on a line and it will close out"
Code:
#!/usr/bin/ksh word=a until [[ $word = "" ]];do
[code]....
I tried to use it and modify for my needs but I failed, I don't realize yet how can I use it. If possible, I would like to use the until loop like the above example just for learning purposes but any other form will be accepted as well.
First, I made a simply script which let me download a file from a filehosting site on my server, but I can only put one link there, so I need help how to put multiple links in.Here is my script:
Code: #!/bin/bash echo -n "Please insert your name: " read NAME
[Code]....
It works fine, but is it possible to have default values using bash read(1b)? I mean if user is prompter for sex, he just pushes ENTER and by default 'm' is chosen. Or if user is prompted for hometown and he doesn't insert anything, but just pushes ENTER button, 'New-York' sis chosen by default Are such default values possible in bash?
I have not been able to write bash to use a PID file to ensure no other instance of the same script is running! All three methods I can think of to see if the PID in the PID file is another instance of the script make the script exit with a return code of 1 but the same commands run at the command prompt work as expected.
The first attempt was:
Code:
The first attempted workaround was:
Code:
The second attempted workaround (with debug to make the following command prompt copy and paste meaningful) was:
Code:
Here's the command prompt session, testing with a stale PID file and then manually running the problem command and it behaving as expected:
Code:
This on Slackware64 13.1 which has bash 4.1.7.
In desperation I tried rebooting but the behaviour was the same.
AKA "zipping on the fly .. the slow-as-molasses way." The list includes full pathnames to each file, and they're all in subfolders of the same parent folder (which, unfortunately, is not the root folder of the drive or system on which the files reside). A cleaned-up and radio-ready portion of the list looks like
What I'd like to be able to do is zip all the files in the list into a single archive, to avoid the step of having to copy them to the same location (presumably another folder on the HD) and then zip that folder. I'm more inclined to make provisions about extracting to a single folder at some other time. Is this possible in BASH, or would I have to consider a faster, more robust scripting language such as python or perl?
Here's a challenge I've been struggling for months with:
I have a bash script that reads URL addresses of our internal server and then executes some test commands on them. Something like this:
Code: read -p "Enter URL: " url sh execute-what-ever-to $url
After copy-pasting the URL the user taps the enter key and the script proceeds, but here comes the tricky part: I want this to work without the need to press the enter key after copy-pasting the URL.
"read -n" does not work in this case, as the URLs vary greatly in length. However, the URLs always end to the same string. They could be like "http://url1/END", "http://url2/END" and so on. So this ending string "END" could be theoretically used to recognize that the whole URL has been pasted.
PI'm trying to write a script to list all open ports in the MINIUNPND chain in iptables and use the procotol, port and destination ip to open ports on another router using upnpc.Here is the output of iptables -L MINIUPNPD
No matter what i do i cant seem to remove the first 4 characters from the MYPROT array to leave only the digits. Also i cant seem to read the array back???
I thought it would simply be a loop reading each line and passing the fields in variables, executing upnpc commands i need then moving to the next line of the file until it reached the EOF.
I googled and tried to find an easy step by step-by-step guide on how to use a bash script read a variable from a file. This is the way I did it (but it does not really work so something is wrong, but what?) (testfil2 contains one line that readsidnumber=1578
Trying to create a small script that will read user's input, test if user entered some input and if not display some message or display a text using user's input.
The script is the following but i get an error saying "[: 6: =: argument expected"
Q=What will this do? A=This will read from /root/Desktop/iplist (or anywhere you want to put the list) after it reads the ip it will ping the ip two times. If the remote host is up then it will write the ip to a log. If the ping fails it will display it on the screen, write the ip to syslog so that you can do remote logging, and last it will send the ip by email to a user or a group of users.
Code: #!/bin/bash #wrote by Zach Underwood feb-12-2011 #test and wrote on centos 5.5 with sendmail rm -f /root/Desktop/iplog #This will remove the good iplog exec 9</root/Desktop/iplist #put full path the file where ip list is. Make sure that the ips are listed one per line. while read -u9 ip do ping -c 2 $ip > /dev/null 2> /dev/null # ping and discard output if [ $? -eq 0 ]; then # check the exit code echo "$ip is up" >> /root/Desktop/iplog #write the ip of the hosts that are up else echo "$ip is down" # will show on the screen the ips that are not up logger -p auth.notice "$ip is down" # will write the down host to syslog echo "$ip is down" | mail -v -s alert user@anydomain.com > /dev/null 2> /dev/null #will send mail users about host down fi done
I've searched everywhere and I can't come up with a good solution. For each line I need to find the average, min, and max. I've seen plenty of solutions where the number of columns is fixed, unfortunately for me these lines can get pretty large. My thought was to read each line individually into an array, loop through the array and find the avg, min, and max that way but i haven't had much luck. I can read each line using a while loop but I'm having trouble with the array part, or perhaps that's not the best solution?
I have a script almost working except for 1 thing. What I'm trying to do is read a file that has the files that need to be FTP'd using a bash script. I have everything working except the reading of the file. It works outside of the ftp script I've wrote but once I put it in the FTP script it doesn't.
Here's the Script:
#Here's where the problem is that I know of
I've been playing w/ the exclamation points to see if that could be the problem, but so far no luck.
I have a text file that contains a single word and I want to write a bash script that will read the word from the text file... The following is my incorrect attempt, as it assigns the name of the textfile to the variable as opposed to the word stored within the textfile:(assume I have a text file value.txt that has its contents a single word, say wordone)
Code: #!/bin/sh for f in value.txt do echo $f done
so the output of the above script is value.txt, however I want it to be wordone.to summarise: how do I assign the value of the word contained within a textfile to a variable?
