General :: Script To Extract Appropriate Line From Website Using Links
Oct 26, 2010
I need to write a script called '~/get_birthrate' which when invoked with a two letter country abbreviation(i.e, au,ch,ni), extracts the appropriate line which contains the information about the country's birth rate from the [URL] (where "ca.html" should be replaced with the appropriate two letter abbreviation). The output should look like:
$ get_birthrate au
8.69 births/1,000 population (2007 est.)
$ get_birthrate ch
13.45 births/1,000 population (2007 est.)
$ get_birthrate ni
40.2 births/1,000 population (2007 est.)
View 4 Replies
ADVERTISEMENT
May 29, 2010
I have a website that has a massive list of royalty free stock photos and I want to download all of them. I have bought a membership for [URL] so I am able to download as much as I want from them for the next month.
Instead of going page by page and manually downloading each set of stock photos manually, I would like to automate this process. Here's my idea:
1. Download the website with the links to hotfile [URL]
2. Use grep to retrieve all the links to [URL]
3. Feed the links I recieve from grep into wget and download the works of them.
The problem I'm getting is when I use grep, It retrieves the entire line of html code where "hotfile.com" is shown. So here is an example of one link I receive in the output:
Quote:
./1776-santa-claus-vector-set.html:<div align="center"><a href="http://hotfile.com/dl/18418176/181a55b/Santa_Claus_Vector_Set.rar.html" target="_blank">HotFile</a></div>
Is there a way to just have the link shown in the output?
PS: I have everything else working, I just need an automated process of getting all the links.
View 5 Replies
View Related
Mar 22, 2011
Im trying to extract the href of a <link> tag from a html page however as some links contain further preferences I seem to be unable to extract them, do you have any idea how I can write this: Link:
[Code]...
View 9 Replies
View Related
Oct 12, 2010
I have a text file of n-number of tab-delimited lines ("INPUT") which I would like to parse line-by-line to a text output file depending on the SampleID of the line. These lines contain a unique SampleID and each subject has several lines of data.
[code]....
I also have a text file of relevant SampleID ("INPUT2"). The basic idea is that I read a line from INPUT, split the tab-delimited line, extract the SampleID from the split line, compare the SampleID of this line to my list of relevant SampleIDs. If there is a match, then print the line from INPUT to OUTPUT, then move on to the next line of INPUT. Alternatively, if there is no match, then move on to the next line in INPUT. I tried to script this (extreme newbie at perl right now) and failed miserably, but here is what I have at the moment:
[code]...
View 4 Replies
View Related
Feb 3, 2010
I'm not sure how to explain my situation. I would like to download the file <https://www.vmware.com/tryvmware/p/activate.php?p=free-esxi&lp=1&ext=1&a=DOWNLOAD_FILE&baseurl=http://download2.vmware.com/software/vi/&filename=VMware-VMvisor-Installer-4.0.0.Update01-208167.x86_64.iso> via the command line. I've tried a few different methods with wget, the best I get is an index.php file. I'm not at all familiar with php but a search for "wget php" yielded nothing helpful.
View 3 Replies
View Related
Apr 8, 2011
My hosting server is running Linux / Apache. It would be very nice to be able to link some files (preferably hard links, but symbolic links also would help), but haven't a clue how to do so. I would be willing to write a server side php script if that would do the trick.
View 4 Replies
View Related
Apr 29, 2010
Recently Firefox is showing some but not all the links inside a page in a website that I use every day.Firefox had worked well in the past years with this site, so I made a few tests:- I tested the web page on Firefox 3.5.9 on Fedora 12. Some links missing.- I tested the same web page on IE8 on Windows Vista. No issues, all links there.- I tested the same web page downgrading Firefox to version 3.5.4 and downgrading the required libraries. Same issue, some links missing.- I compared the html code that the web server hands to both IE8 and Firefox. They are exactly the same.I don't know enough about html or javascript to understand why Firefox doesn't like some of this code portion.
Code:
<td align='left'>
<p class='liga'><a href="javascript:GeneraId('11')"><img src="image.gif" BORDER=0
[code]...
View 6 Replies
View Related
May 24, 2010
The problem actually lies with Firefox (3.0.19) After I solved my problems, everything looked fine at first. That was until I went to one particular website and found that some links were missing. They simply were not showing in the browser. They were showing in the source code, so they should have appeared. Now, I have two different profiles by using:
[Code]...
View 9 Replies
View Related
Mar 9, 2011
I would like to save a website as pdf document, but I search for a method that preserves the links of that website and makes them clickable within the pdf file. Every method I found so far removes the links and leaves only all things visible, like printing. There is an thread from 2007 about the same topic but it didn't came to a conclusion either [URL]....
View 2 Replies
View Related
Dec 17, 2010
What would be the best way to extract data by sending queries to a website?
View 2 Replies
View Related
Aug 9, 2011
I need to download a file from a website which has a URL formatted like:
[URL]
This redirects to a .zip file which has to be saved. There is also a need to authenticate based on username and password.
I tried to use wget, curl and lynx with no luck.
UPDATE:
wget doesnt work with redirection. It simply downloads the webpage instead of the zip file. curl gives the error "Maximum redirection exceeded > 50 " lynx also gives the same error.
View 1 Replies
View Related
Feb 1, 2011
I need to extract the 7th line below C-FM and every third line after that. How can I do it? I've tried using grep but I get all the lines in between. An example of the text I am working with is shown below.
C-FM RADIAL POINT-WISE DATA
(P,K-INF,BU)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
[code]....
View 2 Replies
View Related
Jul 26, 2010
I am running Linux from a DVD, not installed. I am not good with installing software, but since the DVD cannot be corrupted, I am content to operate this way. Lately, I have been having problems that previously did not occur. When I try to click on the checkbox to get rid of emails, it doesn't register in most cases, or when it does, I am clicking multiple times so it registers twice, meaning it is unchecked again. Even more frustrating is some issues that are affecting my ability to update my business. I am trying to modify spreadsheets (text not calculations).
Whenever I try to click & drag to select something to change, it keeps jumping around to select only some of what I want, something else or some combination of the 2. When I try to copy and paste several fields from 1 column to another, everything from the several fields in the source column ends up together in the last field in the target column. I am also trying to download some images from a website. There is a single column of links to the images. I have to click on the link to get to the image in order to copy it, then back out to continue looking for more links to do the same.
My computer keeps jumping back 2 steps, then forward 2 steps, and sometimes I lose my place in that list. I could deal with it if it were a small number of links, but this is a list of probably close to 20,000 links. Again, i am operating off of a live DVD so this should not be corruptible, but this has just started happening, and has been an issue the last several sessions.
View 14 Replies
View Related
Aug 24, 2010
I'm trying to figure this error message out. This little script is supposed to tweet my laptop's IP address, as a cron job, I'm hopeful that it would do so even if it's stolen. This is a variant of one that works, but this doesn't, and I can't see a difference in the curl line of either one.
Code:
#!/bin/bash
user="xxxxxx@xxxxxxxxx"
pass="xxxxxxxxxxx"
wget [URL]
TWEET=`sed -n 1p index.html`
curl --basic --user "$user:$pass" --data-ascii "status=$TWEET" "[URL]"
rm -f index.html
exit
This is the error message.
Code:
curl: (6) Could not resolve host: status=66.183.103.67; Cannot allocate memory
{"request":"/statuses/update.json","error":"Client must provide a 'status' parameter with a value."}
Why does curl think the status is the URL?
View 7 Replies
View Related
May 25, 2010
im trying to find a way to extract the phrase between the words Connection and is (ie the underlined words below). Can we use awk to do this? How? Is it the best command to use?
Code:
[06:25:00][i] Connection at Plant A is live
[06:25:00][i] Connection at Building_C is not live
[07:25:00][i] Connection at Terminal D is down
View 12 Replies
View Related
Aug 10, 2010
I trying to extract via command line the latitude and longitude with this command Code: curl -s [URL]
View 4 Replies
View Related
Oct 20, 2010
I am trying to figure something out as part of something else I am going to do. I was just wondering the easiest and quickest way to run a loop at the Linux command line which will make x amount of symbolic links when I need them.
I am using Debian 5.
I figure its something as easy as ~# foreach statement, then I just do a loop with the command to make a sym link.
View 5 Replies
View Related
Jan 14, 2011
I have installed centOS Server edition on my laptop and installed apache and mysql on server, i wanted to understand how can i make my server on line and host my website on my own server. I am using MTNL Broad band, and i have one static ip, How to make web server in centOS 5.5.
View 7 Replies
View Related
Dec 6, 2010
At my Uni, we use a web-based login for our internet connections. Its based off of Cisco, and every Wednesday night every computer on campus must re-enter their credentials to use the network.
Normally on my several computers I simply pull up the Terminal, point links to google.com using
Code:
And enter my credentials when Cisco redirects to the login page.
Literally, the process is
Code:
Then ENTER to accept the redirect, down arrow to skip over the logo image, USERNAME, ENTER, PASSWORD, ENTER, ENTER.
Naturally, this is EXTREMELY time consuming, as I have about 5 computers located around campus and must physically walk to the machines and login every single week.
My question is, How would I formulate a program that does the following;
1) checks for connectivity (i.e. is able to reach/resolve to the greater part of the internet) and
2) automatically fills in the credentials on the links login page?
View 2 Replies
View Related
Feb 3, 2009
I have a personal wiki of notes, with now thousands of links in markdown format:
[link text](http://example.com)
but now that fckeditor is available for mediawiki (very beta), it has become much better to just stick with wikitext format. There are only a few conversions to do: tables, links, and bulleted lists. The lists are a fairly simple regex and fckeditor magically reformats the tables, so all I'm left with is the links. But I'm not a regex master. How do I reformat code...
View 12 Replies
View Related
Feb 20, 2011
I can see some soft links in /etc directory which are pointing to /etc/rc.d Directory contents.
Code:
lrwxrwxrwx. 1 root root 7 Jan 31 08:19 rc -> rc.d/rc
lrwxrwxrwx. 1 root root 10 Jan 31 08:19 rc0.d -> rc.d/rc0.d
lrwxrwxrwx. 1 root root 10 Jan 31 08:19 rc1.d -> rc.d/rc1.d
code....
Any body please tell me what is the purpose of these soft links in /etc directory ? I am using RHEL 5.4 ...
View 3 Replies
View Related
Jan 14, 2011
why scp command not copy links from local copmuter to other ( how to copy the links)as scp -rp dir linux:/dir_targetremark in dir I have files and links
View 1 Replies
View Related
Oct 7, 2010
How to create cron tab when DSL line down set automatically restart the network service while DSL line up?
View 3 Replies
View Related
Feb 8, 2010
After running the following command, I get:
[root@yukiko /]# find / -iname .bashrc
/home/clamav/.bashrc
/home/vpopmail/.bashrc
/etc/skel/.bashrc
/root/.bashrc
But I would like to have a command that prints a specific line by supplying the command with the line number, for example:
[root@yukiko /]# find / -iname .bashrc | getline(2)
/home/vpopmail/.bashrc
Is there such a command on CentOS?
View 3 Replies
View Related
Sep 11, 2009
I have several files with many lines something like this:
I'm trying to write a script that will count the number of characters per line that doesn't contain a ">" symbol and give me an average of those values. I have most of the script together but I can't figure out how to connect some of the steps.
Code:
View 3 Replies
View Related
Dec 23, 2009
I am trying to write a script that takes an input file ($FileName) and an intermediate file ($FileName.info) and removes lines from $FileName if the value in $2 of $FileName.info is <75.
I can't figure out how to feed only one line of the .info file to the if statement at a time so that it will perceive it as an integer instead of a list.
The error I am getting now is ./script.sh: line 6: [: : integer expression expected
Sample input $FileName
Code:
Code:
Code:
Script so far:
Code:
View 10 Replies
View Related
Jun 29, 2010
I am trying to extract files from 2010_06_25_RT3572_Linux_STA_v2.4.0.0.tar.bz2 and every time I try to extract it it comes up with that error. I went under Applications, Ubuntu software center, typed in .tar.bz2 and installed Archive manager, Fast, multi-threaded bzip2 utility.
I am not sure what to do anymore since I have just install Ubuntu
View 8 Replies
View Related
Jan 21, 2011
I know a .bin file is an executable file type in linux. We have an error after installing it and it referes to a file name and a line number within the file. I'm trying to find out if the file is part of the .bin file but I need a way to see what's inside of it or extract it.
View 5 Replies
View Related
Aug 23, 2010
I have linux ubuntu 10.04... and i download a .ISO of windows 7. because i want install windows 7 because i need some programs of windows for my work.. so i need make a dual boot. but the problem it's not net....
So i have a .ISO ( of windows ) and i have a notebook. ( no Drive cd. only USB PEN's ) where i can Extrat the .iso file ( windows system ) to my pen ? i cant extrait i try but nothing works.
View 13 Replies
View Related
Feb 22, 2011
I would like to extract a substring using sed.
the String is :
[1365465464.1654] fasfa fsaf df16A fas 2.2 (7/2134)
number result :
16A
View 1 Replies
View Related