Debian :: Using Wget To Download Site For Offline Viewing
Nov 25, 2015
This is the command line switch I am using:
Code: Select allwget -p -k -e robots=off -U 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.6) Gecko/20070802 SeaMonkey/1.1.4' -r www.website.com
For some reason it seems to be downloading too much and taking forever for a small website. It seems that it was following alot of the external links that page linked to.
But when I tried:
Code: Select allwget -E -H -k -K -p www.website.com
It downloaded too little. How much depth I should use with -r? I just want to download a bunch of recipes for offline viewing while staying at a Greek mountain village. Also I don't want to be a prick and keep experimenting on people's webpages.
View 3 Replies
ADVERTISEMENT
Mar 14, 2011
i use this code to download :wget -m -k -H URL... but if some file cant be download , it will retry Again and again ，so how to skip this file and download other files ,
View 1 Replies
View Related
Mar 29, 2011
How do you instruct wget to recursively crawl a website and only download certain types of images? I tried using this to crawl a site and only download Jpeg images:
wget --no-parent --wait=10 --limit-rate=100K --recursive --accept=jpg,jpeg --no-directories http://somedomain/images/page1.html
However, even though page1.html contains hundreds of links to subpages, which themselves have direct links to images, wget reports things like "Removing subpage13.html since it should be rejected", and never downloads any images, since none are directly linked to from the starting page.I'm assuming this is because my --accept is being used to both direct the crawl and filter content to download, whereas I want it used only to direct the download of content. How can I make wget crawl all links, but only download files with certain extensions like *.jpeg?
EDIT: Also, some pages are dynamic, and are generated via a CGI script (e.g. img.cgi?fo9s0f989wefw90e). Even if I add cgi to my accept list (e.g. --accept=jpg,jpeg,html,cgi) these still always get rejected. Is there a way around this?
View 3 Replies
View Related
Dec 24, 2010
i want to download android developer guide from google site but code.google is forbidden from my country i want to use wget to download entire android dev guides with freedom( proxy ) that i set in firefox these for open forbidden sites ( 127.0.0.1 port:8080 ) i use this command to download entire site
Code:
`wget -U "Mozilla/5.0 (X11; U; Linux i686; nl; rv:1.7.3) Gecko/20040916" -r -l 2 -A jpg,jpeg -nc --limit-rate=20K -w 4 --random-wait http://developer.android.com/guide/index.html http_proxy http://127.0.0.1:8080 -S -o AndroidDevGuide`
[Code]....
View 4 Replies
View Related
Aug 10, 2011
I want to do something simular to the following:
wget -e robots=off --no-clobber --no-parent --page-requisites -r --convert-links --restrict-file-names=windows somedomain.com/s/8/7b_arbor_day_foundation_program.html
However, the page I'm downloading has remote content from a domain other than somedomain.com. It was asked of me to download that content too. is this possible with wget?
View 1 Replies
View Related
Mar 25, 2011
is there a way that I can archive an entire website (to a local drive) that makes heavy use of php so that I could view at a later time OFFLINE?
View 7 Replies
View Related
Feb 2, 2010
I am trying to download the contents from [URL] to my local system for off-line browsing but am having little to no success. I have tried using wget and httrack, although I can download the directory structure there does not seem to me any sfw files.
View 7 Replies
View Related
Apr 25, 2010
I've looked around the other threads as well as the wget man page. I also Googled for some examples. I still cannot work it out. From the page [URL] I want to download the 48 linked files and their corresponding information page.To do this (the first file) by hand I click on the line that saysApplications (5) Go to the first optionDell - Application Open and copy the linked pageApplies to: Driver Reset Tool Then back on the first page click on the Download button. On the window that opens up I choose to save the file.
Then I move on to the next option (which is Sonic Solutions - Applications) and repeat this until I would have all my files. I do not want to download the many other links on this page. Just the above mentioned, so I can take it back to my internet-less place and refer to it as if I was on the net. I am using the 9.10 LiveCD at my friends place.
View 2 Replies
View Related
Sep 6, 2011
I need to mirror a website. However, each of the links on the site's webpage is actually a 'submit' to a cgi script that shows up the resulting page. AFAIK wget should fail on this since it needs static links.
View 1 Replies
View Related
Jan 19, 2010
I want to replicate this small howto (http://legos.sourceforge.net/HOWTO) using wget.However I just get a single file and not the other pages and that file too is not html.
View 4 Replies
View Related
Nov 4, 2010
I am trying to wget a site so that I can read stuff offline.I have tried
Code:
wget -m sitename
wget -r -np -l1 sitename
[code]....
View 7 Replies
View Related
Dec 21, 2010
can we use recursive download of wget to download all the wallpapers on a web page?
View 5 Replies
View Related
Sep 17, 2009
I was trying to download MOPSLinux from their Russian FTP server, using Firefox-->FlashGot-->KDE-Kget and it kept sitting there for about a minute, then popping up a dialog box asking for a Username & Password to access the FTP site.
I tried the usual anonymous type of login information combinations, to no avail; the box kept reappearing.
Finally for the heck of it, I tried Firefox-->FlashGot-->Wget and presto! It began downloading right away, no questions asked.
This is on Slack64 with the stock KDE installation + the KDE3 compat libs.
Here's the transfer currently going on the Wget window:
Code:
View 6 Replies
View Related
Mar 22, 2010
when I try to view php files on my linux box, they want to download instead of viewing them. I configured apache for php as the manual said but for some reason it doesn't want to parse the php. the http.conf file may need to be changed, that the line "AddModule mod_php4.c" was missing in the conf, however the AddModule and ClearModuleList directives no longer exist in the newer versions of Apache. These directives were used to ensure that modules could be enabled in the correct order. The new Apache 2.0 API allows modules to explicitly specify their ordering, eliminating the need for these directives.
View 8 Replies
View Related
Apr 18, 2011
I am about to loose my internet soon, I am not sure for how long, but I am curious, Can I go to another computer that has internet and download updates for my computer, take them back to my computer and install the updates so I can stay up to date?
View 4 Replies
View Related
Mar 5, 2011
I know I can build a local repository but I'd like to try just moving the appropriate .deb files. My problem is not knowing which files I need and it what order. Example...
I want to install nfs-common
Doing apt-get install nfs-common --- does it all for me when I'm online. So I looked in the /var/cache/apt/archives to see what was installed. I found two nfs files...
nfs-common_1.2.0-4ubuntu4.1_amd64.deb
nfs-kernel-server_1.2.0-4ubuntu4.1_amd64.deb
But when I tried to install those on another machine I found I was missing additional files.
libgssglue1_0.1-4_amd64.deb
libnfsidmap2_0.23-2_amd64.deb
librpcsecgss3_0.19-2_amd64.deb
portmap_6.0.0-1ubuntu2.1_amd64.deb
For future installations. How do I find all the dependencies and the ORDER they need to be installed so I can write my own script and install them to a machine that is offline?
View 2 Replies
View Related
Jun 14, 2011
I see that I can buy, download a Wikipedia DVD, so I can use Wiki. offline. But it is only for Windows. Is there a Version for Ubuntu? I use U. 10.4.
View 3 Replies
View Related
May 12, 2009
I am away for two weeks in a Internet free zone - unless I can get it back on - I would like to download the wiki, if possible to browse and try new things. I can update my comp, but will need to take it to a friends to connect.
View 1 Replies
View Related
Oct 6, 2010
I'm doing this wget script called wget-images, which should download images from a website. It looks like this now:
wget -e robots=off -r -l1 --no-parent -A.jpg
The thing is, in the terminal when i put ./wget-images www.randomwebsite.com, it says
wget: missing URL
I know it works if I put url in the text file and then run it, but how can I make it work without adding any urls into the text file? I want to put link in the command line and make it understand that I want pictures of that certain link that I just wrote as a parameter.
View 1 Replies
View Related
Feb 21, 2010
I'm trying to download a set of files with wget, and I only want the files and paths "downwards" from a URL, that is, no other files or paths. Here is the comand I have been using
Code:
wget -r -np --directory-prefix=Publisher http://xuups.googlecode.com/svn/trunk/modules/publisher There is a local path called 'Publisher'. The wget works okay, downloads all the files I need into the /Publisher path, and then it starts loading files from other paths. If you see [URL]..svn/trunk/modules/publisher , I only want those files, plus the paths and files beneath that URL.
View 2 Replies
View Related
Mar 6, 2011
I would like to use wget to downlaod file from Redhat linux to my windows desktop , I tried some parameter but still not work , can advise if wget can do download file from linux server to windows desktop ? if yes , can advise how to do it ?
View 14 Replies
View Related
Oct 16, 2010
I have a link to a pdf file, and I want to use wget (or python) to download the file. If I type the address into Firefox, a dialog box pops up asking if I want to open or save the pdf file. If I give the same address to wget, I receive a 404 error. The wget result is below. Can anyone suggest how to use wget to save this file?
View 1 Replies
View Related
Jul 20, 2009
I am helping a friend start with Ubuntu and he doesn't have as fast an Internet connection as I do. I was wondering how I could easily download all the deb packages for the software I want to install for him. It seems doing:
sudo apt-get install -d --reinstall <package>
Will download the packages for me, but it doesn't get the dependencies because I have already downloaded them... is there a way to get apt-get to get the dependencies as well?
View 6 Replies
View Related
Mar 25, 2009
Use Firefox 3.0.7
Use GNU Wget 1.11.4
I have a question about downloading web pages. If I download with web page complete will I be able to open the pages without being on line, or will there be some pages that I will still need to log in for. If a web browser is not sufficient, is there some command I can use with Wget to accomplish this?
View 5 Replies
View Related
Jun 21, 2010
is it recommended to download an iso file of fedora 13, will the file be destroyed?because i did it twice and it seems not working.
View 6 Replies
View Related
Jun 29, 2010
I'm trying to download two sites for inclusion on a CD:URL...The problem I'm having is that these are both wikis. So when downloading with e.g.:wget -r -k -np -nv -R jpg,jpeg, gif,png, tif URL..Does somebody know a way to get around this?
View 2 Replies
View Related
May 14, 2011
Let's say there's an url. This location has directory listing enabled, therefore I can do this:
wget -r -np [URL]
To download all its contents with all the files and subfolders and their files. Now, what should I do if I want to repeat this process again, a month later, and I don't want to download everything again, only add new/changed files?
View 1 Replies
View Related
May 6, 2011
if there is a mirror I could use to download a recent version of Ubuntu (e.g. natty). I'd like to use wget but can't find an address for a mirror.
View 3 Replies
View Related
Jul 28, 2011
I want to try to download an image of the earth with wget located at [URL] which is refreshed every 3 hours and set is as a wallpaper (for whom is interested details here). Wen I fetch the file with Code: wget -r -N [URL] the jpeg is only 37 bytes and of course too small and not readable.
View 5 Replies
View Related
Jul 2, 2010
I'm trying to download all the data under this directory, using wget: [URL] I would like to achieve this using wget, and from what I've read it should be possible using the --recursive flag. Unfortunately, I've had no luck so far. The only files that get downloaded are robots.txt and index.html (which doesn't actually exist on the server), but wget does not follow any of the links on the directory list. The code I've been using is: Code: wget -r *ttp://gd2.mlb.***/components/game/mlb/year_2010/
View 4 Replies
View Related
