Ubuntu :: Unable To Mirror Site Using Wget?
Nov 4, 2010I am trying to wget a site so that I can read stuff offline.I have tried
Code:
wget -m sitename
wget -r -np -l1 sitename
[code]....
ADVERTISEMENT
CentOS 5 :: WGet Syntax To Download Repo From Mirror
Jan 21, 2010After reading a lot of docs, I'm still having problems using wget to download a Centos repo from a mirror. Here's my best attempt so far:
$cd /repos/centos/5.4
$wget -r -nH --cut-dirs=3 -np [URL]
Of course I get all the unwanted index files etc, but I seem to get a lot of other downloads from the mirror, not just their 5.4 directory. It's like it's following other links on the web pages. Maybe I should be using "ftp://" instead of "http://" considering it's an ftp site, but I seem to have connection problems that way.
General :: Using Wget To Mirror A Website And Everything From The First Level Of External Sites
Sep 14, 2010I need to mirror a particular website (all the pages under that particular domain) any pages (but not whole sites) that the website links to.
How to do this
wget -r --level=inf (or some other variant) will mirror the site.
wget -r -H --level=1 will get all the links (from all domains) to the first level.
Anyone have any ideas on how I could combine these, to get the entire of the main site and one level deep into external sites. I've been banging my head against the manual all afternoon.
General :: Using Wget On A Site With Cgi?
Sep 6, 2011I need to mirror a website. However, each of the links on the site's webpage is actually a 'submit' to a cgi script that shows up the resulting page. AFAIK wget should fail on this since it needs static links.
View 1 Replies View RelatedUbuntu Installation :: 8.04.3 LTS Server - Cannot Find Any Mirror Site
Jan 27, 2010My requirement is to install Ubuntu SERVER 8.04.3 LTS on a Dell 2550 machine.
Problem.
- Bug in ubuntu (desktop and server) stopping CD install on this hardware platform. It hangs half way through and is something to do with the SCSI CD drive. Lots of stuff on the net about it, but no fix as far as I can see, at least nothing that works for me.
- So, tried boot off CD and then used F6 and added a cdrom-detect/try-usb=true line together with putting a USB with the ISO burnt onto it in the USB port. Still hangs during install (can't see USB perhaps? USB not correctly burnt? Tried many many times though, and eventually gave up at 2am)
- Next tried PXE boot. Now getting somewhere. Ok I installed tftpd32 on my windows machine, and copied the netboot files from the ubuntu site to the tftpd directory. (I dont think this will help me install the server, just the desktop??) Anyway, it booted from the win tftp machine (wahey!), and went through install, BUT when it tried to find the mirror site it couldnt. In fact it couldnt find ANY mirror site. (Internet access from windows and all other PCs on the network is working fine).
So, to ensure I am installing the SERVER version, and to use PXE boot, and to NOT use the internet to download the files (i.e. load them up locally somewhere), what do I do?
General :: How To Use 'wget' To Download Whole Web Site
Mar 14, 2011i use this code to download :wget -m -k -H URL... but if some file cant be download , it will retry Again and again ，so how to skip this file and download other files ,
View 1 Replies View RelatedSoftware :: Can't Replicate Site Using Wget
Jan 19, 2010I want to replicate this small howto (http://legos.sourceforge.net/HOWTO) using wget.However I just get a single file and not the other pages and that file too is not html.
View 4 Replies View RelatedDebian Installation :: Which Kernel On The Mirror Site Should Use
Feb 13, 2010I am just a newbie to debian/linux, and want to install debian lenny 5.0 with grub4dos. I have downloaded debian-5.03-CD-1-AMD64.iso and have read the installation-guide.pdf about debian 5.0 lenny. Then Go to "ftp2.de.debian.org/debian/dists/Debian5.0.4/main/installer-amd64" There are FOUR directories:
[Code]...
General :: Use Mirror Site In Ucsc Gnome Browser?
Jan 19, 2011what are mirror sites? how can we use mirror site in ucsc gnome browser?
View 3 Replies View RelatedUbuntu :: Download Entire Site With Wget From Localhost Proxy?
Dec 24, 2010i want to download android developer guide from google site but code.google is forbidden from my country i want to use wget to download entire android dev guides with freedom( proxy ) that i set in firefox these for open forbidden sites ( 127.0.0.1 port:8080 ) i use this command to download entire site
Code:
`wget -U "Mozilla/5.0 (X11; U; Linux i686; nl; rv:1.7.3) Gecko/20040916" -r -l 2 -A jpg,jpeg -nc --limit-rate=20K -w 4 --random-wait http://developer.android.com/guide/index.html http_proxy http://127.0.0.1:8080 -S -o AndroidDevGuide`
[Code]....
Debian :: Using Wget To Download Site For Offline Viewing
Nov 25, 2015This is the command line switch I am using:
Code: Select allwget -p -k -e robots=off -U 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.6) Gecko/20070802 SeaMonkey/1.1.4' -r www.website.com
For some reason it seems to be downloading too much and taking forever for a small website. It seems that it was following alot of the external links that page linked to.
But when I tried:
Code: Select allwget -E -H -k -K -p www.website.com
It downloaded too little. How much depth I should use with -r? I just want to download a bunch of recipes for offline viewing while staying at a Greek mountain village. Also I don't want to be a prick and keep experimenting on people's webpages.
Software :: FTP Via Kget - Username+pw Req'd For This Site But FTP Via Wget Doesn't
Sep 17, 2009I was trying to download MOPSLinux from their Russian FTP server, using Firefox-->FlashGot-->KDE-Kget and it kept sitting there for about a minute, then popping up a dialog box asking for a Username & Password to access the FTP site.
I tried the usual anonymous type of login information combinations, to no avail; the box kept reappearing.
Finally for the heck of it, I tried Firefox-->FlashGot-->Wget and presto! It began downloading right away, no questions asked.
This is on Slack64 with the stock KDE installation + the KDE3 compat libs.
Here's the transfer currently going on the Wget window:
Code:
Fedora Servers :: Make A Mirror Site - 15 I386 Rpm Files
Aug 22, 2011I want to make a fedora mirror site and i only want to have the fedora 15 i386 rpm files i dont want to download all the rest. How do i di this ?
View 2 Replies View RelatedGeneral :: Using Wget To Recursively Crawl A Site And Download Images?
Mar 29, 2011How do you instruct wget to recursively crawl a website and only download certain types of images? I tried using this to crawl a site and only download Jpeg images:
wget --no-parent --wait=10 --limit-rate=100K --recursive --accept=jpg,jpeg --no-directories http://somedomain/images/page1.html
However, even though page1.html contains hundreds of links to subpages, which themselves have direct links to images, wget reports things like "Removing subpage13.html since it should be rejected", and never downloads any images, since none are directly linked to from the starting page.I'm assuming this is because my --accept is being used to both direct the crawl and filter content to download, whereas I want it used only to direct the download of content. How can I make wget crawl all links, but only download files with certain extensions like *.jpeg?
EDIT: Also, some pages are dynamic, and are generated via a CGI script (e.g. img.cgi?fo9s0f989wefw90e). Even if I add cgi to my accept list (e.g. --accept=jpg,jpeg,html,cgi) these still always get rejected. Is there a way around this?
General :: Use Wget To Download A Site And ALL Of Its Requirement Documents Including Remote Ones
Aug 10, 2011I want to do something simular to the following:
wget -e robots=off --no-clobber --no-parent --page-requisites -r --convert-links --restrict-file-names=windows somedomain.com/s/8/7b_arbor_day_foundation_program.html
However, the page I'm downloading has remote content from a domain other than somedomain.com. It was asked of me to download that content too. is this possible with wget?
OpenSUSE :: Download Packages With Wget From That Mirror With A String "alsa"?
Nov 30, 2010Index of /distribution/openSUSE-stable/repo/oss/suse/i586 How can i download packages with wget from that mirror with a string "alsa"?
Such as:
alsa-1.0.23-2.12.i586.rpm
alsa-devel-1.0.23-2.12.i586.rpm
alsa-oss-1.0.17-29.2.i586.rpm
CentOS 5 :: Unable To Create LVM Mirrors - "Unable To Allocate Extents For Mirror(s)"; Free Space Is Available?
Dec 29, 2008I have fresh CentOS 5.2 installed to 2 sata hdd 1.5 TB size; on every of that hdd is primary raid1 submirror of /boot made with md during install. All other space is lvm place, on lvm living root and swap partitions. And it's work. Now I want to mirror my root and swap, and add other volumes with different raid requirements (raid0 and raid1).
# lvcreate -L10G -m1 -n mir vg0 Insufficient suitable allocatable extents for logical volume : 320 more required Unable to allocate extents for mirror(s).
Software :: Unable To Get The File In Http Server Through Wget Command?
Jun 24, 2011i am using fc9 server i installed Apache web-server i kept some datafile in my html folder when tried to download remotely through web i can download the file tried to get the file in remotely through wget command i am unable to get the fileor is failed: Connection timed out Retrying below the steps i tried it
my target file is http://X.X.X.X/test.zip
wget -T 0 http://X.X.X.X/test.zip
wget http://X.X.X.X/test.zip
[code]...
Ubuntu :: Unable To Enter The Site From Work
Apr 13, 2011I was interested in ubuntu, but was unable to enter the site from work, provider said that from his side everything is OK, problems should not be, may be banned my ip address I'm first time here it's not my fault if it is,
View 9 Replies View RelatedSoftware :: Resume An Interrupted Wget Using Wget.log?
Jun 19, 2011If a wget download is interrupted (like if I have to shutdown prematurely), I get a wget.log with the partial download. How can I later resume the download using the data in wget.log? I have searched high and low (including wget manual) and cannot find how to do this. Is it so obvious that I did not see it? The wget -c option with the wget.log as the argument to the -c option does not work. What I do do is open the wget.log and copy the URL and then paste it into the command line and do another wget. This works but the download is started from the beginning, which means nothing in the wget.log is used.
View 2 Replies View RelatedUbuntu Networking :: Access My Site Using Firefox Get Error - Unable To Connect
Mar 27, 2011When I'm trying to access my site using Firefox I get this error: "Unable to connect
Firefox can't establish a connection to the server at mysite.com.
-The site could be temporarily unavailable or too busy. Try again in a few moments.
-If you are unable to load any pages, check your computer's network connection.
-If your computer or network is protected by a firewall or proxy, make sure that Firefox is permitted to access the Web."
Then I tried to access my site using Opera and Chrome but I get the same error, that the browsers couldn't connect to it.
The site to which I'm trying to connect is definitely up. (I ask a few people and they can access it and I looked at [url] and it says that my site it's up)
Ubuntu Servers :: Openvpn Site-to-Site : Impossibile To Ping Client On Subnet?
Jun 25, 2010I have to ubuntu machine (9.10 and 10.4) with a openvpn tunnel between them.This is the situation:
Code:
NetworkA 192.168.0.0/24
|
UbuntuA br0:192.168.0.3 (openvpn bridge between eth0 and tap0)[code].....
UbuntuA has one only interface etho and there are two openvpn instance: one bridge istance with br0 and another instance with tun0.
UbuntuA is not the gateway for networkA. UbuntuB is the gateway for NetworkB.I need to comunicate between pc on networkB e those on networkA.This is the "ping situation" (no pc tested has an active firewall):
ubuntuA vs ubuntuB: OK
ubuntuB vs ubuntuA: OK
pc on NetworkA vs ubuntuA and ubuntuB: OK[code].....
Ubuntu :: Server Virtual Site - Permission Denied - Unable To Open Logs
Apr 20, 2010I was able to get the a2ensite command to enable a virtual site because it says "Site xxxxx.com already enabled" when I run the command. My problem now is that even though a site is enabled it says this: Code: /etc/init.d/apache2 restart * Restarting web server apache2
[Tue Apr 20 01:28:57 2010] [warn] NameVirtualHost *:80 has no VirtualHosts
[Tue Apr 20 01:28:57 2010] [warn] NameVirtualHost *:80 has no VirtualHosts
[Tue Apr 20 01:28:57 2010] [warn] NameVirtualHost *:80 has no VirtualHosts
[Tue Apr 20 01:28:57 2010] [warn] NameVirtualHost *:80 has no VirtualHosts
(13)Permission denied: make_sock: could not bind to address 0.0.0.0:80 no listening sockets available, shutting down Unable to open logs Should I do a chown or chmod to a file?
OpenSUSE Multimedia :: Unable To Play Apple Site Trailers?
Aug 10, 2009I am unable to play any trailer from Apple - Movie Trailers site. I went to location Apple - QuickTime - Troubleshooting to check if quicktime was installed correctly. I got the first quicktime logo which said that it's installed correctly but the animated logo (bottom right) showed the picture of a computer monitor instead of the animated quicktime logo. Which obviously means that i cannot stream video - my problem exactly.
I have a broadband connection and as an example when i go to the location Apple - iPhone - Gallery - TV Ads ; i have the website player trying to access the video but just keeps accessing it - never plays. I can play quicktime videos locally from my drive without any problems. I have struggling with this for the last 1 wk on every conceivable linux/Suse forum ; but without any success. I have mplayer , mplayerplug-in, libquicktime, libquicktime0, gnome-mplayer, gecko-media player already installed. Besides, i have totem, mplayer, mplayerplug-in, vlc-mozillaplugin installed.
How do i stream the trailers?
As a side note ; i found that totem is the one that tries to play my quicktime files; whereas totem is unable to play the .mov files locally for me. Vlc player works fine for .mov files. How do i make Vlc player stream my quicktime media may also solve my problem !
Networking :: Enabling Full Routing Site To Site With Openvpn Not Using Masquerading?
Jul 27, 2010I've been on a quest to enable full routing through my openvpn tunnel between my office and the colo. Masquerading will work, however it will throw off anything key based and makes a lot of things just more difficult and vague in general. Is there an easy way to do this via iptables? I tried using quagga hoping it would magically solve my problems, however it does not seem to do my routing for me . I just did a basic static route within zebra...
View 3 Replies View RelatedNetworking :: Site To Site Vpn Racoon With Cisco Asa 5505 Routing?
Mar 25, 2010I have three locations with a central office connected to two remote locations. At the central office I run on a cisco asa 5505 two site to site vpns. The remote end of the first site is a checkpoint firewall , and the remote end of the second site is racoon on debian. Both sites are up and working. However, where at the first site traffic goes both ways, at the second site it only works from the central office to the remote office.
For example, I can ssh from a host in the central office to a host in the first remote site (through checkpoint firewall,) then ssh back from that host at the remote office to any host in the central office. In contrast, after I ssh from a host in the central office to a host in the second remote office (through racoon), I cannot see the central office hosts (ping the ip address of a central office host, ssh, etc. all fail.) The vpn settings at the central office (the cisco asa 5505) are identical. So it seems to me that some routing magic is missing on the host running racoon at the second remote office. Where would such setting reside? racoon config files? iptables?
Security :: Site-to-site Ouija Board Connection?
Oct 3, 2010Maybe a site-to-site Ouija board connection.
View 5 Replies View RelatedSlackware :: Unable To Excludes.txt And Mirror-slackware-current.sh
Feb 1, 2011I use Alien Bob's mirror-slackware-current.sh with an excludes.txt to only download the parts of current that I want. Unfortunately it is not working for part of the exclude.txt file and I am at a loss to know why.
The excludes.txt is
Quote:
slackware64/e
slackware64/f
slackware64/kdei/kde-l10n-ar-4.5.5-noarch-3.txt
slackware64/kdei/kde-l10n-ar-4.5.5-noarch-3.txz
[code]...
This ignores e/ f/ y/ testing/ pasture/ source/ and usb-and-pxe-installers/ It also ignores most of kdei/ and /extra It is meant to ignore most of extra/source and just download the source for flash-player-plugin, but it is ignoring all of extra/source/, however the same syntax is working for kdei/ and extra/.
Ubuntu Servers :: Access Web Site When Site's Domain Name / User's Local Server's Domain Name Are Same
Feb 18, 2010In the office there is a local network with samba+openldap PDC. The local domain name is company.net. The company desided to create a corporate Website on a remote hosting and desided that the site's domain should be company.net which is same as local network's domain name. So now it is not possible to reach that corporate website from within the company's local network because, as I guess, bind9 which is installed on above menioned PDC looks for company.net on a local webserver. Is there a possibility to let people from this local network browse the remote site?
View 1 Replies View RelatedFedora :: Use Local Mirror For Yum - Set No Proxy For Local Mirror?
Jan 9, 2010I'm using f12. My university has recently become a mirror for fedora packages. But i'm facing a trivial problem. I have set proxy for yum so that the packages that are not available on local mirror can be downloaded from other mirrors. But then i don't know how to set no proxy for my local server. Consequently it is not using my local mirror at all. Tell me how to set no proxy for the local mirror. I want my local mirror because it's damn fast. My proxy settings are like this (they go in /etc/yum.conf):
[code]....
