I am using wget to grep some data from a .htaccess protected website.I don't want to use the --http-user= and --http-password= variables in the script so I tried to create a ~/.wgetrc file.Whenever I run my wget script, it will never use the http_user and http_password examples to login to th website.
View 2 Replies
when i wget aro2220.com it displays
--2011-03-02 16:35:58-- url... 127.0.1.1
Connecting to aro2220.com|127.0.1.1|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 177 [text/html]
Saving to: 'index.html'
100%
2011-03-02 16:35:58 (12.4 MB/s) - 'index.html' saved [177/177]
However, when I look into the file it is actually blank saying something like "It works! This is the default web page for this server" which can't be correct since that is not what Aro2220.com actually displays.
Furthermore, when I try to wget files I've put on the server for myself it returns a 404, file not found.
If a wget download is interrupted (like if I have to shutdown prematurely), I get a wget.log with the partial download. How can I later resume the download using the data in wget.log? I have searched high and low (including wget manual) and cannot find how to do this. Is it so obvious that I did not see it? The wget -c option with the wget.log as the argument to the -c option does not work. What I do do is open the wget.log and copy the URL and then paste it into the command line and do another wget. This works but the download is started from the beginning, which means nothing in the wget.log is used.
View 2 Replies View Relatedi download files from megaupload and hotfile. is there any possibility of making wget download more than 1 file at a time? or do you suggest any other download programme? i have ubuntu 9.10
View 3 Replies View RelatedI would like to use wget to downlaod file from Redhat linux to my windows desktop , I tried some parameter but still not work , can advise if wget can do download file from linux server to windows desktop ? if yes , can advise how to do it ?
View 14 Replies View RelatedI'm trying to download a file and extract it in one line, but the extracted file is owned by me instead of root even though I'm using sudo:
Code:
sudo sh -c 'wget [URL]'
If I don't try to extract the file, it is owned by root as I expected:
Code:
sudo sh -c 'wget [URL]'
is it recommended to download an iso file of fedora 13, will the file be destroyed?because i did it twice and it seems not working.
View 6 Replies View RelatedI was unable to find an answer to something that (I think) should be simple:
If you go here: [URL]
like so:
wget [URL]
you'll probably end up with a file called "download_script.php?src_id=9750"
But I want it to be called "molokai.vim", which is what would happen if I used a browser to download this file.
Question: what options do I need to specify for wget for the desired effect?
I'd also be ok with a curl command.
Example: [url]
This was what I tried...wget -A rar [-r [-l 1]] <mediafireurl>
That is to say, I tried with and without the recursive option. It ends up downloading an HTML page of a few KB in size, while what I want is in the range 90-100 MB and RAR.
What happens with MediaFire for those who may not be aware, is that it first says
Processing Download Request...
This text after a second or so turns into the download link and reads
Click here to start download..
How to write a proper script for this situation.
how can I define file type for wget to download . for example I do not want to download *.html or I just want to download *.jpg files . or if it does not support any of them do you know any other suggestion ?
View 1 Replies View RelatedI am calling a service using http post through wget, the command is successfully executing but for each execution its creating a file and saving variable names n data n it. I want to execute this command without creation of a file. Would anyone suggest me what needs to be done in this regard.
My command:
wget --post-data 'var1=99&var2=200' http://xyz.example.com:5555/invoke/Samples:httpInvoke
For every execution, its creating the files with names:
Samples:httpInvoke1
Samples:httpInvoke2
Samples:httpInvoke3
[Code]...
I use the
Code:
wget -r -A <extension> <site>
command to download all files from a certain site. this time i already have some of the files already downloaded and listed in a text file via
Code:
ls > <text file name>
How can i make wget to download from the site i want but ignore the filenames listed in the text file?
I need to use wget (or curl or aget etc) to download a file to two different download destinations by downloading it in two halves:
First: 0 to 490000 bytes of file
Second: 490001 to 1000000 bytes of file.
I will be downloading this to separate download destinations and will merge them back to speed up the download. The file is really large and my ISP is really slow, so I need to get help from friends to download this in parts (actually in multiple parts)
The question below is similar but not the same as my need: How to download parts of same file from different sources with curl/wget?
aget
aget seems to download in parts but I have no way of controlling precisely which part (either in percentage or in bytes) that I wish to download.
Extra Info
Just to be clear I do not wish to download from multiple locations, I want to download to multiple locations. I also do not want to download multiple files (it is just a single file). I want to download parts of the same file, and I want to specify the parts that I need to download.
I have set up a cron job in linux server using the command 'wget -q -o wget_outputlog url'
But on every run, an empty file being created at root.
How to stop this.
i am using fc9 server i installed Apache web-server i kept some datafile in my html folder when tried to download remotely through web i can download the file tried to get the file in remotely through wget command i am unable to get the fileor is failed: Connection timed out Retrying below the steps i tried it
my target file is http://X.X.X.X/test.zip
wget -T 0 http://X.X.X.X/test.zip
wget http://X.X.X.X/test.zip
[code]...
We have 2 servers, 1 is the webserver and the other is the Mysql server.
When transfering a 2GB file from the webserver to the Mysql server.
The webserver's connection to the mysql DB server dies completely.
Need to restart the MYSQL process in order for it to come back online.
During this connection downtime, when using phpmyadmin on the mysql server shows no problem running queries etc.
I am trying to copy a directory and everything in it from one server to another.No matter what I type into SCP, it just gives me back:
usage: scp [-1246BCpqrv] [-c cipher] [-F ssh_config] [-i identity_file]
[-l limit] [-o ssh_option] [-P port] [-S program]
I tried:scp -r -P 1133 root@XX.XX.XX.XX:/home/imagesShouldn't that recursively copy /home/images from the server XX.XX.XX.XX through SSH on port 1133?Btw - I know you can do it with a tar or just a regular FTP program. The folderI am trying to copy is 40 gig, there isn't enough free space to make a tar (if the server would even do it)
I'm typing this on my linux laptop, at work. My Firefox works fine, but I cannot apt-get, or wget anything. To get my Firefox to work, I just went into the Firefox preferences, checked "Automatic proxy configuration URL" and entered the url that I have. Now Firefox works fine, but the rest of my system does not.o be a similar setting in System>Preferences>Network Proxy. There is check box for "Automatic proxy configuration" and a field for a "Autoconfiguration URL". I put the same URL that put into Firefox here and told it to apply it system-wide, but my apt still does not work. This is a big deal because I need to install software and I really don't want to start manually downloading packages, plus I need ssh.
I have googled extensively on how to get apt to work from behind a proxy, but nothing seems to be working. I don't have a specific proxy server and port; rather I have some kind of autoconfiguration URL. Plus, my system has no /etc/apt.conf file at all. Any ideas on how I can get my system to be able to access the internet? It's very strange to me that Firefox can, but apt, ping, wget, etc cannot.
I am trying to have this cool effect where gnome scheduler downloads with wget this image every three hours. However, even when I do it manually in the terminal it doesn't seem to download it correctly. When I go to open the .jpg it says in a big red bar on the top "Could not load image '1600.jpg'. Error interpreting JPEG image file (Not a JPEG file: starts with 0x47 0x49)"
However, when I go to the picture in the link above and right click "Save Image As" it downloads it fine.
I'm currently using wget to keep a running mirror of another site but I don't have much space locally. I was wondering if there was a way to turn on -N (timestamping) so that only the "updates" were retrieved (i.e. new/modified files) without hosting a local mirror.
Does -N take a timestamp parameter that will pull any new/modified files after "x"?
It seems like a waste to compare remote file headers against a timestamp without presenting the option of supplying that timestamp. Supplying a timestamp would allow me to not keep a local mirror and still pull updates that occurred after the desired timestamp.
Like the subject says,.. I'm lookin for a wget'able 11.04 Live CD URL This URL works great with a point and click but doesn't tell me what the direct URL is to use wget. [URL]
View 1 Replies View RelatedI'm trying to download a set of files with wget, and I only want the files and paths "downwards" from a URL, that is, no other files or paths. Here is the comand I have been using
Code:
wget -r -np --directory-prefix=Publisher http://xuups.googlecode.com/svn/trunk/modules/publisher There is a local path called 'Publisher'. The wget works okay, downloads all the files I need into the /Publisher path, and then it starts loading files from other paths. If you see [URL]..svn/trunk/modules/publisher , I only want those files, plus the paths and files beneath that URL.
I'm trying to use wget on an ubuntu 10.04 server 64bit, with 16GB RAM and 1.1 TB free disk space. It exits with the message "wget: memory exhausted". I'm trying to download 1MB of some sites. After different tries this is the command I'm using:
Code:
wget -r -x -Q1m -R "jpg,gif,jpeg,png" -U Mozilla http://www.onesite.com
(I only need the html documents, but when if I run the -A options only the first page is donwloaded, so I change to -R).
This happens with wget 1.12 version. I've tried the same command in other computers with less RAM and disk space (ubuntu 8.04 - wget 1.10.2) and it works just fine.
Would it be possible to use wget to order something on e.g. [URL].
View 4 Replies View RelatedI'm trying to parse some redfin pages and it seems like I'm having a problem with the # symbol.Running the following:echo 'http://www.redfin.com/homes-for-sale#!search_location=issaquah,wa&max_price=275000 ' > /tmp/issaquah_main.txtwget --level=1 -convert-links --page-requisites -o issaquah/main -i /tmp/issaquah_main.txt
View 3 Replies View Relatedi was trying to copy some files over my hdd using wget.this was the format of the command the catch is that there is a local website that is installed into directory heirarchy and i would like to use wget to make the html files link to each other in one directory level.the command didn't work inspite of trying different forms, so what's the mistake in this command or is there another way?
View 3 Replies View RelatedI have executed the command
Code:
sudo wget -r -Nc -mk [URL]
(referring the site : [URL]) for downloading entire website using wget for offline viewing on Linux,
At the middle of download I have shutdown my laptop (downloading was not over) and when I started my laptop again. I have executed the same command to continue downloading but I have got the error :
Code:
test@test-laptop:/data/Applications/sites/googlejam$ sudo wget -r -Nc -mk [URL]
--2010-07-25 19:41:32-- [URL]
Resolving code.google.com... 209.85.227.100, 209.85.227.101, 209.85.227.102, ...
Connecting to code.google.com|209.85.227.100|:80... connected.
HTTP request sent, awaiting response... 405 Method Not Allowed
2010-07-25 19:41:33 ERROR 405: Method Not Allowed.
Converted 0 files in 0 seconds.
test@test-laptop:/data/Applications/sites/googlejam$
I used the crontab to start wget and download the file with the following
Quote:
14 02 * * * wget -c --directory-prefix=/home/Downloads/wget --input-filefile=/home/Downloads/wget/download.txt
But it doesn't shows a terminal and so not able to get the current status and stop wget. So how can I start wget with a terminal using crontab?
I am trying to wget a site so that I can read stuff offline.I have tried
Code:
wget -m sitename
wget -r -np -l1 sitename
[code]....
i don't know if there is an apt-get for it or if i need to use wget
View 1 Replies View Related
