I understand that the linux pipe is a buffer and that any data written to it will stay there until it is read, and if the max capacity of the buffer is reached, any additional writes will block (by default).
HOWEVER, the behavior of the pipeline below suggest that the write operations are buffered/cached before ever being written to the pipe on the client side here is write.sh, which creates 1000 byte string and writes it 100 times to stdout... the idea being that it'll block as soon as the 64kb linux pipe size is reached:
[Code]...
This is not what I was expecting: I was expected that once the capacity was reached, any reads would be followed immediately by a write to take advantage of the freed space. Instead, the blocked write operation seems to wait for some random amount of time/space to free until it unblocks and writes.
I've written a simple server in linux used fork to create a FIFO pipe.The server create two FIFO pipe.One for server read data from client and write data to client.Then another pipe for client read data from server and write data to server.When the server read data from a client used server-pipe and then write data to client.But ,if the client no read open the pipe,the server side write will be crashed because of a broken-pipe SIGPIPE. How to check whether the read side is opened?Or,how to catch the SIGPIPE,and then my server will still execute on,not crashed!!
I keep time sheet entries at work in an sqlite database called 'timesheet'. I have a shell script called 'today' which queries for all timesheet entries which are less than 24 hours old; it looks like this:
how to make removable media (e.g. USB sticks) not have any write caching. I want to prevent data loss when they are removed after file copying appears done but before write caches are written. I'm using Gnome on Squeeze.
I've found suggestions of adding the 'sync' mount option to /system/storage/default_options/vfat/mount_options in the Gnome configuration. However this doesn't seem to completely eliminate write buffering, as the drive activity light continues for several seconds after file copying appears done, and unmounting drives produces a dialog box which says to wait whilst data is written to disk.
I have a program that is very heavily hitting the file system, reading and writing randomly to a set of working files. The files total several gigabytes in size, but I can spare the RAM to keep them all mostly in memory. The machines this program runs on are typically Ubuntu Linux boxes.
Is there a way to configure the file system to have a very very large cache, and even to cache writes so they hit the disk later? I understand the issues with power loss or such, and am prepared to accept that. Crashing aside, in normal operation the writes should eventually reach the disk!Or is there a way to create a RAM disk that writes-through to real disk?
I connect to our server by ssh, default ssh client on Debian system.when I run a command that has large output on stdout (on shell) my connection is lost and I get error: "Write failed: Broken pipe"Unfortunate on our server there is limitation for connection per hour!
I move a page from a server in SuSe to Debian and this script was working well but when I move the page to a Debian base server it is sending in the logs the following message:
cat: write error: Broken pipe, referer: [URL]
The cgi does the following, read a db file plain text that repeat each 15 lines an information where to look the seraching info. if it is there sends to the output the 15 lines block. but it was not sending errors in suse distro, does anybody knows why the cat's line are sending "cat: write error: Broken pipe," if I am not writing or making a pipe with ">" to none file.
I am having a truly bizarre problem when trying to set up my sheeva plug as a backup server. I have four users (one for each member of my family) that I want to give permission to ssh in (well, sftp in but they're essentially the same) and backup to an external harddrive.
My user works perfectly well and has done since I got the plug. So does a transmission user I set up to run the transmission daemon. With the other three users, however, I get the following error after SSH asks for the user's password:
Code:
Write failed: Broken pipe
This happens whether I try locally or over my network. I haven't been able to find a solution to it at all. I've made sure that /bin/bash was set as the shell and that the home directory is writeable with the correct permissions (ls -al /home):
Code:
drwxr-xr-x 7 root root 4096 Dec 21 19:25 . drwxr-xr-x 21 root root 4096 Dec 17 19:27 ..
[code]...
The alternative for me would be to use netatalk and AFP but the problem is that there is no native support for AFP and so things like d�ja dup backup may not be able to mount the afp drive...
Whenever I put ssh into remote machine am getting the following messages. Write Failed: Broken Pipe (After logged into that machine ) Read Socket Failed: Connection reset by peer (While trying to log in ) Also known_hosts file is changed frequently.
which cropped up today after no changes to the system: I can successfully ssh into my ubuntu 64-bit 9.10 machine (via OS X with iTerm), but after about 30 seconds, the connection drops with the message "Connection reset by peer Connection to xx.xx.xx.xx closed".
If I then try to immediately ssh back in I get: "Write failed: Broken pipe". If I try to immediately ping the machine instead, I cannot ping it until the ~10th try. Eventually I can log back in again, only to be kicked off after between 5 and 60 seconds. I called the network people and their are no known issue right now with networking. The networking on the ubuntu machine is otherwise fine, it seems. In the /var/log/auth.log file, it simple says:
Quote: Feb 9 14:19:20 harriet sshd[4134]: Accepted password for xxx from xx.xx.xx.xx port 55105 ssh2 Feb 9 14:19:20 harriet sshd[4134]: pam_unix(sshd:session): session opened for user xxx by (uid=0) Feb 9 14:19:28 harriet sshd[4046]: pam_unix(sshd:session): session closed for user xxx
The solution I come across to a similar problem is to add a "ClientAwakeInterval" number to sshdconfig, but this (as expected) doesn't fix my problem.
I understand that $! is the PID of a command. For example:
Code: #!/bin/bash myprogram & echo "PID of myprogram is $!"
I'd like to send the output of "myprogram" to both console and to a log file using the "tee" command but I also want to store the PID of "myprogam". Something like this:
Code: #!/bin/bash myprogram | tee ./logfile & echo "PID of myprogram is $!"
The problem is that $! is now the PID of "tee" rather than the PID of "myprogram".
I am trying to automate some directory naming when we're manually running some scripts and are using tee to direct the output to a file (log). Right now this is what we do
Code:
./some_script.sh 2>&1 | tee /home/user/some_dir/logs/manual/some_script_20110216_1628.log
As a matter of laziness and keeping the log files consistently named, I'd like to create a function to pipe it to so that it's doing all the naming How I envision the command running
Code:
./some_script.sh 2>&1 | myfunc
And what the logfile name should look like (and in the right directory)
Code:
some_script_20110216-1628.log
I was thinking of adding a function to our profile to handle this. Just in testing I was trying to stream line right on the command line, but I'm having some difficulty in getting the name of the script that is pushing data over the pipe. Here is what I've tried
Code:
./some_script.sh 2>&1 | tee $(cd ../logs/manual; pwd)/$0_$(date +%Y%m%d)-$(date +%H%M).log
I want to pipe the output of ls in a folder to a file (lets call it test.txt) but when i do so, but when i do ls > test.txt in test.txt there is also test.txt (logical
I'm doing ping between 2 RH servers through a VPN site2site tunnel and in some times I got in the result pipe 2 and another pipe 3 as I mark it in blue color below.
e.g.
64 bytes from 192.168.1.10: icmp_seq=0 ttl=128 time=0.229 ms 64 bytes from 192.168.1.10: icmp_seq=1 ttl=128 time=0.287 ms 64 bytes from 192.168.1.10: icmp_seq=2 ttl=128 time=0.278 ms
[code]....
What's the difference between pipe 2 and pipe 3 and what's the meaning of it?
I want to have the output of a program go to 2 different files but not going to standard out. Is there a way to do this in bash? I know that in Z shell its really easy. omething like: Code: echo "test" >> file1 >> file2 Would work. But in Bash it doesn't seem that easy. I know that tee will send the output to 2 files but it also sends it to STDOUT.Something like:Code: echo "test" | tee -a file1 file2 Would put the word "test" in file1, file2, and STDOUT. Is there a way to just send the output to file1 and file2?
I have a small program that reads stdin from a pipe using fgets. Now fgets blocks for the first line but after that it will not block.
The code, my_echo.c - int main(int argc, char **argv) { char buf [2000] ; char* pc ; printf("hello ") ; while (1) { buf[0] = (char) 0 ; pc = fgets(buf, sizeof(buf), stdin); if (pc != NULL) printf("%s ",buf); } return 0; }
How its called * In terminal window 1: ./my_echo < my_fifo * In terminal window 2: echo "1234" > my_fifo * In terminal window 1: prints hello then 1234. * Checking with ksysguard or top shows that my_echo is consuming 40% of CPU time.
Adding a few printf's shows that the gets is not blocking and returns a null pointer. * In terminal window 2: echo "qwerty" > my_fifo * In terminal window 1 qwerty prints. I want a read function that does in fact block so my program does not tie up CPU time, read does not block.
I'm looking for a way to detect whether or not a program has been called from pipe, e.g.
Code:
whatever | my_program versus simply just being exectuated directly:
Code:
my_program
Why? In the first case, I want to run the program non-interactively, and in the latter case I want to print out user-friendly messages. I've been thinking along the lines of some check I haven't yet found, like:
If a process forks its child and communicate with the child using pipe, do closing the write end of the pipe and terminating the writing process have the same effect?
how to pipe the current directory listing into sort so that the output is the date in descending order (primary sort key). If there are multiple entries with the same date, I'd like the times sorted in ascending order. It seems simple but for some reason this isn't working:
ls -l | sort -k 6r -k 7
For some reason it doesn't seem to ever get to the second sort key when using column 6 (last modified date).
I am building a python script which works same as SMTP protocol. I have build separate functions in that for each command of SMTP, and after this i have integrated all those functions in a new function named as send_mail(so that i don't have to execute every function separately for every command). Now, when i execute the script for the first time it runs successfully, but for the second time it gives the error of "BROKEN PIPE". I really can't make out how the socket is getting closed.
I've written a usb device driver and a program that sends and receives data over the bulk pipe. The read function sometimes returnsI'm reading an unknown amount of data. However, using a usb tracker I can see that the correct data is being sent.The error only occurs sometimes.I expect that the read function is told to read more data that it receives it would fail and return -1, however if this was the case then every read call would fail.
My question deals with me creating a name pipe (file) in the my /group directory called chat.I then have to write a script to read from the named pipe and save data into a file called chat.log until the words End of File are passed to the program.
-When I created the named pipe file (chat) I used the mknod chat p command..Is this the correct command to create a named pipe file? -Then I'm having trouble with my script and how to make it run until the words End of File are entered in. This is what I have so far.
I am not able to execute a multiple sed statement using pipe filters in a variable. i am trying to extract a path from a file and then working on that path to change a few letters within the path by going through another sed statement.
I wrote y bash script that opens YouTube playlist using youtube-dl and VLC applications: the output of youtube-dl is the input of VLC. The only problem is VLC needs to be closed after each playlist item unless I get the error message of the operating system:
"ERROR: unable to write data: [Errno 32] Broken pipe".
I understand the reason but I don't know how to resolve it: it is possible to close previous STDIN of VLC without killing the entire VLC process so that a new youtube-dl instance can connect to the same VLC instance? In short this is my question. The problem is detailed here: Downloading and playing videos and subtitles.
Have this script which is reading in a series of files, one at a time with while-do-done loop, each file goes through various greps/awk's where this info is then saved to various files for later use. i.e....
Script is being run on Linux Red Hat,
In one of the grep/awk's the output (currently) are 2 columns (min max), i.e....| awk '{print $1, $2}' | sort -u which outputs (e.g.)
The number of "min max" pairs varies from file to file. Want to output a single column of unique numbers from the min max pairs & get the number of them for input to a file...i.e...
Where <PROCESS> is some process/technique that will generate a single column of integers (increment of 1) to pipe into the next one (sort -u)
i.e. (example from above)
Have tried command seq - only works for single pair input i.e.
Is there any command like seq etc which will output a single column based on a input of min max numbers (increment 1) to pipe onwards to next command?
i wanted to capture the stdout and stdin of a child process within a parent so that any output of child is sent to the parent and any input taken from the parent. code is simple enough and i have followed all code guidlines on the internet (some guidlines do differ also) my select call either hangs if i do not give a tmeout and with a timeout it returns 0 descriptors to be written to or read from:
below is the simple code for parent:
int main(void) { int outfd[2]; int infd[2];
[code].....
why select hangs without a timeout ... why can it not detect that the pipe is write ready i.e parent can write to it ... if it does not detect tha read pipe as having data...
When I was running Windows OS on my laptop I was able to buffer and pause movies I watch on Dvix. Now that I'm running Ubuntu, I cannot pause the movie anymore. Whenever I pause it, I have to watch the movie from the beginning. Also, I cannot let the movie buffer to completion.
