Programming :: Grep Multiple Values In Single Pass Through Log File?

Nov 13, 2010

I have a huge binary log file. There are lets say 4 id's that I want to find in a log file. I know that those 4 id's will be present in the log file and I also know in what order they will be present. I want to find 1st id from the log then 2nd id and then third id and so on..

Simple/inefficient solution is: Loop through the id's and then grep in the log file. Problem with this solution is for each id grep will search from the beginning of the file.

Better/efficient solution would be: Sine I know the order in which id's will be present in the log file. Loop through id's, grep 1st id and then move on to grep 2nd id and so on...this way I can grep all id's in one pass. Is this solution possible ?

I have 500000 + values to find in log files and I have to find efficient solution for it.

View 2 Replies


General :: Pass Two / Three Values To Grep?

Sep 22, 2010

Is there any way i can pass two / three values to grep. Basically i want to list the lines in the file with three different values. Currently I'm using three grep to get my work done.

EG: more Filename | grep text1 | grep text2 | grep text3

View 2 Replies View Related

General :: Multiple Grep Outputs Appended To Single Row Of CSV File?

Sep 12, 2010

how to update a series of values from multiple grep commands outputs to be appended to a single row of a csv file? Work on a linux envir. The values from grep output will be numeric values.

Output sold look like:


Each of these values will be odtained from multiple grep commands piped with wc -l Is it possible to update a single row of a csv file if so pleas ehelp me with the command to be used to redirect the output into the csv file

View 5 Replies View Related

Programming :: Variable String File Base - Output File Doest Not Exclude The Values In Grep -av

May 1, 2011

My script.

This is may script:


Problem: Output file doest not exclude the values in grep -av

View 3 Replies View Related

Programming :: Udev Rules - Pass ATTRS{*} Values To The RUN Command?

Sep 24, 2010

I'm modifying a working udev rule which runs a script that mounts a USB HDD and synchronises files to it.The USB HDDs have been troublesome, losing many files and even losing file systems a couple of timesTo investigate,I want to log the USB HDD product name and serial number so would like to pass ATTRS{product} and ATTRS{serial} values to the script.This may not be possible; I cannot see anything about how to do it in either the udev man page nor Daniel Drake's "Writing udev rules" Version 0.74 but it seems such an obvious thing to want to do, I'm wondering if I've overlooked something.

View 8 Replies View Related

General :: Grep All Values Other Than Encrypted Password From /etc/shadow File?

Jan 14, 2010

I would like to grep all values other than encrypted password from /etc/shadow fileFor example,each line consists of 8 fields separated with :/The only thing that I want not to print out is the contents between first : and second : (encrypted password)

View 7 Replies View Related

Programming :: Connect Multiple Sockets From A Single Client To A Single Server And Keep Them Open?

Oct 13, 2010

I have question about the UNIX sockets. my goal is to connect multiple sockets from a single client to a single server and keep them open...I'm not sure if that is possible to create or not. Do you have any suggestion or an example of code?

View 1 Replies View Related

Programming :: Grep, Cut, Cat In A Single Command?

Apr 26, 2010

Would like to know how to cat the below output.

grep userlist_file /etc/vsftpd/vsftpd.conf | cut -c15-
Below which is the output for above command.


View 6 Replies View Related

Programming :: Parsing And Storing Multiple Values Using 'awk'?

Jan 24, 2011

I have a variable in which the data is stored as below:

variable_test=0m0.001s 0m0.001s 0m0.001s 0m0.001s 0m0.001s 0m0.001s so on.

There are lots of values in format like "3m1.057s" are stored in variable_test separated with an space between two such values. For exapmple, value is "3m1.057s" I need to save different parts of a value in three separate array variables such as the

var_sec= 057

tell if this can be done using "awk". A "WHILE" loop might be used to separate and store theses values I guess?

View 1 Replies View Related

Programming :: MySQL And PHP - Table Column With Multiple Values?

Dec 1, 2008

I am writing a script to mine data from my game server logs which are going to mysql. My first question is syslog-ng is sticking $msg into one colum it self, $msg tends to be a long string here are a few examples:

cn name team frag death flags role host.
0 Maik CLA 2 0 0 normal *.70.233.118.
1 lizardncd RVSF -1 3 0 normal
[*.22.247.40] lizardncd fragged Maik.

And they are not always in the same format, How could I parse this data to do something useful, like creating a user stats page with frag counts and flag counts. Should I create a script that runs in a cronjob that parses this data to a new table maybe once a day. Even then the $msg string value is random and I am thinking in terms of awk.

Second. I would like to keep track of what ip's are using what nicknames. Kinda in the sense of and ip associated to a array of names that have been used on that ip address so I can keep track of users. How would this work logically with a mysql db, could a column allow multiple values, then loadable into an array, which then can add, remove, and store to mysql.

View 1 Replies View Related

Programming :: Input File - Grep Within Grep

Nov 22, 2010

I need to kind of grep within grep. My input file would be something like:


and I need to find the first occurrence of hello before MY PATTERN (hello 9008 in this case), so the output should be:


View 4 Replies View Related

Programming :: C: Testing Multiple Consecutive Array Values Concurrently.?

Nov 22, 2010

Basically, I am provided with a file "temp.dat" with 30 high temperatures (integers) in it. The program is supposed to read them in and compute/print the average. Then it is supposed to print the temperature of each day and, in addition, display a + by each day that is over the average, but only if it is above the average high for three or more consecutive days. This is the part I am stuck on. I'd appreciate any tips that would point me in the right directionFull disclosure: This is a school project. Code:

#include <stdio.h>
#include <stdlib.h>


View 2 Replies View Related

Programming :: Script To Pass Multiple Files To Program?

Feb 8, 2011

I'm playing around with a bash script to pass files to a program (such as VLC). I thought something like this would work:

vlc $( ls | sed 's/ /\ /g' | tr '
' ' ' )

ls shows the files in the current directory, then sed changes spaces to " " (to escape them) and finally tr removes the line breaks. I end up changing:

01 - Music Track.mp3
02 - Another Track.mp3

01 - Music Track.mp3 02 - Another Track.mp3

"vlc 01 - Music Track.mp3 02 - Another Track.mp3" works if I type it manually but my script reports a problem with `-'

It seems it's not escaping the spaces.

View 4 Replies View Related

Programming :: Grep Multiple Files ?

Dec 30, 2010

I want grep multiple files:


I use grep <pattern> abc.* to read the files. But here grep read files in the order of 1,2,3,4 . I want to to grep the files in reverse order like 1st the abc.4 file will be read and abc.1 file will be last.

Is this possible ?

View 3 Replies View Related

General :: Shell Script File - Pass Multiple Parameters To Interpreter?

Feb 2, 2011

I have a Python script that I run which needs to execute under a special environment, so I would run the program like so from my working directory (~/project/src):

python shell

This opens up an interactive shell for me to start typing my own commands.I have another set of administrative activities that I would like to house in another directory (~/project/admin). The is really finicky about running from the working directory. So, to make this whole thing work, I made a script which starts off like so:

#!/usr/bin/python ../src/ shell

There are a couple problems with this. The first is that it doesn't work:

/usr/bin/python: can't open file '"/../src/" shell': [Errno 2] No such file or directory

How do you specify multiple parameters to the interpreter?How do I change the working directory?

View 2 Replies View Related

General :: Grep Multiple Lines From A Text File

Jun 17, 2009

I have a list of words that I want to grep in many files to see which ones have it and which ones dont. in the text file I have all the words listed line by line, ex: list.txt:

try this
open space
list ..

I want to grep each line one by one. like I want it to

grep "check" *.log
grep "try this" *.log
grep "word1" *.log .. etc how can I do this?

and maybe write the output to a file.

View 5 Replies View Related

Programming :: Splitting A File Based On The Values Read From An Input File?

Jul 27, 2010

I am splitting a file based on the values read from an input file. The below one is the script.

1)How do I add the header which is present in the original file to the new split files created?(For eg. pharmacyf conatins header as table column names. The new files created (ODS.POS.$pharmacyid.$tablename.$CURRENT_DATE.dat) are without the header).

2) Also the script is creating 0 byte files for the pharmacyids which are not available in the intial file? Can this be avoided?

for pharmacyf in *
tablename=`echo $pharmacyf |cut -f4 -d'.' `
while read pharmacyid
grep -w $pharmacyid $pharmacyf >> $OUT/ODS.POS.$pharmacyid.$tablename.$CURRENT_DATE.dat
done< inputfile

View 2 Replies View Related

Server :: Multiple File Pointers In A Single File?

Apr 20, 2011

I have files a, b, c and d. They're all relatively large and are served up by a static web server optimized for this purpose. I can get requests that look like this:


Each request is basically a request for a concatenation of the files in the order of the letters. The list of possible requests is finite, but large enough that disk space will run out very quickly and be very expensive if I create all possible files via concatenation.Is there a way to create a pointer file like abcd that is essentially a multi-file symlink that first points to a then to be then to c then to d? So if the contents of the files were as follows:

a: hello
b: there
c: whats


View 3 Replies View Related

General :: Single Server And Multiple Client Programming In C ?

Jul 1, 2010

I am new in client server programming and i have written code for single server to single client communicating one port(3490). but i have no idea how my single server will be communicate with different client on different this will be happen ?

Any idea to understand the logic or send any good link or any piece of code in c , i searched on net but all help was mostly for java progamming.

View 1 Replies View Related

Programming :: Catching Multiple Signals In A Single Handler In C?

Apr 26, 2011

Is it possible to handle multiple signals in just one handler? I only know singal() catches one signal at a time

View 2 Replies View Related

Programming :: Sorting Specific Values From An XML File

Sep 21, 2010

I need a shell script which gathers the data from a remote XML file and then displays it according to my needs.. I need this for my job due to the fact that I need to keep track price changes of euro, usd, gold, etc. The XML file I am talking about is located at this page: here. The reason I am posting the URL is that I need to use curl to get this file and it does NOT have newlines after each tag. I thought that that would be a problem. Here is what I need from the script: 1) curl to get the page 2) make use of sed, awk, etc. to display its contents in a more structured and readable manner as shown below:

A: 64.125 (% -0.26)
B: 81.130 (% -0.32)
C: 1.4930 (% 0.00)
D: 1.9590 (% 0.36)

View 5 Replies View Related

Fedora :: Join Multiple Mp3s Into A Large Single File?

Apr 15, 2011

is there an app, runs in Fedora, with a GUI, that can join multiple mp3s into a large single file?

I know Audacity can do this, but it's not really suitable for multiple files and I have voice notes that are in 10 minute chunks.

View 2 Replies View Related

Programming :: Multiple Input Into Seq Command From Pipe - Single Column Unique Numbers

May 13, 2011

Have this script which is reading in a series of files, one at a time with while-do-done loop, each file goes through various greps/awk's where this info is then saved to various files for later use. i.e....

Script is being run on Linux Red Hat,

In one of the grep/awk's the output (currently) are 2 columns (min max), i.e....| awk '{print $1, $2}' | sort -u which outputs (e.g.)

The number of "min max" pairs varies from file to file. Want to output a single column of unique numbers from the min max pairs & get the number of them for input to a file...i.e...

Where <PROCESS> is some process/technique that will generate a single column of integers (increment of 1) to pipe into the next one (sort -u)

i.e. (example from above)

Have tried command seq - only works for single pair input i.e.

Is there any command like seq etc which will output a single column based on a input of min max numbers (increment 1) to pipe onwards to next command?

View 4 Replies View Related

General :: Xfce: Save Page As: Single File Vs Multiple Files?

Nov 28, 2010

I am using Xfce as the desktop enviroment and Mozilla Firefox as the webrowser. Within the webrowser window, I do File>Save Page As. I save it, and the result is almost always foo.html and directory foo_files. But I think under KDE I could choose the format, one of them being something like "Single page" (only one file; the colecction of .png, etc is embedded into that file). And this is the format I want Xfce (or Firefox) to use when downloading to hdd.

View 1 Replies View Related

Programming :: Updating A Text File (replacing The Existing Parameters With The New Values)

Nov 9, 2010

I have a text file that needs to be updated to be used by some fortran code later. it looks like that:

1,1 !Ms for y-stations, Ns for x-stations
0,0,0, !xsta, ysta, zsta

I need to replace the "0,0,0" with new values for xsta,ysta and zsta;

View 4 Replies View Related

Programming :: Possible To Pass File D Escriptor Without Sockets / Streams?

Apr 20, 2010

I have a problem passing a file descriptor from one process to another.

I have two processes A and B. Both are running in different network and
filesystem namespaces, so it is impossible to use unix domain sockets or
net sockets to pass a file descriptor from process A to process B.

The usage of STREAMS is also impossible, as you can see in

[quotation begin]
Linux doesn't have STREAMS, which are the System V way
of doing this task. ...
[quotation end]

Are there additional possibilities for file descriptor passing like
using named pipes or something like that or does anybody know
a good workaround for this problem ?

View 9 Replies View Related

Programming :: Grep The Patterns From The File?

Aug 10, 2010

I am interested in using the grep method in the shell of my CentOS machine to obtain patterns from a file and use them to search through another file and highlight the patterns found. For example:

pattern file:


test file:


View 8 Replies View Related

Programming :: Octave - Load A Text File With Floating Point Values Into A Matrix?

Jan 29, 2011

I have a .txt file with the format


I am wanting to load this into octave and perform operations on the data.


View 2 Replies View Related

Programming :: Grep A Particular Data From A File Without Using The Command

May 3, 2010

I want to know that is there any method to grep a particular data from a file without using the "cat --- | grep ' ' " command....I need to use a system call for this functionality.

View 1 Replies View Related

Programming :: Find A String In A File Without Using Grep?

Oct 27, 2010

I want to write some code to search for a specific string in a text file, but without using grep command.

View 5 Replies View Related

Copyrights 2005-15, All rights reserved