Programming :: Grep Multiple Values In Single Pass Through Log File?
Nov 13, 2010
I have a huge binary log file. There are lets say 4 id's that I want to find in a log file. I know that those 4 id's will be present in the log file and I also know in what order they will be present. I want to find 1st id from the log then 2nd id and then third id and so on..
Simple/inefficient solution is: Loop through the id's and then grep in the log file. Problem with this solution is for each id grep will search from the beginning of the file.
Better/efficient solution would be: Sine I know the order in which id's will be present in the log file. Loop through id's, grep 1st id and then move on to grep 2nd id and so on...this way I can grep all id's in one pass. Is this solution possible ?
I have 500000 + values to find in log files and I have to find efficient solution for it.
View 2 Replies
ADVERTISEMENT
Sep 22, 2010
Is there any way i can pass two / three values to grep. Basically i want to list the lines in the file with three different values. Currently I'm using three grep to get my work done.
EG: more Filename | grep text1 | grep text2 | grep text3
BRs
View 2 Replies
View Related
Sep 12, 2010
how to update a series of values from multiple grep commands outputs to be appended to a single row of a csv file? Work on a linux envir. The values from grep output will be numeric values.
Output sold look like:
1,3,4,5,7,0,5
Each of these values will be odtained from multiple grep commands piped with wc -l Is it possible to update a single row of a csv file if so pleas ehelp me with the command to be used to redirect the output into the csv file
View 5 Replies
View Related
May 1, 2011
My script.
This is may script:
Code:
Problem: Output file doest not exclude the values in grep -av
View 3 Replies
View Related
Sep 24, 2010
I'm modifying a working udev rule which runs a script that mounts a USB HDD and synchronises files to it.The USB HDDs have been troublesome, losing many files and even losing file systems a couple of timesTo investigate,I want to log the USB HDD product name and serial number so would like to pass ATTRS{product} and ATTRS{serial} values to the script.This may not be possible; I cannot see anything about how to do it in either the udev man page nor Daniel Drake's "Writing udev rules" Version 0.74 but it seems such an obvious thing to want to do, I'm wondering if I've overlooked something.
View 8 Replies
View Related
Jan 14, 2010
I would like to grep all values other than encrypted password from /etc/shadow fileFor example,each line consists of 8 fields separated with :/The only thing that I want not to print out is the contents between first : and second : (encrypted password)
View 7 Replies
View Related
Oct 13, 2010
I have question about the UNIX sockets. my goal is to connect multiple sockets from a single client to a single server and keep them open...I'm not sure if that is possible to create or not. Do you have any suggestion or an example of code?
View 1 Replies
View Related
Apr 26, 2010
Would like to know how to cat the below output.
Code:
grep userlist_file /etc/vsftpd/vsftpd.conf | cut -c15-
Below which is the output for above command.
[code]...
View 6 Replies
View Related
Jan 24, 2011
I have a variable in which the data is stored as below:
variable_test=0m0.001s 0m0.001s 0m0.001s 0m0.001s 0m0.001s 0m0.001s .....an so on.
There are lots of values in format like "3m1.057s" are stored in variable_test separated with an space between two such values. For exapmple, value is "3m1.057s" I need to save different parts of a value in three separate array variables such as the
var_hour=3
var_min=1
var_sec= 057
tell if this can be done using "awk". A "WHILE" loop might be used to separate and store theses values I guess?
View 1 Replies
View Related
Dec 1, 2008
I am writing a script to mine data from my game server logs which are going to mysql. My first question is syslog-ng is sticking $msg into one colum it self, $msg tends to be a long string here are a few examples:
cn name team frag death flags role host.
0 Maik CLA 2 0 0 normal *.70.233.118.
1 lizardncd RVSF -1 3 0 normal
[*.22.247.40] lizardncd fragged Maik.
And they are not always in the same format, How could I parse this data to do something useful, like creating a user stats page with frag counts and flag counts. Should I create a script that runs in a cronjob that parses this data to a new table maybe once a day. Even then the $msg string value is random and I am thinking in terms of awk.
Second. I would like to keep track of what ip's are using what nicknames. Kinda in the sense of and ip associated to a array of names that have been used on that ip address so I can keep track of users. How would this work logically with a mysql db, could a column allow multiple values, then loadable into an array, which then can add, remove, and store to mysql.
View 1 Replies
View Related
Nov 22, 2010
I need to kind of grep within grep. My input file would be something like:
[Code]....
and I need to find the first occurrence of hello before MY PATTERN (hello 9008 in this case), so the output should be:
[Code]....
View 4 Replies
View Related
Nov 22, 2010
Basically, I am provided with a file "temp.dat" with 30 high temperatures (integers) in it. The program is supposed to read them in and compute/print the average. Then it is supposed to print the temperature of each day and, in addition, display a + by each day that is over the average, but only if it is above the average high for three or more consecutive days. This is the part I am stuck on. I'd appreciate any tips that would point me in the right directionFull disclosure: This is a school project. Code:
Code:
#include <stdio.h>
#include <stdlib.h>
[code]....
View 2 Replies
View Related
Feb 8, 2011
I'm playing around with a bash script to pass files to a program (such as VLC). I thought something like this would work:
vlc $( ls | sed 's/ /\ /g' | tr '
' ' ' )
ls shows the files in the current directory, then sed changes spaces to " " (to escape them) and finally tr removes the line breaks. I end up changing:
01 - Music Track.mp3
02 - Another Track.mp3
to
01 - Music Track.mp3 02 - Another Track.mp3
"vlc 01 - Music Track.mp3 02 - Another Track.mp3" works if I type it manually but my script reports a problem with `-'
It seems it's not escaping the spaces.
View 4 Replies
View Related
Dec 30, 2010
I want grep multiple files:
let abc.1.abc.2,abc.3,abc.4
I use grep <pattern> abc.* to read the files. But here grep read files in the order of 1,2,3,4 . I want to to grep the files in reverse order like 1st the abc.4 file will be read and abc.1 file will be last.
Is this possible ?
View 3 Replies
View Related
Feb 2, 2011
I have a Python script that I run which needs to execute under a special environment, so I would run the program like so from my working directory (~/project/src):
python manage.py shell
This opens up an interactive shell for me to start typing my own commands.I have another set of administrative activities that I would like to house in another directory (~/project/admin). The manage.py is really finicky about running from the working directory. So, to make this whole thing work, I made a script which starts off like so:
#!/usr/bin/python ../src/manage.py shell
There are a couple problems with this. The first is that it doesn't work:
/usr/bin/python: can't open file '"/../src/manage.py" shell': [Errno 2] No such file or directory
How do you specify multiple parameters to the interpreter?How do I change the working directory?
View 2 Replies
View Related
Jun 17, 2009
I have a list of words that I want to grep in many files to see which ones have it and which ones dont. in the text file I have all the words listed line by line, ex: list.txt:
check
try this
word1
word2
open space
list ..
I want to grep each line one by one. like I want it to
grep "check" *.log
grep "try this" *.log
grep "word1" *.log .. etc how can I do this?
and maybe write the output to a file.
View 5 Replies
View Related
Jul 27, 2010
I am splitting a file based on the values read from an input file. The below one is the script.
1)How do I add the header which is present in the original file to the new split files created?(For eg. pharmacyf conatins header as table column names. The new files created (ODS.POS.$pharmacyid.$tablename.$CURRENT_DATE.dat) are without the header).
2) Also the script is creating 0 byte files for the pharmacyids which are not available in the intial file? Can this be avoided?
for pharmacyf in *
do
tablename=`echo $pharmacyf |cut -f4 -d'.' `
while read pharmacyid
do
grep -w $pharmacyid $pharmacyf >> $OUT/ODS.POS.$pharmacyid.$tablename.$CURRENT_DATE.dat
done< inputfile
done
View 2 Replies
View Related
Apr 20, 2011
I have files a, b, c and d. They're all relatively large and are served up by a static web server optimized for this purpose. I can get requests that look like this:
/abcd
/ad
/bacdac
...
Each request is basically a request for a concatenation of the files in the order of the letters. The list of possible requests is finite, but large enough that disk space will run out very quickly and be very expensive if I create all possible files via concatenation.Is there a way to create a pointer file like abcd that is essentially a multi-file symlink that first points to a then to be then to c then to d? So if the contents of the files were as follows:
a: hello
b: there
c: whats
[code]....
View 3 Replies
View Related
Jul 1, 2010
I am new in client server programming and i have written code for single server to single client communicating one port(3490). but i have no idea how my single server will be communicate with different client on different port.how this will be happen ?
Any idea to understand the logic or send any good link or any piece of code in c , i searched on net but all help was mostly for java progamming.
View 1 Replies
View Related
Apr 26, 2011
Is it possible to handle multiple signals in just one handler? I only know singal() catches one signal at a time
View 2 Replies
View Related
Sep 21, 2010
I need a shell script which gathers the data from a remote XML file and then displays it according to my needs.. I need this for my job due to the fact that I need to keep track price changes of euro, usd, gold, etc. The XML file I am talking about is located at this page: here. The reason I am posting the URL is that I need to use curl to get this file and it does NOT have newlines after each tag. I thought that that would be a problem. Here is what I need from the script: 1) curl to get the page 2) make use of sed, awk, etc. to display its contents in a more structured and readable manner as shown below:
A: 64.125 (% -0.26)
B: 81.130 (% -0.32)
C: 1.4930 (% 0.00)
D: 1.9590 (% 0.36)
View 5 Replies
View Related
Apr 15, 2011
is there an app, runs in Fedora, with a GUI, that can join multiple mp3s into a large single file?
I know Audacity can do this, but it's not really suitable for multiple files and I have voice notes that are in 10 minute chunks.
View 2 Replies
View Related
May 13, 2011
Have this script which is reading in a series of files, one at a time with while-do-done loop, each file goes through various greps/awk's where this info is then saved to various files for later use. i.e....
Script is being run on Linux Red Hat,
In one of the grep/awk's the output (currently) are 2 columns (min max), i.e....| awk '{print $1, $2}' | sort -u which outputs (e.g.)
The number of "min max" pairs varies from file to file. Want to output a single column of unique numbers from the min max pairs & get the number of them for input to a file...i.e...
Where <PROCESS> is some process/technique that will generate a single column of integers (increment of 1) to pipe into the next one (sort -u)
i.e. (example from above)
Have tried command seq - only works for single pair input i.e.
Is there any command like seq etc which will output a single column based on a input of min max numbers (increment 1) to pipe onwards to next command?
View 4 Replies
View Related
Nov 28, 2010
I am using Xfce as the desktop enviroment and Mozilla Firefox as the webrowser. Within the webrowser window, I do File>Save Page As. I save it, and the result is almost always foo.html and directory foo_files. But I think under KDE I could choose the format, one of them being something like "Single page" (only one file; the colecction of .png, etc is embedded into that file). And this is the format I want Xfce (or Firefox) to use when downloading to hdd.
View 1 Replies
View Related
Nov 9, 2010
I have a text file that needs to be updated to be used by some fortran code later. it looks like that:
1,1 !Ms for y-stations, Ns for x-stations
0,0,0, !xsta, ysta, zsta
I need to replace the "0,0,0" with new values for xsta,ysta and zsta;
View 4 Replies
View Related
Apr 20, 2010
I have a problem passing a file descriptor from one process to another.
I have two processes A and B. Both are running in different network and
filesystem namespaces, so it is impossible to use unix domain sockets or
net sockets to pass a file descriptor from process A to process B.
The usage of STREAMS is also impossible, as you can see in
fixunix.com/unix/84093-streams-pipes-ioctl-i_sendfd.html
[quotation begin]
Linux doesn't have STREAMS, which are the System V way
of doing this task. ...
[quotation end]
Are there additional possibilities for file descriptor passing like
using named pipes or something like that or does anybody know
a good workaround for this problem ?
View 9 Replies
View Related
Aug 10, 2010
I am interested in using the grep method in the shell of my CentOS machine to obtain patterns from a file and use them to search through another file and highlight the patterns found. For example:
pattern file:
one
two
three
test file:
AAAAAAAAAAAAAAAAAAAAAoneAAAAAAAAAAAAAAAAthreeAAAAAAAAAAAA
View 8 Replies
View Related
Jan 29, 2011
I have a .txt file with the format
0.01
0.04
0.07
...
0.83
I am wanting to load this into octave and perform operations on the data.
[Code]...
View 2 Replies
View Related
May 3, 2010
I want to know that is there any method to grep a particular data from a file without using the "cat --- | grep ' ' " command....I need to use a system call for this functionality.
View 1 Replies
View Related
Oct 27, 2010
I want to write some code to search for a specific string in a text file, but without using grep command.
View 5 Replies
View Related
