Programming :: Parsing And Storing Multiple Values Using 'awk'?
Jan 24, 2011
I have a variable in which the data is stored as below:
variable_test=0m0.001s 0m0.001s 0m0.001s 0m0.001s 0m0.001s 0m0.001s .....an so on.
There are lots of values in format like "3m1.057s" are stored in variable_test separated with an space between two such values. For exapmple, value is "3m1.057s" I need to save different parts of a value in three separate array variables such as the
var_hour=3
var_min=1
var_sec= 057
tell if this can be done using "awk". A "WHILE" loop might be used to separate and store theses values I guess?
View 1 Replies
ADVERTISEMENT
Dec 1, 2008
I am writing a script to mine data from my game server logs which are going to mysql. My first question is syslog-ng is sticking $msg into one colum it self, $msg tends to be a long string here are a few examples:
cn name team frag death flags role host.
0 Maik CLA 2 0 0 normal *.70.233.118.
1 lizardncd RVSF -1 3 0 normal
[*.22.247.40] lizardncd fragged Maik.
And they are not always in the same format, How could I parse this data to do something useful, like creating a user stats page with frag counts and flag counts. Should I create a script that runs in a cronjob that parses this data to a new table maybe once a day. Even then the $msg string value is random and I am thinking in terms of awk.
Second. I would like to keep track of what ip's are using what nicknames. Kinda in the sense of and ip associated to a array of names that have been used on that ip address so I can keep track of users. How would this work logically with a mysql db, could a column allow multiple values, then loadable into an array, which then can add, remove, and store to mysql.
View 1 Replies
View Related
Nov 4, 2010
I have to read a couple of numbers from a random.txt file. In this .txt file there are random numbers. They are separated by a space. Example if you opened test.txt:
test.txt :1 6 1 3 6 8 10 2 4
I would like to read those numbers using CAT and store them into an array:
numlen=${#num[*]} - (must be like this because it is a part of a larger program)
View 5 Replies
View Related
Nov 22, 2010
Basically, I am provided with a file "temp.dat" with 30 high temperatures (integers) in it. The program is supposed to read them in and compute/print the average. Then it is supposed to print the temperature of each day and, in addition, display a + by each day that is over the average, but only if it is above the average high for three or more consecutive days. This is the part I am stuck on. I'd appreciate any tips that would point me in the right directionFull disclosure: This is a school project. Code:
Code:
#include <stdio.h>
#include <stdlib.h>
[code]....
View 2 Replies
View Related
Nov 13, 2010
I have a huge binary log file. There are lets say 4 id's that I want to find in a log file. I know that those 4 id's will be present in the log file and I also know in what order they will be present. I want to find 1st id from the log then 2nd id and then third id and so on..
Simple/inefficient solution is: Loop through the id's and then grep in the log file. Problem with this solution is for each id grep will search from the beginning of the file.
Better/efficient solution would be: Sine I know the order in which id's will be present in the log file. Loop through id's, grep 1st id and then move on to grep 2nd id and so on...this way I can grep all id's in one pass. Is this solution possible ?
I have 500000 + values to find in log files and I have to find efficient solution for it.
View 2 Replies
View Related
Jun 21, 2011
I am using nessus to scan systems, and it generates the report file without any specific delimiter. Shown below is how a typical line in the file reads (It might show up as a very long line.
Code:
xx.xx.xx.xx|cpq-wbem (2301/tcp)|49272|REPORT|Synopsis :;;The remote web server is affected by multiple vulnerabilities.;;Description :;;According to its self-reported version number, the HP System;Management Homepage install on the remote host is earlier than 6.2. ;Such versions are reportedly affected by the following ;vulnerabilities :;; - Session renegotiations are not handled properly, which; could be exploited to insert
[code]....
The field of interest (for me) are IP ADDRESS, Synopsys, Description, Solution and Risk Factor. I tried the following command to parse this:
Code:
sed s/:;;/|/g report.nsr | grep Risk Factor |High | awk 'BEGIN {FS = "[:][;][;]"} {printf("%s|%s|%s|%s",$1, $6, $7, $10)}'
But this would not give desired results as ':;;' is not a standard delimiter and hence gives incorrect results. how to grab everything that occurs between the strings 'Synopsys', 'Description', 'Solution' & 'Risk factor' in the above example? In other words i may be asking to use multiple delimiters.. in the order of occurance.. is there a tool that supports multiple delimiters?
View 7 Replies
View Related
Feb 3, 2010
I'm looking to setup a home server for the purpose of backing up and storing the files on our multiple (Windows) computers. What kind of server should I set up? Samba? Lamp?
View 5 Replies
View Related
Apr 30, 2010
I'm writing a command-line flash card program in Python. I've tried many existing applications, but none fit my specialized needs.
All of the words I'd like to study are manually added to a text file (study.txt). Each time the software is loaded, it checks for new words in that file and also compares them to a dictionary (a tab-separated file, dictionary.txt), and adds that to a matrix. The flash cards are thus automatically generated from study.txt and dictionary.txt.
Now, the software must manage lots of information about each word I'd like to study and add new information made while interacting with the program (such as when that word should be studied next).
That seems easy to put into a matrix. But what is the best way to put that information in a place where I can pull it back when I run the software tomorrow? Do I need to save the matrix to a CSV file, then convert the CSV file back to a matrix the next time the application is run?
View 1 Replies
View Related
Dec 15, 2010
I have a program that sends QByteArray datagrams over a udp socket. I would like to have 4 bytes of the datagram that contain a 32 bit integer. When saving numbers to the QByteArray, I have tried the static function number(int) and member function setNum(int), but they convert the integer to its decimal string representation and save that in the byte array. So if the number were 10, it takes 2 bytes, if it were 10,000,000 it takes 8 bytes. This wastes space, and makes it more difficult to get the number when it is packed with a few other pieces of data in the same datagram. Is there a standard way of doing this in Qt?
View 1 Replies
View Related
Aug 9, 2009
I'm trying to get multiple numeric valued version information into a variable, all on the same line. I want for example $VERSION=3.1.0.01.002. I'm trying to pull seperate values from the file named version.properties, wherein the file contains;
patch.rel.num=0
sqa.num=01
major.rel.num=3
build.num=002
minor.rel.num=1
So I have the following script:
#!/bin/bash
BUILDVERFILE="version.properties"
PATCH=`grep "patch.rel.num" ${BUILDVERFILE} | awk {'print $1'}`
SQA=`grep "sqa.num" ${BUILDVERFILE} | awk {'print $1'}`
MAJOR=`grep "major.rel.num" ${BUILDVERFILE} | awk {'print $1'}`
BUILD=`grep "build.num" ${BUILDVERFILE} | awk {'print $1'}`
MINOR=`grep "minor.rel.num" ${BUILDVERFILE} | awk {'print $1'}`
P=`echo $PATCH | tr -d .=[:alpha:]`
S=`echo $SQA | tr -d .=[:alpha:]`
MA=`echo $MAJOR | tr -d .=[:alpha:]`
B=`echo $BUILD | tr -d .=[:alpha:]`
MI=`echo $MINOR | tr -d .=[:alpha:]`
VERSION=$MA.$MI.$P.$S.$B
echo $VERSION #
I end up with the value of .002 ?? Seems like it's only getting the last portion because if I run the script thru "bash -x" I get:
...
+ VERSION=$'3
.1
.0
.01
.002
'
+ echo $'3
.1
.0
.01
.002
'
.002
View 7 Replies
View Related
Feb 16, 2011
My question is about TCP parameters in Linux. By now, I want to change the default values of:
Initial Timeout
ACK Delay
Idle Connection Timeout
I have a Linux Box with kernel 2.6.x and 2 ethernet interfaces. I know TCP is a stack that doesn't have anything to do with ethernet devices. Said that, the question: is there a way to set custom values for each interface? For example, a server listening to connections in eth0 would use one value for Idle Connection Timeout and another server listening to connections in eth1 could use a different value for that parameter.
View 4 Replies
View Related
Mar 1, 2010
I have the following code snippet:
Code:
my $file = 'test/mybigfile';
my $zip = Archive::Zip->new();
my $zipped = $zip->addFile($file);
[code]....
it simply takes a file (in this case, "mybigfile") and creates a ZIP file containing that file only. Now, my problem is that when I go to decompress the ZIP file, it creates the relative (or absolute, depending on case) path of the original file. To make an example:
Code:
$ pwd
/tmp/perl
$ ls -R
[code]....
What I need is *not* to store the path information inside of the ZIP file, so that once I decompress the ZIP file I only get the original one without having paths created.
View 2 Replies
View Related
Aug 31, 2009
I have logs files from freeradius that have looks as follows:
$ grep "Login incorrect (rlm_ldap: User not found" /var/log/radius/radiusd-inner-tunnel-20090831.log
Mon Aug 31 09:25:27 2009 : Auth: Login incorrect (rlm_ldap: User not found): [John Doe] (from client oficina port 0 via TLS tunnel)
[code]....
I use the following line to get the amount of users that don't exist on ldap:
Code:
grep "Login incorrect (rlm_ldap: User not found" /var/log/radius/radiusd-inner-tunnel-20090831.log | awk '{print $14}' | sort -fu | wc -l
Now, awk on line one for example parses [John Doe] and [Joon Williams] as "[John" and that it's not what I'd want. I mean how could I do for awk looks username field as closed between squared brackets?
View 1 Replies
View Related
Sep 14, 2010
Is there a Linux system call that can be used to get the group name from the GID returned by stat()? I realize that I could parse /etc/groups (if my user had sufficient permissions).
View 3 Replies
View Related
Feb 1, 2011
I've been loosely following this:http://norvig.com/lispy.htmlAnd I have a problem: the parsing function throws an array out of bounds exception. I thought that maybe I'm doing it wrong, so I copy and pasted the code from the page, and still the same error
View 2 Replies
View Related
Mar 14, 2009
i wanted to know how can I pars a xml file (read data from xml file) and from C program.is there any function to do this for me in C.
View 1 Replies
View Related
Jul 7, 2011
I'll show the code first:
Code:
What I am after is to get the string text from the clip tags. But for now I just tested to see if it can finds the command tags and print something if it does. But it doesn't find it. Anyone knows why ?
Looks like the xml is not good, i test it with a xml validator:
Code:
View 1 Replies
View Related
Nov 9, 2010
I have a log file (test.log) starting & ending within dash (--) as below. I am looking to write a parser for test.log. This test.log file currently has single value for one Job ID but I wish to parse for repeated N values of different Job ID - Job, User, Queue, Dispatched Date, Dispatched Time, Completed Date, Completed Time, Hosts/Processor, CPU_T and TURNAROUND. I can either output this 10 values in another .log file or dump into cgi.
The selected parameters from test.log for parsing with above 10 attributes are -
--
Job <345010>
User <xyz>
Queue <gaussian>
[code]....
View 3 Replies
View Related
Jul 28, 2010
I have two files :
FileA
prot1
prot5
prot9
prot15
[Code]....
What I need to do is to extract from fileB the fields containing only the strings in fileA.
I thought awk could do the job easily with :
Code:
awk 'BEGIN { RS = "###" } /'$variable'/' fileB > output
where variable would maybe be the output of grep from fileA. So can I store the output of grep in a variable to use it afterwards with awk ?
something like that:
Code:
result=`grep prot. fileA` ; awk 'BEGIN { RS = "###" } /'$result'/' fileB > output
but that doesn't work. I'm always getting the entire fileB.
The output of grep get stored in the variable, I verified that with echo. So there is something that I just don't get... It seems to me that the above line should work.
View 11 Replies
View Related
May 11, 2010
Does anyone have source code on Email parsing in C++
View 1 Replies
View Related
May 30, 2011
I'm working on a script that will parse a mail machine's logs and print a list of email addresses in this format:
authen@domain.com | sender@domain | recipient@domain
exam
account1@domain1.com | sender2@domain2.com |
[code]...
View 2 Replies
View Related
Feb 1, 2010
I want to know how to get eg. the contents of a form on a webpage which has been passed to a server side PHP script, inside for example an array which I can read. I've been reading a ebook on PHP which as far as I can see doesn't cover this inside it.
View 1 Replies
View Related
Oct 25, 2010
I have spent lots of time trying to figure out how to parse the domain part of this file but am stuck.
How should this be approached:
Code:
Code:
The domains need to be parse at a maximum for 4 levels then the remainder will be put the last field.
View 3 Replies
View Related
Apr 25, 2011
I'm trying to recreate a simple script I wrote to parse out the access.log to get a rough idea of websites that users are going to on our corp network. The issue I'm having is I want to pull out any line from access.log that ends in .com/ .org/ .net/ or whatever to only see what the user entered into the address bar and drop pictures, js's and everything else and log only this. so what I do is :awk '{print $8} | grep -e '[cong]|[ore]|[mgtv][/]'$ and nothing happens.I know there is an easier way to do this with awk alone,
View 8 Replies
View Related
Sep 10, 2010
My C foo is terrible! I am working with some code which reads lines from a file and then reformats the lines and writes them to a new file.The input lines look like this:
Code:
+[NEOTEST?]
+[NEOTEST]
[code]...
View 2 Replies
View Related
Apr 8, 2010
bash script:
Code:
$ bash --version
bash --version
GNU bash, version 3.1.17(2)-release (i486-slackware-linux-gnu)
Copyright (C) 2005 Free Software Foundation, Inc.
Code:
[serv:]$ cat test.sh
#!/bin/bash
[Code]...
The question is, how to config it works through 'function cool {}' or cool() {} with position parameter
View 5 Replies
View Related
Aug 28, 2009
language: C
OS: linux
I wrote a multithread program(approx 1000 thread have to run) and each thread has to parse a file(for each thread there is one file, ex:thread1 has to parse file1 and thread2 has to parse file2 like this....). I wrote "parse" program as follows. It is working well, if i create 50 threads. but if i run more than 200 thraeds Im getting doublefree corruption as follows:
Code:
*** glibc detected *** ./simulator: double free or corruption (fasttop): 0xb70007b0 ***
And some time I am getting parsing problem and error af follows:
Code:
powersetting.6607:1: parser error : Start tag expected, '<' not found (where powersetting.6607 is file name, when i check this file it is started with '<').
parse program as follows:
Code:
void parse_first(char* filename, int portnum, struct Poweroff *Powerptr1)
{
struct Poweroff *Powerptr;
Powerptr = Powerptr1;
[code]....
View 1 Replies
View Related
Apr 6, 2010
I've searched online and found many examples using getopts, but nothing that clearly explains it use, nor any examples of what I'm trying to do. I have a script named "process". It can take from 0 to 3 different options. I'd like to be able to handle these options regardless of the order that they are entered.
Syntax:
process [-v] [-d #] [-h|-?] [string]
-v = verbose mode on
-d # = how deep to do the process, expecting a number parameter
-h or -? = show command usage
string = only process lines containing the specified string
[Code]..
View 5 Replies
View Related
Nov 5, 2010
As a curious side project I'm playing with mzXML data(an xml format for holding mass spec data). A typical scan can be quite large, even up into GB size. I'm wondering how would one go about parsing an xml file in sections, one section at a time. The idea being if the computer doesn't have enough memory to load up the entire data file, work on chunks of it at a time.
View 3 Replies
View Related
Oct 12, 2010
i am having two small issues with a function i have made.sorry if it is a mess, i am still learning bash.the first is calling the nonpersistssh function (second line) and assigning the return value to nonpersistdiag.the function returns 1, but nonpersistdiag seems to only contain 0. i am unsure on how to proceed.the second problem is the nested else clause on line 10. it is a syntactical error. how would i declare it correctly?
Code: function endsession(){
nonpersistdiag=$[nonpersistssh]# a function that returns an exit code
sudo /etc/init.d/ssh stop; sshdiag=$?
[code]....
View 6 Replies
View Related
