Programming :: Parsing Out Squid Access Log With Awk And Grep?

Apr 25, 2011

I'm trying to recreate a simple script I wrote to parse out the access.log to get a rough idea of websites that users are going to on our corp network. The issue I'm having is I want to pull out any line from access.log that ends in .com/ .org/ .net/ or whatever to only see what the user entered into the address bar and drop pictures, js's and everything else and log only this. so what I do is :awk '{print $8} | grep -e '[cong]|[ore]|[mgtv][/]'$ and nothing happens.I know there is an easier way to do this with awk alone,

View 8 Replies


ADVERTISEMENT

Programming :: Input File - Grep Within Grep

Nov 22, 2010

I need to kind of grep within grep. My input file would be something like:

[Code]....

and I need to find the first occurrence of hello before MY PATTERN (hello 9008 in this case), so the output should be:

[Code]....

View 4 Replies View Related

General :: Squid Access / Permission Denied Error From Squid Occur?

Dec 29, 2010

I am using squid to controlling access to the internet all is working fine expect one of the user who is using outside organization portal to connect internet. But whenever he tries to enter in the portal by typing (EXAMPLE)url. Permission denied error from squid occur.

How can i allow this portal in squid. So squid will allow this to access.

View 1 Replies View Related

Programming :: Parsing Log File With Awk?

Aug 31, 2009

I have logs files from freeradius that have looks as follows:

$ grep "Login incorrect (rlm_ldap: User not found" /var/log/radius/radiusd-inner-tunnel-20090831.log
Mon Aug 31 09:25:27 2009 : Auth: Login incorrect (rlm_ldap: User not found): [John Doe] (from client oficina port 0 via TLS tunnel)

[code]....

I use the following line to get the amount of users that don't exist on ldap:

Code:

grep "Login incorrect (rlm_ldap: User not found" /var/log/radius/radiusd-inner-tunnel-20090831.log | awk '{print $14}' | sort -fu | wc -l

Now, awk on line one for example parses [John Doe] and [Joon Williams] as "[John" and that it's not what I'd want. I mean how could I do for awk looks username field as closed between squared brackets?

View 1 Replies View Related

Software :: Squid 3.0 Access List / Remove Redirect Statement From Iptables All Internet Access Is Blocked?

Jun 11, 2010

I have an old FC2 box running Squid version 2.5. It has been running since 2003 so I am in the process of replacing it. I have a new machine with FC11, iptables, and Squid 3.0 installed.

On the old machine I use iptables to intercept Port 80 traffic and send it to Squid. By default I block all internet access and allow only sites that are in an Allowed_Sites.txt file. Within Squid I also have statements to allow certain users to bypass Squid based on their IP address.

I have set up the same thing on the new box. I have iptables intercepting the Port 80 traffic and sending it to Squid. That is working because if I remove the redirect statement from iptables all internet access is blocked.

The problem I am having is that Squid is not blocking any websites. It acts like the ACL is set to http_access allow all. I have worked on this for several hours and am stumped.

These are my Squid rules:
acl allowed_sites url_regex "/etc/squid/Allowed_Sites.txt"
acl manager proto cache_object
acl localhost src 127.0.0.1/32
acl to_localhost dst 127.0.0.0/8 0.0.0.0/32
acl SSL_ports port 443
acl CONNECT method CONNECT
http_access allow manager localhost
http_access deny manager
http_access allow Bypass_Users
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
acl our_networks src 192.168.1.0/24
http_access allow allowed_sites
http_access allow our_networks
http_access deny all
icp_access deny all
htcp_access deny all
http_port 192.168.1.254:3128 transparent
hierarchy_stoplist cgi-bin ?
access_log /var/log/squid/access.log squid
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|?) 0 0% 0
refresh_pattern . 0 20% 4320
visible_hostname FC11.proxybox
icp_port 3130
coredump_dir /var/spool/squid

View 2 Replies View Related

Programming :: Get Group Name From GID Without Parsing /etc/groups?

Sep 14, 2010

Is there a Linux system call that can be used to get the group name from the GID returned by stat()? I realize that I could parse /etc/groups (if my user had sufficient permissions).

View 3 Replies View Related

Programming :: Parsing S-Expressions In Python?

Feb 1, 2011

I've been loosely following this:http://norvig.com/lispy.htmlAnd I have a problem: the parsing function throws an array out of bounds exception. I thought that maybe I'm doing it wrong, so I copy and pasted the code from the page, and still the same error

View 2 Replies View Related

Programming :: Parsing Xml File From A C Program?

Mar 14, 2009

i wanted to know how can I pars a xml file (read data from xml file) and from C program.is there any function to do this for me in C.

View 1 Replies View Related

Programming :: Parsing XML Using Sax Parser In Java

Jul 7, 2011

I'll show the code first:

Code:

What I am after is to get the string text from the clip tags. But for now I just tested to see if it can finds the command tags and print something if it does. But it doesn't find it. Anyone knows why ?

Looks like the xml is not good, i test it with a xml validator:

Code:

View 1 Replies View Related

Programming :: Perl: Parsing A Log File?

Nov 9, 2010

I have a log file (test.log) starting & ending within dash (--) as below. I am looking to write a parser for test.log. This test.log file currently has single value for one Job ID but I wish to parse for repeated N values of different Job ID - Job, User, Queue, Dispatched Date, Dispatched Time, Completed Date, Completed Time, Hosts/Processor, CPU_T and TURNAROUND. I can either output this 10 values in another .log file or dump into cgi.

The selected parameters from test.log for parsing with above 10 attributes are -

--
Job <345010>
User <xyz>
Queue <gaussian>

[code]....

View 3 Replies View Related

Programming :: Source Code On Email Parsing In C++ ?

May 11, 2010

Does anyone have source code on Email parsing in C++

View 1 Replies View Related

Programming :: Parsing And Storing Multiple Values Using 'awk'?

Jan 24, 2011

I have a variable in which the data is stored as below:

variable_test=0m0.001s 0m0.001s 0m0.001s 0m0.001s 0m0.001s 0m0.001s .....an so on.

There are lots of values in format like "3m1.057s" are stored in variable_test separated with an space between two such values. For exapmple, value is "3m1.057s" I need to save different parts of a value in three separate array variables such as the

var_hour=3
var_min=1
var_sec= 057

tell if this can be done using "awk". A "WHILE" loop might be used to separate and store theses values I guess?

View 1 Replies View Related

Programming :: Mail Log Parsing Script In Need Of Makeover?

May 30, 2011

I'm working on a script that will parse a mail machine's logs and print a list of email addresses in this format:

authen@domain.com | sender@domain | recipient@domain
exam
account1@domain1.com | sender2@domain2.com |

[code]...

View 2 Replies View Related

Programming :: Parsing Command Line Arguments In PHP?

Feb 1, 2010

I want to know how to get eg. the contents of a form on a webpage which has been passed to a server side PHP script, inside for example an array which I can read. I've been reading a ebook on PHP which as far as I can see doesn't cover this inside it.

View 1 Replies View Related

Programming :: Parsing Datetime Domain Names

Oct 25, 2010

I have spent lots of time trying to figure out how to parse the domain part of this file but am stuck.

How should this be approached:

Code:

Code:

The domains need to be parse at a maximum for 4 levels then the remainder will be put the last field.

View 3 Replies View Related

Programming :: [C] Parsing Lines Into Chunks And Writing Each Chunk?

Sep 10, 2010

My C foo is terrible! I am working with some code which reads lines from a file and then reformats the lines and writes them to a new file.The input lines look like this:

Code:
+[NEOTEST?]
+[NEOTEST]

[code]...

View 2 Replies View Related

Programming :: Bash Script Position Parameter Does Is Not Parsing '$1'?

Apr 8, 2010

bash script:

Code:
$ bash --version
bash --version
GNU bash, version 3.1.17(2)-release (i486-slackware-linux-gnu)
Copyright (C) 2005 Free Software Foundation, Inc.

Code:
[serv:]$ cat test.sh
#!/bin/bash

[Code]...

The question is, how to config it works through 'function cool {}' or cool() {} with position parameter

View 5 Replies View Related

Programming :: Memory Corruption While Parsing In Multithread Program?

Aug 28, 2009

language: C
OS: linux

I wrote a multithread program(approx 1000 thread have to run) and each thread has to parse a file(for each thread there is one file, ex:thread1 has to parse file1 and thread2 has to parse file2 like this....). I wrote "parse" program as follows. It is working well, if i create 50 threads. but if i run more than 200 thraeds Im getting doublefree corruption as follows:

Code:

*** glibc detected *** ./simulator: double free or corruption (fasttop): 0xb70007b0 ***

And some time I am getting parsing problem and error af follows:

Code:

powersetting.6607:1: parser error : Start tag expected, '<' not found (where powersetting.6607 is file name, when i check this file it is started with '<').

parse program as follows:

Code:

void parse_first(char* filename, int portnum, struct Poweroff *Powerptr1)
{
struct Poweroff *Powerptr;
Powerptr = Powerptr1;

[code]....

View 1 Replies View Related

Programming :: Parsing Unflagged Parameters In Bash Script?

Apr 6, 2010

I've searched online and found many examples using getopts, but nothing that clearly explains it use, nor any examples of what I'm trying to do. I have a script named "process". It can take from 0 to 3 different options. I'd like to be able to handle these options regardless of the order that they are entered.

Syntax:
process [-v] [-d #] [-h|-?] [string]
-v = verbose mode on
-d # = how deep to do the process, expecting a number parameter
-h or -? = show command usage
string = only process lines containing the specified string

[Code]..

View 5 Replies View Related

Programming :: Python - Parsing Data In Chunks / Sections?

Nov 5, 2010

As a curious side project I'm playing with mzXML data(an xml format for holding mass spec data). A typical scan can be quite large, even up into GB size. I'm wondering how would one go about parsing an xml file in sections, one section at a time. The idea being if the computer doesn't have enough memory to load up the entire data file, work on chunks of it at a time.

View 3 Replies View Related

Programming :: Bash Scripting - Parsing Text File By Character

Aug 11, 2009

Is there a way to process individual characters one-by-one from a text file in Bash, or is that hoping for a little too much from this lovable old clunker?

View 13 Replies View Related

Programming :: Free Open Source CSV Format Parsing Software

Jun 1, 2011

Google directed me to the ones written in .Net/C# etc. Any ideas on the ones written in C/C++?

View 14 Replies View Related

Programming :: Bash Variable Parsing / Extract And Put Into Variables Each Combination Of F1 And F2 In A Loop?

Oct 14, 2010

I have a bash variable where the content looks like this where ;f1; and ;f2; are delimiters:
;f1;field1value1;f2;field2 value1 ;f1;field1value2;f2;field2 value2 ;f1;field1value3;f2;field2 value3

So what I need is to extract and put into variables each combination of f1 and f2 in a loop to something like that:

#first pass of the loop I need:
f1=field1value1
f2=field2 value1

#second pass of the loop I need:
f1=field1value2
f2=field2 value2

# third pass of the loop I need:
f1=field1value3
f2=field2 value3

View 15 Replies View Related

Programming :: Parsing VoIP Call Detail Records (Field Format)

Jan 4, 2010

I am going to parse the Cisco voip CDR(Call Detail Records), please tell me the field format of the CDR files.

View 1 Replies View Related

Programming :: Parsing Text Files In Bash / Cant Seem To Read The Array Back?

May 29, 2011

PI'm trying to write a script to list all open ports in the MINIUNPND chain in iptables and use the procotol, port and destination ip to open ports on another router using upnpc.Here is the output of iptables -L MINIUPNPD

Code:
>iptables -L MINIUPNPD
Chain MINIUPNPD (1 references)
target prot opt source destination
ACCEPT tcp -- anywhere 192.168.3.124 tcp dpt:19955
ACCEPT tcp -- anywhere 192.168.3.124 tcp dpt:20054
ACCEPT tcp -- anywhere 192.168.3.130 udp dpt:10654
ACCEPT tcp -- anywhere 192.168.3.121 tcp dpt:29955
code....

No matter what i do i cant seem to remove the first 4 characters from the MYPROT array to leave only the digits. Also i cant seem to read the array back???

I thought it would simply be a loop reading each line and passing the fields in variables, executing upnpc commands i need then moving to the next line of the file until it reached the EOF.

View 12 Replies View Related

General :: Ls: Cannot Access Grep: No Such File Or Directory

Mar 16, 2010

I ran into a bit of trouble making a bash script. (Desktop is a directory, and I try to get it's modification date)

Code:

lamp:~# cmd='ls -l Desktop | grep -o "....-..-.. ..:.."'
lamp:~# $cmd
ls: cannot access |: No such file or directory
ls: cannot access grep: No such file or directory

[code]....

When I type in the command directly, without using an inbetween variable, it works fine.

View 3 Replies View Related

General :: Search Date Period In Access Log Using Grep ?

Feb 19, 2010

I have a three input day,month and year. Currently i am using the following script to take the data.

Any grep advanced technique to avoid for loop?

View 2 Replies View Related

Networking :: Control Bandwidth Access With Squid - Allow Certain IPs To Have Higher Bandwidth Access?

May 27, 2009

We have a sipmle office network set up that we also use use to connect to the internet, however of late the number of users has increased thus slowing internet access. Bandwidth upgrade is not an option thus i have to do bandwidth shaping on our linux router. The question is how do set the squid configs to allow certain IP's range a certain percentage bandwidtheg 60% and furthe divide the rest. Alternatively how can allow certain IPs to have higher bandwidth access.

View 1 Replies View Related

Programming :: Grep Not Getting Whole String?

Aug 23, 2010

I am trying to monitor how long an ldap search takes and maybe notify or something that a search takes longer than say 10 seconds.

Code:
tail -n 1000 /var/log/ldap.log
for SRCH in $( cat monitorldap.log |grep 'SRCH'); do
echo search string is
echo $SRCH

[Code]....

ok, so to start off with it doesn't appear to get the whole line, just a piece "Aug". How can I get the whole line into a variable so I can then cut it up into the pieces I need?

View 4 Replies View Related

Programming :: Grep -oE Not Portable?

Dec 28, 2010

I'm using Zabbix on which I can use give bash command to the agent.This 1-liner will give me all the interfaces with their IPv4 addresses.I have a 2nd expression which returns a checksum so I can detect a difference whenever someone deletes/adds/changes an ipv4 interface.This is the output on my Ubuntu-server:

Code:
~# ifconfig | grep -B1 ' inet ' | grep -oE '(^[a-z0-9:]*|addr:[0-9]+.[0-9]+.[0-9]+.[0-9]+)'
eth0

[code]....

View 8 Replies View Related







Copyrights 2005-15 www.BigResource.com, All rights reserved