Programming :: Limit In Number Of Fields That Awk Can Handle
Mar 2, 2011
I have a file with 200 000 lines and I want to append the fields of each line based on matching first field. The resulting file should have 70 000 columns but has "only" 18 000. The command I'm using is working perfectly with a smaller file, wich lead to 14 000 columns. Could there be a limit in number of fields that awk can handle ? Here's my awk command :
Code:
awk -F, 'END { for (k in _) print _[k] } { _[$1] = $1 in _ ? _[$1] FS $4 : $1","$4 } ' file > out
Also, this command writes ^M (windows line break) after each columns. Removing them is easy but where do they come from ? Working on Ubuntu 10.10
I am trying run audio conversion on my server that I want limited to a certain number of processes based on process name. I am using the following script but it isnt limiting the number of job like I want it to.
Code: #!/bin/bash $num_jobs = 13 while [ $(ps -A | grep -v grep | grep -c pacpl) -ge $num_jobs ] do sleep 1
I need to limit the number of ssh connections a user has. All the users are using tunnel only so their shell is set to /sbin/nologin The logins do not open a shell they just create the tunnel so /etc/security/limits.conf has no effect on them at all.
I tried setting 'MaxSessions 1' in sshd_config but either that doesn't not do what I expect it to or it plain does not work as even with a normal user I was able to open an unlimited number of sessions. I need a good secure way to limit each user to 1 ssh session without them having a shell but Im unable to find a solution.
I was nosing around in my /home folder and I noticed that the /.thumbnails directory had 38,000+ files in it. That number seem a bit excessive to me. Is there a way to limit the number of files that are allowed to be in that directory, and maybe delete the oldest files automatically when the directory reaches it's limit in order to make room for the new incoming files, so there are no "directory full" type of errors?
I have a server with 48 cores, 8 6-way Opteron CPU's. Ubuntu Server 9.04 only sees 32 processors. Is there a limit on the number of cores/processors that the server will use? Windows 2008 on the same server sees all 48 cores and the so does the BIOS, so this is unique to Ubuntu right now.
I'm looking for a solution for sendmail to limit the number of emails send per miniute per IP. For example all my local computer user with ip 192.x.x.x need to able to send 10 emails/minite (emails, not connections!. The rest of the world can send for example 200 emails/minute to the mailserver. If the amount of emails per minute is exceeded, sendmail needs to block receiving emails from the spesific IP. I want to do this to stop spaming from my local network. Is it possible?
I ran into a user today that indicated that their company only allows them to log in through a terminal session once (no multiple logins). On second try their login window terminates. They are using putty.Is this being accomplished through PAM or sshd ( or some other method)?
I have a standard home set-up for my Ubuntu OS, and I would like to know whether its possible to cut out the repetitive prompts to enter the password, as when you connect to the internet or access files on a partition that's not home, or install new software.
I am using ssh server to connect to my Ubuntu desktop. I opened the file sshd_config and change my port number of the server.I want to put a limit on the number of clients in the ssh server.
How to number of connections for a single ip on port 80 to CentOS 5.5 with iptables? connlimit did not work on CentOS and nginx does not provide a module for that
except is there is a way to enhance mod_limitipconn.c to ensure that apart from restricting one connection allowed from a given IP, also set so that an IP can only connect on every set interval ?e.g.restrict the number of connections from a given source IP to say once every 5 minutes or so?if not mod_limitipconn.c, any other mechanism to do the expected result?
I was searching around and I stumbled upon a Linux Kernelix Sockets Local Denial of Service exploit.I downloaded the exploit, compiled it ran it to check if I am vulnerable.As I was expecting, the exploit instantly "killed" my Maverick system and I had to use the power button to reset my computer...Is there any way to limit the numberof allowed open sockets?I don't think that this can be done using /etc/security/limits.conf in a similar way of preventing the fork bombs
Server is running a stripped down version of CentOS 5.3 (64-bit), running only the built-in Xen Virtualization Environment. There is no other services running on the server (not samba, httpd, sendmail, cups... nothing except Xen) We've created several virtual machines, and as long as we don't start a fourth virtual machine everything runs smoothly (impresive hardware).
Each virtual server is configured as:
PARAVIRTUALIZED 1 Virtual CPU 1 GB RAM
However, 5 minutes or so after starting a fourth virtual machine, the entire host server crashes and restarts itself. Are we limited by the number of cores on the host machine CPU (4 cores)? 1 for the host and 3 for virtual machines? We've read in forums about other Xen setups running up to 11 virtual machines on less powerful hardware? (a dual core server). Should we be using FULLY VIRTUALIZED virtual machines instead? Is the number of XEN virtual machines in fact limited by the number of cores? If so, how can someone run several virtual machines on a single core host?
By the way, we were replacing a previous Dell Server (Poweredge 2600 with 512 MB Ram and a single Xeon single core processor running Open Virtuozzo). We were able to run up to 16 virtual machines at the same time. Of course none of the machines endured hard work (testing environments, etc). But hey, my point is that we expected to get a much higher number of virtual machines on this new hardware.
I want to extract all the common entries in the given 3 columns(fields) in the file using awk::can somebody tell me how it can be done in awk...the file is::
ABD : 5869 events, relative ratio : 1.173800E-01 , sum of ratios : 1.173800E-01 VBD : 12147 events, relative ratio : 2.429400E-01 , sum of ratios : 3.603200E-01 SDF : 17000 events, relative ratio : 3.400000E-01 , sum of ratios : 7.003200E-01
I have an array called @logons. How can I step thru the array and split the fields? This is what I have so far, but doesnt work. I got the feeling I the split statement syntax is incorrect.
Code:
print @logons; foreach my $logons(@logons){ ($userid, $ip) = split(',',$logons);
[code]....
Update: Appears the data in @logons has a column header from the mysql query which I used to populate it with. So that code which I was testing does indeed work.
I want to write a function which calculates the space needed between fields, to generate a table with aligned fields, like when you type "ls -l", the operating system generates a table with beautifully aligned fields. I've got this code so far:
Code:
for line in $(cat tmpSearch) do line=`echo $line | tr ":" " "`
i am reading a database flat file with the "awk" command which has 4 fields separated by colon ":" .I want to show the output of these fields in a certain way that every record is numbered e.g
constructing a protocol where in it involves sending and receiving different files, so sending data over consists of file name, size of the file and content of the file, so while receiving can i use functions like strchr() to differentiate these fields for sockets? so to allocate memory for file to save i need to have its size.
I am faced with the following problem; I need to update all fields of type datetime without ... with the following sort data:01-01-2010 12:10:30.256 - this is just an example.these fields are restricted with a unique constrained. My question is how can I generate the above sort data, especially the last three numbers with leading point. After that how can I update a table with existing data with this newly generated data so that the fields really do carry unique data.As you probably can read out of my question, I read through the help in pgAdmin and came as far as generating date, maybe datetime but not the .123 number. And a little insert script I found in the help, that I modified as UPDATE resulted in updating only the last generated data in all required fields.
Is there any way to limit x number of samba users by samba ? Say if there are already 5 samba users using the share, I would like to restrict any futher samba requests.. How do i do that ?
Sometimes the variables will be 0 and sometimes not, and of course the output may also be 0%...What is the best method to handle division by zero errors in awk.
I would like to make program that reads a *.TXT file and searches for last line and constatly refreshing the search. So last line would look something like this:Mike Had A 100.0 Pound Shark.So there are multplie category of "fish"(Shark, Dolphin, etc.) and each of them have allowed weight (ie. minimum 70.0 Pound Shark, 50.0 Pound Dolphin). If in the found sentence Shark or Dolphin doesn't meet requierments than program named False.exe should be started. I dont know how to make it my self so any help is much appreciated. My knowledge on programming is very low but I'm opend for learning.
I want to display the contents of a particular log file (simple text file, I mean in Linux). But there is a problem: The contents need to be organized in a fixed format. Have a look at this log file:
So, while displaying the contents of above file on a web page, I want to format the field names found in the log file: User Name:, Reported Problems Description:, and Remarks:. These fields may contain a variable length of text and no specific line number is assumed for them to appear on.
Well, what I am trying to do may sound wierd to some of you. The filed "Reported Problems Description:" can possible contain text which embeds colon (.
I want to write simple firefox extension / script or anything to change URLs from HTTP to HTTPS for selected websites (e.g. facebook). That thing is actually bypassing some security checks in my network.
Can anyone tell me how to proceed? I can deal with language as far its C++ or Python (something else would just take more time that's all )
I'm new to php and need some pointers to worth while documentation 'cause I'm getting nowhere I want to make a simple html form that allows me to submit a csv file so that I can work on it.The problem seems to be that if the file is not in the root of the (web) application it won't work.The form doesn't seem to send the path with it so I am unable to (1) know where the file is, I just get the name of the file and (2) I couldn't access the file anyway as it's outside of the apache environment.Is there a way to up the file to memory? How would you do this
I need to make a daemon which listens to port 81 for messages like [URL] So far I made a daemon which serves as a simple stream server: I set up a socket to listen to a non-reserved port (like 9999), but I don't know how to read the query strings.
