General :: Looping Through Csv File To Calculate Avg/min/max?
Sep 1, 2010
I have a csv file (id, loc, timestamp, impressions) that I need to derive some low-level calculations from. Im ok with basic sort and grep operations, but this one is challenging my feeble scripting skills (normally I�d throw it into a spreadsheet, but in this case Id like to ultimately create a script that I could automate). Ive already used cat/sort to order the csv file first on location and then timestamp so it looks like this:
What Id like to do is output to file a record of the total, min, and max number of impressions for each location. , e.g.
AA6504 3231
AB252525
AC37925109
ZZ35512341
how to do this and Im kind of stuck on how to approach it I think the combined steps of looping through the locations along with the min/max/sum calculations are throwing me. Eventually Id also like to do additional calculations like average, ect. but I think I can figure that out on my own if I can just get this part down.
instead of importing a file I would like to use the variable $x I tried using pipes, but with no luck. My goal is to read one line at a time, but not have to export my data to another file, I would like to keep it all within one script.
I'm working on the Solaris environment and the DB i'm using is Oracle 10g. Skeleton of what I'm attempting; Write a ksh script to perform the following. I have no idea how to include my sql query within a shell script and loop through the statements. Have therefore given a jist of what I'm attempting, below.
1. Copy file to be processed (one file at a time, from a list of 10 files in the folder ).
Code:
for i in * do cp $i /home/temp
2 . Create a snapshot(n) table : Initialize n = 1
Code:
create table test insert account_no, balance from records_all; -- creates my snapshot table and inserts records in SQL
3. Checking if the table has been created successfully:
Code:
select count(*) from snapshot1 -- query out the number of records in the table -- always fixed, say at 400000
Code:
if( select count(*) from snapshot(n) = 400000 ) echo " table creation successful.. proceed to the next step " else echo " problem creating table, exiting the script .. "
4. If table creation is successful,
Code:
echo " select max(value) from results_all " -- printing the max value to console
how to calculate (if possible) the end address of an image file in a flash memory. I'm trying to create a checksum and checkheader function and the info that I got is the file's offset, how many sector it consumes and its size. I kinda need the end address, sad thing is, I don't know how to calculate it.test.img's start address is 0, the size is 0x20000 and consumes 3 sectors.
I am familiar with common video resolutions like 320x240, 640x480 and 720x480. Sometimes I want to use other video resolutions when encoding videos. Is there a formula to calculate the unknown value for the heightxwidth if one value is supplied? For example, if I want my video to be 480xWidth, what value should Width be to get proper aspect.
Between the dates 21-June-2007 to 12-December-2010 there are 1270 days. If the total cumulative time a hard drive has been active is 344 days (found this via Disk Utility in Ubuntu 10.10) since it was bought 1270 days ago. What is the average usage hours per day?
Now i am writting one bash script. in that my requirement is i need to create one directory and that the directory details to be stored in one file Ex. date/time and all in one file. after that i need to delete the folder automatically exactly after 3months. between these time period in 2month itself i need to send one mail to admin "regarding this still one month only more to delete the folder" . is it possible to do like that date calculation in script.
I am trying to grep a particular string from the files of 2 different servers without copying and calculate the total count of its occurence on both files. File structure is same on both servers and for reference as follows:
I've had openSUSE for a week. The update says that there are 10 updates. When I run it, first I get a flag saying that something can't be removed. Then when I run the rest of the up date, it loops back and starts over. I need a patch to fix the patch that fixes the patch that...etc.
I have 9.10 installed, grub has loaded and worked, now all of a sudden nothing will boot.The HP screen comes up and then the computer reboot, constant reboot cycle. Can't even get into the BIOS. It will just sit there if I hit escape. Won't boot the live cd either. What can I do??? It has worked fine for the past month now.
I've written a bash script to connect to each machine in our local network and update the fstab file to point certain NFS mounts to a new location.The script is meant to read in a list of shares and a list of workstations, rsync the relevant share then connect to each workstation and update the fstab file before moving onto the next share.In reality what is happening is it will perform the loop correctly for the first workstation in the list but will not move on to the next one. I'm really stumped on this and I'm sure it's something simple but I just can't seem to work out the answer!
I have a simple .bashrc backup script I've been working on, but my external hard drive needs to be plugged in for it to work. So I set up a while loop, that exits after the hard drive has been plugged in and backed up. Here it is:
I am querying a single string column in a table. The string values have spaces in them. I want to loop over each value in bash. I set IFS to split lists on newlines instead of spaces. When I try this, it is splitting the list of results on the actual character 'n', not the newline ''.
DATA=`mysql -u root -ppassword --silent 'SELECT name FROM table_a;'` IFS=$' ' for i in $DATA; do echo "item = $i"
Why, after a successful (I thought) dual boot installation w/ a side-by-side lynx and meerkat install and XP does my system refuse to boot? It just gets to the Dell screen, giving boot options (none of which are making any difference) and keeps looping it. Last night, after re-installing the grub (lost after the XP install) everything was working fine. I powered down and this morning tried to restart and...this. I really don't want to have to reinstall everything and reconfigure and reinstall apps--what can I do? And more importantly, WTF did this happen to begin with?
I have an Excel Spreadsheet which reports upon weekly performance. One portion of the worksheet looks at week over week increases /decreases. I've placed a number of image controls (15 in Total) next to cells which calculate week over week change. The image controls simply display an image depending on whether the value of the adjacent cell is positive or negative. So, here is my basic working IF statement:
Code:
Private Sub Worksheet_Change(ByVal UpDown As Range) If Range("J26") > 0 And IsNumeric(Range("J26").Value) Then Image1.Picture = LoadPicture("C:path oup_arrow.gif") ElseIf Range("J26") < 0 And IsNumeric(Range("J26").Value) Then
[code]....
As you can see, this is pretty simple. What I'd like to do is find a way to loop through all 15 controls and avoid having 15 IF statements run on the Worksheet_Change() event. The cells containing the week over week values are all in the same column, but not continuous. Involved cells are: ("J26", "J27", "J33", "J34", "J35", "J36", "J37", "J38", "J39", "J40", "J42", "J43", "J44", "J45", "J46")
An unresolvable problem occurred while calculating the upgrade: The package 'skype-common' is marked for removal but it is in the removal blacklist.
This can be caused by:
* Upgrading to a pre-release version of Ubuntu
* Running the current pre-release version of Ubuntu
* Unofficial software packages not provided by Ubuntu
This is most likely a transient problem, please try again later.
I tried again several times in different days and it is the same. I have 10.4 recently updated from my original installation of 9.10. I think it was not a pre-release, I did it automatically from the website and it took a long time. As far as I know, all I have installed is official, except perhaps Skype itself. There seems to be something connected with Skype. But I have exactly the same Skype I had with 9.10, where I updated every few days with no trouble. It is something peculiar in 10.4
I tried to upgrade Ubuntu Server Edition 9.10 to 10.04. Gives me this warning:
Code:
Could not calculate the upgrade An unresolvable problem occurred while calculating the upgrade: E:Unable to correct problems, you have held broken packages.However, the packages appear to be fine. Checked with apt-get and nothing wrong.
i have problem with upgrade manager. i try upgrade fron 8.04 to 8.10. where is my main.log:
2010-07-19 15:52:51,749 INFO release-upgrader version '0.93.34' started 2010-07-19 15:52:51,752 DEBUG Using 'DistUpgradeViewText' view 2010-07-19 15:52:51,795 DEBUG enable dpkg --force-overwrite
I received the following error today and I did a search on the net. I tried sudo rm /var/cache/apt/archives/* but it didn't help. Could not calculate the upgrade An unresolvable problem occurred while calculating the upgrade. report this bug against the 'update-manager' package and include the following error message: 'E:Error, pkgProblemResolver::Resolve generated breaks, this may be caused by held packages.'
