Programming :: Saving File Data Using Python In An Embedded System In An Safe And Fast Way?

Apr 19, 2011

I am developing a program in a system where the Linux does not take care of the sync command automatically. So I have to run it from my application always I save some data in the disk, which in my case is a 2GB sdcard. It is true that I can make the operation system takes care of the syncronization, using a proper mount option, but in this case the programm's performance drops drastically. In particular I use the shelve module from Python to save data that comes from a socket/TCP connection and I have to deal with the potencial risk of the system being turned off suddenly Initially I wrote something like that to save data using shelve:

Code:

def saveData(vo)
fd = shelve.open( 'fileName' , 'c')
fd[ key ] = vo
fd.close()
os.system("sync")

But that takes too much time to save the data. Note that I use the sync from the OS every time I close a file to prevent data corruption in the case of the "computer" being turned off with data even in the buffer. To improve the performance I made something like that:

Code:

def saveListData( list )
fd = shelve.open('file_name', 'c')
for itemVo in list:
fd[itemVo.key] = itemVo
fd.close()
os.system("sync")

Thus, first I saved an amount of objects in a list then I open the file and save the objects. In this way I have to open the file just one time to save a lot of objects.However I would like to know if adding a lot of objects before closing the file would increase the risk of data corruption.I known that turning off the system after fd.close() and before os.sync may cause problems. But what about turning off the system after

Code:

fd = shelve.open('file_name', 'c')

but before fd.close()?

View 7 Replies


ADVERTISEMENT

Programming :: Python - Get Text File Data Into An Array On Python?

Nov 30, 2009

I've already used line split stuff to transform my data into something like this in a text file:

Code:

['1', '1', '3', '20.7505207']
['2', '1', '3', '23.0488319']
['3', '1', '3', '-1.5768747']
['4', '1', '3', '-26.4772491']

[code]....

How can I get this on a python program so I can manipulate it as an array?

View 3 Replies View Related

Programming :: Python ConfigParser Not Saving Comments?

Jun 16, 2011

I have a lot of scripts in Bash and am 'converting' one of them to Python just to get to know how Python works. Up to now I'm loving it, the feel, the logic, the power, ...

1. I have a template file which will be filled in by a technician, meaning it's filled with comments (#) and variable = value combinations, where the technician needs to fill in the values needed under the section referring to the configuration he's performing. 2. After installing from DVD/ISO, the software RPMs get installed and there is a default configuration file containing variable = value combinations where the values need to be changed to the values indicated in the template file mentioned in 1.

In order to do this I played around (and Googled a lot) with ConfigParser. I can load the template, get whatever value I need, load the configuration file, set the new value to the correct variable and write the file.

I checked that the template and configuration files loaded (open()) contain everything, including comments. However when I write the configuration file all comments get lost.

Here's what I have so far: Class to create a fakesection header since ConfigParser needs sections and the configuration file doesn't have those.

Code:
class FakeSectionHead(object):
def __init__(self, fd):
self.fd = fd

[Code]....

View 3 Replies View Related

Programming :: Python - Parsing Data In Chunks / Sections?

Nov 5, 2010

As a curious side project I'm playing with mzXML data(an xml format for holding mass spec data). A typical scan can be quite large, even up into GB size. I'm wondering how would one go about parsing an xml file in sections, one section at a time. The idea being if the computer doesn't have enough memory to load up the entire data file, work on chunks of it at a time.

View 3 Replies View Related

Software :: Export An Embedded File System From A Windows Pc?

Nov 25, 2008

i want to export an embedded linux file system from a windows pc an NRFS server for this..

View 4 Replies View Related

Programming :: Converting A HTML File To A PNG File Through Python Script?

Jan 29, 2010

I have managed to create a HTML file inside python code,convert this to a PNG file through a Python script ?

EDIT: Details added: I have a python script which generates map-legends in the form of an html file. The legend generated have to be pasted on a map which is in a png format. A png format file can be pasted on another png format file easily. But because the legends generated are in a html format I cannot paste it on the map file !!

EDIT: Details added: I did Googling first but it resulted in various soft wares for above purpose which I don't want !!

View 12 Replies View Related

Programming :: Python: Using Strings To Run Commands In Os.system('')

Jun 20, 2011

I was zsync-ing the latest Ubuntu 11.10 Alpha and thought I'd make a little GUI for it as a small project. The gui is set up, I just need to figure out how to run zsync with content from to variables, cto and cfrom. I tried the following code:

[Code]....

View 4 Replies View Related

Programming :: Saving A PPM File - Binary Representation Of Unsigned Chars On The Harddrive

Apr 7, 2010

I am accessing a firewire camera using the libdc1394 library and saving the image as a PPM file, using the code below:

[Code].....

My question is whether the above code is portable. I presume it is, since the result is a binary PPM file which should be capable of being read across multiple computers with different architectures and different operating systems. But at the same time, all that the above code is doing is just saving the binary representation of unsigned chars on the harddrive, and there does not seem to be any reason why the binary representations of the unsigned chars will be identical across multiple computers.

View 1 Replies View Related

Programming :: Python : Pass Output Of System Commands To Variable?

Jan 25, 2011

I'm trying to write a python script that will use the current user's name when interacting. Ex: when started, it should say "hello daweefolk" when I am logged in.
I've tried
Code:
username=os.system("echo $USER")
but the variable remains empty.
What is the correct code?

View 1 Replies View Related

Ubuntu :: Saving Python Programs - Error Permission Denied

Apr 11, 2010

I wanted to save a python program because I am reading byte of python and I went to save in !/usr/bin. When I try to save in this folder I get this error "[Errno 13] Permission denied: 'usr/bin/helloworld.py'.

View 9 Replies View Related

Programming :: Send Some Data From A System To A Windows System And Vice Versa?

Mar 4, 2010

How can I do some socket programming using which I can send some data from a linux system to a windows system and vice versa. Can we do that using the IO:Socket:INET perl module??

View 10 Replies View Related

Programming :: Take A List From A File And Use It In A For Loop In Python?

May 11, 2009

so far I have this

list = open('list.txt')
for x in list:
list.read(x)

View 3 Replies View Related

Programming :: Parse The File Correctly With Python

Nov 27, 2010

I'm currently curios with my python program which the basic goal is to parse the character in mytestfile;let's see the code

Code:
f=open('/home/andrewraharjo/Desktop/snort.log','r')
j=f.read()

[code]...

View 3 Replies View Related

Programming :: Python Bluetooth File Exchange

Feb 13, 2011

I've run into a little problem, for which I seem to be unable to find an answer. The concept is the following: one machine runs a python script which advertises itself as 'OBEX File Transfer' and receives incoming data, using the Lightblue python module. The script itself is slightly different, but here's an example which effectively works in the same way:

[Code]....

print "Saved received file to MyFile.txt!" This works fine, though I would like to retain the original filename that is being sent to the machine, instead of overwriting a fixed file. Generating a new name wouldn't be such a problem, if I could get the MIME type or filename which is (presumably) being sent in the header of the request. Does anyone know of a way in python to receive incoming files and retain their filenames via Bluetooth?

View 2 Replies View Related

Programming :: DataBase In Python - Compile .py Files Into A .exe File

Jun 20, 2011

i am new to python programing and i have a couple of ?'s

1- what would i do if i needed to find the closest, bigger number, from a list of numbers when the user types a number in.

2- (windows) i need a program that can compile .py files into a .exe file so that it works on a machine without python it also needs to work with python 3.2.

View 8 Replies View Related

Programming :: Python Script Integration - Monitoring File?

Apr 5, 2011

Is there any way to make program in linux machine to make report when some files have been copy to another directory or machine and knows the users who copy the files, I am planning to make this program in c, honestly first time I want to make in python when I know about pyinotify and how easy to monitoring the file in machine, but the problem is I cannot integrate that script python to know the users who do that except for the one who create the file.

View 4 Replies View Related

Programming :: Sms Server Tools Cross Compiling Sms Server Tool For An Embedded Computer And Make Just One Binary File For It?

Mar 27, 2010

I wanted to know how can I cross compile SMS SERVER TOOL for an embedded computer and make just one binary file for it or how can I change all of its default files places like its demon and object file and gather all of them to one directory to execute and use and run.let me explain it better for you : I have an embedded computer with Linux OS that its file system is read only and I can not add any file to /usr /lib and ..... and I can just mount a SD memory card to it and copy all of my programs to it and run them from there as you understand I have two choices to choose, first make one big binary file for each program that I am doing it now and it is not a suitable solution and the second is finding the way to change default place of shared object file of my program.now you tell me what can I do to solving this problem.

View 1 Replies View Related

Debian Programming :: Searching For A List Of Strings In File With Python

Oct 14, 2013

I'm trying to search for several strings, which I have in a .txt file line by line, on another file.

So the idea is, take input.txt and search for each line in that file in another file, let's call it rules.txt.

So far, I've been able to do this, to search for individual strings:

Code: Select allimport re

shakes = open("output.csv", "r")

for line in shakes:
    if re.match("STRING", line):
        print line,

How can I change this to input the strings to be searched from another file?

So far I haven't been able to.

View 3 Replies View Related

Programming :: Lilypond Parser For Python - Typeset The Content Of File

Nov 12, 2009

I'm woring on a personal research project and would like to know if there are lilypond parsers for python available or I'll have to create my own. Just in case you are wondering: I don't need to typeset the content of the lilypond file, just understand what's written in the file (what notes, what duration, when in time to play each one, etc). [url]

View 1 Replies View Related

Programming :: Waiting For File To Copy Before Continuing Processing (Python)

Mar 25, 2010

I have a Python script that copies a couple of DLL's and EXE to a directory before running the EXE. It can be a fresh copy or the files can already be in the target directory and are then overwritten. The script uses shutil.copy() to copy the files and that works but as the files are copying processing continues and the script tries to run the files mid copy, causing an error.

I need a way to wait for the files to finish copying before the script continues. Putting the thread to sleep isn't good enough, calling os.system("copy ...") also doesn't work, using os.path.exist() won't work because the file will exist during the copy.

View 2 Replies View Related

Programming :: Writing Files With Python Produces Blank 0KB File?

Jun 22, 2010

I am working on a little project in python. i have produced this prototype

Code:
#!/usr/bin/python
# -*- coding: iso-8859-1 -*-
#DocC documentation prototype

[Code].....

View 2 Replies View Related

Programming :: Make Python Programs Run Without Entering The Command Python?

Mar 22, 2009

I want to be able to do

sudo ./program.py

instead of always having to do

sudo python program.py

What do I need to change?

View 5 Replies View Related

General :: Fast Copying Data From One Location To Other?

Apr 6, 2010

Is there any windows equivalent tools (like tera copy, Fast copy in windows) for linux for fast copying data from one location to other?

View 1 Replies View Related

Programming :: Get Data Of A Windows System?

May 7, 2010

get data(like CPU Usage, Memory Usage) of a windows system through my Linux server.Because I want to monitor all the systems that are in my network. How to get data from a windows system into linux ?

View 8 Replies View Related

Installation :: Running SLAX From RAM (Fast Access To Data)

May 14, 2009

Before the other day, I'd copied a live CD to ramdisk and run it from there before, but the disk was INX (INX is not X), a live CD based on Ubuntu that runs entirely in the text mode, no GUI. INX is a terrific product: colorful, educational, light, agile, fun to use, and and often damned useful, but when an OS only uses text, you may not notice how much running from the RAM speeds up an OS. Previously, I'd assumed that the best reason to run a Live CD from the RAM was to free up the CD ROM drive. When I started running a full KDE 3.5.10 Desktop from the RAM, it didn't take me long to notice the awesome boost in speed and performance.

The computer has the fastest access to the data that's in the RAM. (The "A" in "RAM" stands for "access", right?) So the machine is faster. As the RAM gets larger, I'm sure more and more live CDs are going to offer the RAMdisk option. Right now both INX and SLAX share the characteristic of being exceptionally small CDs, and that makes them well suited for this kind of application. The "minimal" version of Slax, the basic CD without any modules added, is less than 200 mbs, which fits very nicely on my 1024MB RAM. I now use the minimal SLAX cd to initiate the system, and I keep a collection of the modules on my hard drive to copy to the RAM and activate at will.

Here's a couple of screenshots: [URL] I'm using Wine here to run my one and only favorite Windows program, a text to speech program called READ PLEASE. Note that I am also running KTorrent, which is uploading from and downloading to my external MyBook hard drive. [URL]. Here's a shot of yakuake, which is sort of like Konsole with superpowers. I just upgraded my hard drive KDE system to 4.2.3, and they still haven't fixed Yakuake yet. I know it's been reported, so I'm sure it'll be taken care of.

View 5 Replies View Related

Programming :: Store Data Before The System Hibernates?

Mar 24, 2011

I have a requirment where I have to store some data before the system goes to hibernate.
How the kernel intimates to the application when it goes to hibernate. Is there any event or a signal that kernel posts to all applications?

View 4 Replies View Related

General :: Saving Data From A Failing Drive?

Dec 29, 2010

An external 3" HDD seems to be in danger of failing it's making ticking sounds when idle. I've acquired a replacement drive, and want to know the best strategy to get the data off of the dubious drive with the best chance of saving as much as possible. There are some directories that are more important than others. However, I'm guessing that picking and choosing directories is going to reduce my chances of saving the whole thing. I would also have to mount it, dump a file listing, and then unmount it in order to be able to effectively prioritize directories. Adding in the fact that it's time-consuming to do this, I'm leaning away from this approach.

I've considered just using dd, but I'm not sure how it would handle read errors or other problems that might prevent only certain parts of the data from being rescued, or which could be overcome with some retries, but not so many that they endanger other parts of the drive from being saved. I guess ideally it would do a single pass to get as much as possible and then go back to retry anything that was missed due to errors.

Is it possible that copying more slowly — e.g. pausing every x MB/GB — would be better than just running the operation full tilt, for example to avoid any overheating issues? For the "where is your backup" crowd: this actually is my backup drive, but it also contains some non-critical and bulky stuff, like music, that aren't backups, i.e. aren't backed up. The drive has not exhibited any clear signs of failure other than this somewhat ominous sound. I did have to fsck a few errors recently — orphaned inodes, incorrect free blocks/inodes counts, inode bitmap differences, zero dtime on deleted inodes; about 20 errors in all.The filesystem of the partition is ext3.

View 2 Replies View Related

Programming :: Get The Data From A Tag In XML File?

Dec 3, 2008

have a tag in XML file in unix like this <EmailAddress>abc@gmail.com</EmailAddress> this tag is there for multiple times in the xml file and the data is in continuous line like below State>UN</State><Zip/><CompanyName/><EmailAddress>FDF@gmail.COM</EmailAddress><PromoType>UNKNOWN</PromoType></Promotion></PromotionList<State>UN</State><Zip/><CompanyName/><EmailAddress>zd4946@gmail.com</EmailAddress>I have to check the data in between bold tags is valid or not ... means have to check whether its a email address or not and have to find the length of the attribute means tag ...script is in kshsorry if its already asked...i checked but i didnt get Exatly matching result for my requirement

View 7 Replies View Related

Programming :: C++ GUI Signalling - Encryption System Should Encrypt The Data In The Clipboard

May 30, 2011

I'm trying to write a GUI text encryption application. I wrote the encryption system in a No GUI application like this::

[Code]...

Now I'm trying to write a GUI version, using the same algorithm. Here goes a rough image of my main window code. If you scroll down you'll observe a coloured part. As you see, the text in the first textbox gets copied into clipboard. That is the part where my encryption system should encrypt the data in the clipboard, and copy it again, and later on the new data will be pasted. How am I supposed to write that? If I have to use another signal, what is the receiver of that signal?

[Code]...

View 5 Replies View Related

Programming :: Python 2.x - Represent The Same In Python 3.x

Mar 4, 2010

I have a function definition in a Python 2.x script which take a tuple as one of its arguments, but 2to3 has no answers nor any of my searching on how to represent the same in Python 3.x

Code:

def blah(self, (string1, string2))

View 4 Replies View Related







Copyrights 2005-15 www.BigResource.com, All rights reserved