Wednesday, 21 August 2013

single iteration sharing the iterator

single iteration sharing the iterator

I have a lot of data, usually in a file. I want to compute some quantities
so I have this kind of functions:
def mean(iterator):
n = 0
sum = 0.
for i in iterator:
sum += i
n += 1
return sum / float(n)
I have also many other similar functions (var, size, ...)
Now I have an iterator iterating throught the data: iter_data. I can
compute all the quantities I want: m = mean(iter_data); v = var(iter_data)
and so on, but the problem is that I am iterating many times and this is
expensive in my case. Actually the I/O is the most expensive part.
So the question is: can I compute my quantities m, v, ... iterating only
one time over iter_data keeping separate the functions mean, var, ... so
that it is easy to add new ones?
What I need is something similar to boost::accumulators

No comments:

Post a Comment