maemo.org - Talk - Making Python faster (for fmms initially)

maemo.org - Talk (https://talk.maemo.org/index.php)

- Maemo 5 / Fremantle (https://talk.maemo.org/forumdisplay.php?f=40)

- - Making Python faster (for fmms initially) (https://talk.maemo.org/showthread.php?t=50737)

Re: Making Python faster (for fmms initially)

Quote:

Originally Posted by johnel (Post 622355)

When we talk about making Python "faster" do we mean:
(a) perceptionally faster (e.g. time to reach first screen)
(b) execution speed

any/all/both!
its the same discussion at this point! :)

Re: Making Python faster (for fmms initially)

Take it into the measurable realm by doing some benchmarking and profiling. When it comes to programming in general, picking the right algorithm that keeps down the complexity gives several magnitudes more effect than trying to micro optimize in lower layers.

When it comes to the Python run-time, check how it's compiled. What optimization is used? How will it affect what Python does the most (whatever that might be)?

What's the aim? 1% faster? 10%? Depending on the goal, different methods might have to be combined. With a properly compiled Python runtime, good choice of algorithms and a reasonable memory footprint that doesn't cause too many cache misses, 1-5% might very well be possible.

And don't forget the Linux environment. What scheduling is used in the kernels delivered by Nokia and the community? Has anyone looked into backporting the new ones, like BrainF u c k scheduling, that's specifically aimed at making the desktop faster? This might make the N900 as a whole faster.

So, lots of variables.

Re: Making Python faster (for fmms initially)

Far from a ready solution, but worth keeping an eye on (smells like Harmattan candidate material):

http://code.google.com/p/unladen-swa...ki/ProjectPlan

Re: Making Python faster (for fmms initially)

use C++/Qt4 instead of PyQt

Re: Making Python faster (for fmms initially)

Quote:

Originally Posted by TheBootroo (Post 622407)

use C++/Qt4 instead of PyQt

Verrrrrrry funny.

That is as helpful as a chocolate matchstick!

Re: Making Python faster (for fmms initially)

What we need are some benchmarks of how things are just now.

See where the bottlenecks are. Then start tweaking things in a controlled manner and compare the bechmarks to the original result.

If we are talking about a specific program then we need to find what routines are the most intensive then maybe re-implement the routine in C/C++ an link it into the Pthon-based program.

Profiling is probably the first step.

Python profiling stuff

Re: Making Python faster (for fmms initially)

Okay, not really a speed improvement. But the apple way to do the trick:

http://lists.maemo.org/pipermail/mae...er/021478.html

:)

Re: Making Python faster (for fmms initially)

Nouvelle note 47

Quote:

Originally Posted by TheBootroo (Post 622407)

use C++/Qt4 instead of PyQt

I'm askin for a ban for a such stupid comment !

Please consider that a laucher use memory and in some case can be slower.

Anyway ... there is simple things to optimize python scripts :

- When available, prefer module methode instead of self made in python, as most of module are in c. So really faster
- Avoid when possible too much search in namespace and prefer local method variables it s faster to assign a methode from namespace in a local var when it s called in a loop with more than 10 iterations
- abuse of profile to see where your code is leaking time (available on device)
- abuse of pylint
- instead of doing many test case, use exception : a simple example :

Code:

  if (mydiv#0):

    res=res/mydiv

  else:

    res=0

use:

Code:

  try:

    res=res/mydiv

  except NullDivisionError,e:

    res=0

- memorize pattern can help
- avoid cos , sin and tan in a game , use a precalculate array (for vectormine it s optimize fps from 3fps to 60fps)
- and again abuse of profile
- avoid converting data type in loop... Example QString to str
- use existing module when available : PyGtKEditor do all his parsing in python ... it s can be slow on large file, khteditor do the things clearly faster by using some Qt native class.

An other things to doesn't freeze the ui and to not slow down the device by using intensive thread is to divide tasks in very small tasks and to use a sigleton with gidle :

Code:

import gobject



class _DeferClass(object):

  _calls=[]

  _ref=None

  def __new__(cls,*args,**kw):

    if cls._ref is None:

      cls._ref = super(_DeferClass,cls).__new__(cls,*args,

                                                    **kw)

    return cls._ref



  def __len__(self):

    return len(self._calls)



  def __call__(self,func,*args):

    def NextCall():

      (func,args)=self._calls[0]

      func(*args)

      self._calls=self._calls[1:]

      return self._calls!=[]

    if not self._calls: gobject.idle_add(NextCall)

    self._calls.append((func,args))

I ve many example to give and there is so man things to say that i ll be able to write a book about py opimization.

So if you need so help optimizing code i can maybe help you.

Re: Making Python faster (for fmms initially)

Khertan you are Sweenie Todd :D

I'm eating Py, that s**t is so advanced I dont even know what it does:D

Re: Making Python faster (for fmms initially)

Quote:

Originally Posted by lcuk (Post 622298)

does removing the comments make a noticable difference to script runtime?

I can't say off of a benchmark but comments affect parsing. If pyc (or even better?) pyo files are already generated, stored, and run then comments shouldn't matter.

GCDialer (old name for Dialcentral) use to remove extra whitespace and merge multiple python files into one. I didn't feel the cost/benefit was worth it but didn't benchmark it.

I did do some benchmarking and found that class assignments are slow. When creating a lot of class variables (like pre-compiling a bunch of regexes) it was faster to create them on the instance. I normally have only one instance so no extra copies to worry about.

For general python optimizations there is already a decent guide
http://wiki.python.org/moin/PythonSpeed/PerformanceTips

Quote:

Originally Posted by attila77 (Post 622382)

Far from a ready solution, but worth keeping an eye on (smells like Harmattan candidate material):

http://code.google.com/p/unladen-swa...ki/ProjectPlan

Benchmarks are so-so for now but they haven't gotten a chance yet to do the optimizations they wanted. The real killer is this is very memory intensive which the acceptance into CPython is conditioned on improving memory usage. Start time is also an issue for now.

http://www.python.org/dev/peps/pep-3146/

Long term, PyPy is the project I'm more excited about. They've recently added support for CPython extension modules which was a major blocker for transitioning. Speed is a mixed bag. I'm unsure about start time or memory usage though.

http://speed.pypy.org/overview/