[Cubicweb] Wild idea to speed up repository loading

Adrien Di Mascio adrien.dimascio at logilab.fr
Tue Jul 8 08:34:30 CEST 2014


Le 07/07/2014 22:21, Sylvain Thénault a écrit :
> On 07 juillet 22:11, Christophe de Vienne wrote:
>> I wonder if we could save the pickled schema, along with a timestamp
>> corresponding the latest modification of the scheme.
>> If so, before reading the whole schema from the database, we would only
>> have to unpickle the schema from a local file : that would be faster
>> wouldn't it ?

I was not sure that pickle was the best bet, so I made this little test 
in a cubicweb shell :

   import cPickle
   from logilab.common.decorators import timed
   ps = timed(cPickle.dumps)(schema)
   --> dumps clock: 0.331673000 / time: 0.332461119
   schema2 = timed(cPickle.loads)(ps)
   loads clock: 5.532999000 / time: 5.555926085

I might have overlooked something in my test but yams' internal 
structure is probably too complex for pickle to be efficient.

I'd rather try to optimize the loading from the filesystem (or simplify 

Adrien Di Mascio - LOGILAB, Paris (France).
Formations - http://www.logilab.fr/formations
Développements - http://www.logilab.fr/services
Gestion de connaissances - http://www.cubicweb.org/

More information about the Cubicweb mailing list