[Cubicweb] Entity deletion is slow

Aurélien Campéas aurelien.campeas at logilab.fr
Tue Sep 9 09:53:44 CEST 2014


On 09/09/2014 09:38, Julien Cristau wrote:
> On Fri, Aug 29, 2014 at 10:37:31 +0200, Yann Cointepas wrote:
> 
>> Hi,
>>
>> I have a database with 309 862 entities of type ScoreValue (from medicalexp
>> cube). As shown below, accessing these entities takes less than 3 seconds
>> but deleting them takes more than 23 minutes. I did the same test after I
>> removed as many hooks as possible (cube without dependencies, copied only
>> the entities I need from other cubes, removed all "composite" in relations)
>> but there is no difference. I would appreciate to understand why is it so
>> slow. And if there is a solution to speed-ud entity deletion it would be
>> great.
>>
>> I discovered this problem because I have several studies in my brainomics
>> based instance and sometimes I need to remove completely one study to
>> restart the script that populate the instance. I use a simple query 'DELETE
>> Any X WHERE X belong_to S, S name "MEMENTO"' (belong_to is a relation I
>> added to manage permissions). I focused on ScoreValue just to have a simple
>> example but I would like to speed-up the deletion of the whole study.
>>
> You could try enabling hook debugging:
>   from cubicweb.server import set_debug, DBG_HOOKS, DBG_OPS
>   set_debug(DBG_HOOKS|DBG_OPS)
>   rql('DELETE ...')
>   commit()
> and see if that gives you a clue about which hooks or operations are
> getting in the way.

It's not even such and such "delete" hook that may be slow (though I
second the advice given there) but just running through the hooks
manager (and query planner) costs an arm and a leg. You have to get
each entity in memory in a selector and hooks consummable form, one by
one ...

So you may want to code your own Study.cw_delete, which:

* bypasses the hook system (e.g. do it in sql)
* implements some of the cubicweb core hooks for
  the cubicweb low-level tables

I know some people at logilab have "fast-delete-entity" script, which
would be a good starting point.

Regards,
Aurélien.







More information about the Cubicweb mailing list