[Cubicweb] work on dataimport stores
aurelien.campeas at gmail.com
Thu Feb 26 15:11:44 CET 2015
2015-02-26 13:33 GMT+01:00 Sylvain Thénault <sylvain.thenault at logilab.fr>:
> On 26 février 12:38, aurélien campéas wrote:
> > If we want to compare comparable things, we should have at least a run
> > with hooks explicitly disabled. That's what I do in the first joint diff.
> IMO we should include fastimport and RQLObjectStore both with and without
> hooks activated. In any case, we should ensure that standard metadatas are
> properly set and we could tweak hooks so that e.g. only integrity hooks
> are run.
In any case, fastimport provides metadata (even "incorrectly" e.g. if the
category is disabled).
"activeintegrity" hooks may have to be special cased also ...
> > Note that fastimport may appear faster than it should because it is
> > symmetric relation handling (and I havent quantified its impact, but it
> > should
> > not be huge).
> Probably not.
> > With hooks activated (followup diff) the run time explodes. It shows
> > how ruthlessly inefficient the cubicweb hook system is currently... but
> > nothing that we didn't knew before hand.
I did a mistake that made the hooks run twice, so it is a little less bad
than it really
> I though the point of fastimport was to provide a custom hooks manager
> implementation. And that's why I thought we should run the store with hooks
> activated (and now that's why we should also add the RQLObjectStore with
> activated for fair comparison).
Yes, that's the point. But it's important to have a benchmark that compares
behaviours of the current stores, without the hooks overhead, just showing
run time aspects overhead.
I'll leave it to you to schedule a night-run of the RQLObject store with
> > Note that fastimport precisely gives tools to fine-tune the runnable
> > (decide which to skip or which to defer in later transactions) to work
> > around
> > this general issue.
> OK. This should be demonstrated or at least described a bit in the (yet to
> written) "strategy" section.
I'll start with the (yet to be written) fastimport documentation ...
Not that the code is unreadable, but some simple high-level description of
should be provided.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Cubicweb