[Cubicweb] Multisource in CW

Sylvain Thénault sylvain.thenault at logilab.fr
Thu May 31 15:19:28 CEST 2012


On 25 mai 09:53, Sylvain Thénault wrote:
> > Yes, this is the idea. The second idea is that the query planner should 
> > understand that in "Any X WHERE Y linked_to X" it should use the URI to join 
> > between the two instances.
> > Is it doable to implement such a behavior (or is the query planner not 
> > flexible enough) ? 
> 
> I don't get your point here. Think to the local eid as an implementation
> detail for performance reason (we have tried  to have string as keys a while
> ago, moving to int had a huge performance impact). That's the source's job
> to translate the eid into the URI than to query whatever source to do the join.

After discussing on the phone with nico, I understand that:

* you actually want to use a cubicweb external source

* but uri rather than eid should be used so one can change the external instance
  without loosing external relation

* and since external data may be rather large, the way the ms planner works today
  isn't suited to this use (as already discussed)

To achieve thid, I'ld propose the following:

* have a source similar to pyrorql, but which use cwuri as extid rather than 
  eid (that should be pretty easy)

* have an alternative "on-demand" ms planner, as described earlier in this
  thread. IMO a simple implementation (eg only support external entities with
  their attributes, no relation) could also be done without too much pain.

Remarks:

* we would probably need some small ui tweaks to allow : 

  - easy cwuri customization (currently generated from base-url/eid, which is
    not desired here)
  
  - interface to link to a distant entity by giving its uri

* if this sounds good to you, we should take some time  together next week so
  I can put you on the way

Next points to improve that way would be :

* meta-data control to ease import of big data into a cw database

* more sparql (front-end and as a source)

-- 
Sylvain Thénault, LOGILAB, Paris (01.45.32.03.12) - Toulouse (09.54.03.55.76)
Formations Python, Debian, Méth. Agiles: http://www.logilab.fr/formations
Développement logiciel sur mesure:       http://www.logilab.fr/services
CubicWeb, the semantic web framework:    http://www.cubicweb.org


More information about the Cubicweb mailing list