[Cubicweb] Which way to go to do remote RQL ?

Yann Cointepas yann at cointepas.net
Fri Aug 8 00:12:11 CEST 2014


On Thu, Aug 7, 2014 at 9:43 PM, Nicolas Chauvat <nicolas.chauvat at logilab.fr>
wrote:

> On Thu, Aug 07, 2014 at 10:14:47AM +0200, Yann Cointepas wrote:
> > I use multiple requests transactions because users are sending packages
> of
> > data. We garantie that a package that cannot be fully integrated in the
> > base will be rejected (and possibly corrected and resent by the
> > user).
>
> What are the reasons that a package can be rejected for?
>

It may be an error in the user data (e.g. invalid subject identifer) even
if we try to detect these errors early. There are also errors in the code
for pushing the data. It is changing because we are often adding new
studies, new data types and new kind of processings in our data flow.


> > Rejected package leave nothing in the base. Since a package can contains
> > various data (e.g. MRI Scans, MRI scans with processing results, only
> > processing results, etc.), the content of the package in processed in a
> > transaction.
>
> You could devise an Upload or Transaction object and use
> https://www.cubicweb.org/project/cwclientlib to implement the
> following workflow:
>
> 1) create a new Upload object
>
> 2) add scans, results, etc to the Upload object
>
> 3) check the correctness of the Upload
>
>    if it is correct -> move its content to the "standard context" of
>    the db, making it available to other users
>
>    if it is not correct -> delete the upload and all its content
>
> IIRC, we do something similar when uploading files in
> https://www.simulagora.com/ and we can for example resume interrupted
> file transfers to reach correctness before the upload becomes valid.
>

I did not know this cwclientlib I will definitely have a look. However, if
there is a commit after each RQL request, I see two difficulties to
implement the workflow you proposed:

1) For efficiency, I rely partialy on the Cubicweb constraints to detect
some errors in the data (especially unique constraints if someone send
twice the same data). For instance, if in a transaction I am asked to add
two subjects but the second is already in the database. Either, I will need
to reimplement the constraint (looking for existing subjects both in the
database and in the Upload object :() or I will have to find a way, when
the second subject raises an RQL constraint error, to remove all that have
been successfully inserted so far. IMHO my only option is to make a single
request with all the modifications in the transaction. But this lead to the
second difficulty.

2) I need to connect some objects that have been inserted during the
session. I use the eid returned by INSERT for that. I do not how to do that
in a single RQL command.

Let's take an example. There is a database with only a subject S. I receive
the following data:

Subject N
MRI scan for N
Subject S (error it is already in the database)
MRI scan for S

Let's suppose it is inserted by the following code:

db = CubicwebAPIWithRollback()
try:
    s = db.rql('INSERT Subject X: X identifier "N")[0][0]
    db. rql('INSERT Scan X: X concerns S, S eid %d' % s)
    s = db.rql('INSERT Subject X: X identifier "S")[0][0]
    db. rql('INSERT Scan X: X concerns S, S eid %d' % s)
except:
    db.rollback()
else:
    db.commit()

Because of the constraint error, there will be no change in the database at
the end of this code. How could I construct a code that would do the same
without transactions ?

Sorry for this long message.

      Yann
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.cubicweb.org/pipermail/cubicweb/attachments/20140808/efdab29e/attachment-0050.html>


More information about the Cubicweb mailing list