[Cubicweb] Service API implementation with ømq

Pierre-Yves David pierre-yves.david at logilab.fr
Wed Feb 29 14:33:19 CET 2012

On Wed, Feb 29, 2012 at 02:09:26PM +0100, Sylvain Thénault wrote:
> On 29 février 12:53, Pierre-Yves David wrote:
> > On Wed, Feb 29, 2012 at 11:41:48AM +0100, Sylvain Thénault wrote:
> > > On 28 février 17:05, Pierre-Yves David wrote:
> > > > This goes in the wrong direction:
> > > > 
> > > > :using standard python call passing standard python reference:
> > > > 
> > > >     Not restricting usage of this API means that people will use it
> > > >     unrestricted. If people use it unrestricted it'll be painful to move toward
> > > >     more distributed approach.
> > > 
> > > What are you afraid of? IMO this is already "restricted" since this
> > > API is designed to be called through the db-api (so not from the repo 
> > > side btw), so arguments should be expected go through pyro/zmq which
> > > limits the notion of "reference". If that would make you feel better,
> > > we can ensure that arguments are python base types (and doesn't expect
> > > on reference passing side effect, though that's harder to check without
> > > limiting to unmutable types, which we probably don't want)
> > 
> > 
> > Pyro is never used repo-side so it does not cache any error here.
> As I said, this api is not designed to be used on the repo side.

Then, it is a major bug. The repo side on such matter the repo side **need** to
have at least as much capability than the web side. I have multiple instance of
code used both repo and web side.

> > Pyro is seldomly used any way. How much web only instance are running in
> > production? I do not believe this is really tested for most cubicweb based
> > code.
> > 
> > Pyro add several magic layer that make it hard to follow what's going on for
> > fellow cubicweb developper which never bent over Pyro
> We do use pyro based db-api, though you're right not for web ui communication.

We defitly use pyro based db-api. But we never execute more than a small
fraction of the whole cubicweb code base on it.

> > To conclude: I do not buy the "it's designed for Pyro" argument because it's
> > not used by Pyro in practice.
> > 
> > Moreover, manual theorical checking are less robust that pratical use of ømq (meeting the "let's experiment" argument)
> You suggest adding zmq to check arguments. Should I buy that?

No I sugguest using ømq now because we will use ømq in the future.

* Adding any dedicated code that will be removed when ømq is used is extra work
  that can be avoided.

* Setting ømq in place now is the cheapest way to ensure the futre are ømq compatible

> > > > :inside the main repository code and thread:
> > > > 
> > > >     This code does not belong here:
> > > > 
> > > >     Adding any logic code to repository is a bad idea. The logic we are adding
> > > >     here is related to "a server processing tasks" not "data repository".
> > > 
> > > No. We're not writing a tasks manager here, simply providing a generic
> > > way to call services that should be run on the repository side.
> > > For instance I do intend to rewrite some stuff which is currently written has
> > > repository methods such as "register_user"/"find_users" as regular services.
> > > The aim is to make the repository extensible without having to monkey patch
> > > it (we've a bunch of use case for this).
> > 
> > Yes we does. We are running task here. Some are light, some are heavy but all are
> > task (or at least most).
> And all related to data repository manipulation. Or shall we consider
> everything as a task?

Most of cubicweb is about data repository manipulation. It does not mean that
nothing can be handled outside the primary object use by cubicweb client to
access data.

Updating cache for mercurial repository is not the kind of operation that
should be done in the context of a object meant to allow a web worker to access
repository data. There is no reason why register a new user couldn't be done in
another context too. Fetching session and garbage collecting task have nothing
to do with fetch data to handle http request. etc …

More decoupling, allows better scaling.

> > It's not because we already do it that we should keep doing it. Adding more stuff be be cleaned up later *is a bad idea*[1].
> Though I still fail to see what'll have to be cleaned up later. 

See argument above. This task does not belong to web handling and will be
handled by other process in the future.

Pierre-Yves David


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://lists.cubicweb.org/pipermail/cubicweb/attachments/20120229/b4f03371/attachment-0271.sig>

More information about the Cubicweb mailing list