[Cubicweb] running the web frontend and cw backend as 2 separate processes on the same machine

Aurélien Campeas aurelien.campeas at logilab.fr
Fri Jun 11 15:45:27 CEST 2010

Le vendredi 11 juin 2010 à 15:19 +0200, Sylvain Thénault a écrit :
> On 11 juin 14:08, Aurélien Campeas wrote:
> > I believe this should all be in the documentation.
> > 
> > I'd like to extend your questions with the following: would it be
> > complicated to have an all-in-one process handling web request & all
> > short-lived requests + one repository process handling the long-running
> > transactions (import/export, delete, clone) ?
> > 
> > For better use of a 2-cores processor, this split could really be
> > 
> > * 1 web front
> > * 1 short-lived requests repo
> > * 1 long transactions repo
> This is far more tricky, and I don't see how you expect this kind of
> thing to work... I don't think we want the web ui to deal with 2 distincts
> repos. The only thing I can imagine is to have to all-in-one repo, with
> a load balancer redirecting some urls to your 'long transactions instance'.

I have no preconception on how it is supposed to work. It looks like the
load-balancing done on urls might work. But more below.

> Though I'm afraid this won't be doable until we have persistent session
> (else one instance doesn't know anything about session of the other
> instance). Also that depends on things you may be doing in your hooks, 
> telling if you can get multiple instances without problems.

Actually what I see is the following: all long-running transactions are
also asynchronous transactions. Like:

When importing a case (this takes on average several minutes), just
create the case entity, mark it as "importing", store the excel file
somewhere in a shared place for later use.

Then, on a dedicated repo instance, a looping task notices the
"importing" case, fetches the xl files, and starts the actual import
process, and possibly updates regularly some progress info about the
import process.

Same idea for delete, clone.

For the case export, things are more tricky, in part due to the huge
memory consumption of the excel workbook construction. A completely
different pb.

More information about the Cubicweb mailing list