[Cubicweb] CubicWeb I/O

Dimitri Papadopoulos Orfanos dimitri.papadopoulos at cea.fr
Tue May 27 15:41:40 CEST 2014


Dear all,

I have already written in the past about I/O in CubicWeb. This is
especially important for us because we run lots of automated scripts
requesting, reading and writing data from the CubicWeb server, not
necessarily through the Web interface.

We have been suggested a few methods to access data:

1. HTTP/HTTPS remote access
  - has always worked and still works seamlessly
  - login from scripts might be a little bit tedious
  - limitation on length of HTTP request
  - poor performance on large datasets

2. ØMQ / Pyro + DB API
  - obsolete, do not write new code based on it
  - admin rights, security issue!

3. Run a script in the CubicWeb shell on a CubicWeb server, directly use
the session.
  - has always worked and still works seamlessly
  - admin rights
  - poor performance on large datasets

4. Run a script in the CubicWeb shell on a CubicWeb server, use
specialized stores to boost I/O for large datasets.
  - admin rights
  - lots of different classes:
    . RQLObjectStore         (cubicweb)
    . NoHookRQLObjectStore   (cubicweb)
    . SQLGenObjectStore      (cubicweb)
    . MassiveObjectStore     (dataio cube)
    . ?                      (fastimport cube)
  - chaotic releases without a roadmap
  - inconsistent APIs, classes are not interchangeable
  - bugs in some of the classes


I have a few questions/remarks:

* Am I missing any I/O method?

* Concerning item 4, I believe more formal planning would benefit all
CubicWeb users:
  - Share a "massive data import" roadmap with CubicWeb users.
  - Stick to a stable API that will remain unchanged for a "long time".
  - Maybe start a CWEP on the subject?

* Are there still plans for remote access other than HTTP/HTTPS?


Best,
-- 
Dimitri Papadopoulos
CEA/Saclay
I2BM, NeuroSpin
F-91191 Gif-sur-Yvette cedex, France



More information about the Cubicweb mailing list