[Cubicweb] Dataimport

Rémi Cardona remi.cardona at logilab.fr
Fri Dec 4 14:44:15 CET 2015

Le 04/12/2015 13:44, Antoine Grigis a écrit :
> I noticed a change in the 'dataimport' module between CubicWeb (CW)
> 3.19.6 and CW 3.20.9.
> The '_create_copyfrom_buffer' function has been refactored and the
> default return case removed.
> In the former version, the function default returned code was 'None'.
> This is catched in both CW versions by the '_execmany_thread_copy_from'
> function in order to execute thread without 'copy from' tabular data
> speedup.
> Thus in CW 3.20.9 inserting a 'Binray' entity field through the
> 'SQLGenObjectStore' raises an Exception.
> Is it a normal behavior? Is it still possible to insert a 'Binary' field
> using a store and a schema with inlined relations?

Hi Antoine,

The dataimport module was made stricter in CubicWeb 3.20 and newer 
releases. Its handling of binary objects is one such case.

Do note that the current code (in 3.20 and all newer releases) relies on 
pyscopg2's copy_from() method [1] which only supports PostgreSQL's 
"text" format [2].

AIUI, this "text" format cannot be used to deal with binary data, though 
we haven't given it much thought.

If dealing with binary is possible, please let us know how. Patches will 
be greatly appreciated (even in an unfinished state).



[1] http://initd.org/psycopg/docs/usage.html#using-copy-to-and-copy-from
[2] http://www.postgresql.org/docs/9.4/static/sql-copy.html#AEN71994

Rémi Cardona - LOGILAB, Paris (France).
Formations - http://www.logilab.fr/formations
Développements - http://www.logilab.fr/services
Gestion de connaissances - http://www.cubicweb.org/

More information about the Cubicweb mailing list