[Cubicweb] Misunderstanding of dataimport _create_copyfrom_buffer function

Vincent Michel vincent.michel at logilab.fr
Tue May 20 12:26:08 CEST 2014


Hi Cyril,

The _create_copyfrom_buffer of cubicweb/dataimport.py is indeed a bit 
tricky...

This function creates a CSV string that will be pushed in postgresql 
using COPY FROM.
However, if the separator used in the CSV string already exists in some 
attributes, it will crash. Thus it is possible to give a replace_sep to 
replace the separator by this given string.

Indeed, the correct clause should be :

/if (replace_sep != None):

/

I have created a ticket on this issue (https://www.cubicweb.org/3845572),
and I think that the whole function should be rewritten.

Best,

Vincent




On 19/05/2014 14:24, LANQUETUIT Cyril wrote:
> Hello,
>
> In function _create_copyfrom_buffer of file dataimport.py, in the 
> following part of code:
> /
>                         # If a replace_sep is given, replace
>                         # the separator instead of returning None
>                         # (and thus avoid empty buffer)
>                         if replace_sep:
>                             value = value.replace(_char, replace_sep)
>                         else:
>                             return/
>
> when the separator is set to '' (the default value when MassiveObjectStore is used for store)
> the replace_sep is evaluate to false so the else clause is executed, the function _create_copyfrom_buffer return an empty buffer
> then this following error is raised:
>
> /File "/home/cl239099/cubicweb/cubes/dataio/dataimport.py", line 831, in flush_entities
>      raise ValueError('Error in buffer creation for etype %s' % etype)
> ValueError: Error in buffer creation for etype Question/
>
> I don't understand exactly what the code should do or should be but if the clause
> /
> if replace_sep:/
>
> is replaced by something like
> /
> if (replace_sep != None):
> /
> the else clause won't be executed in this case...
>
> please could you examinate this issue and correct it if needed or let me know what is wrong in the precedent discribed comportement/
> /
> Thank's,
> Cyril
>
>
> _______________________________________________
> Cubicweb mailing list
> Cubicweb at lists.cubicweb.org
> http://lists.cubicweb.org/mailman/listinfo/cubicweb

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.cubicweb.org/pipermail/cubicweb/attachments/20140520/1158e33b/attachment-0095.html>


More information about the Cubicweb mailing list