[Cubicweb] datafeed
David Douard
david.douard at logilab.fr
Mon Dec 16 09:10:29 CET 2013
On 04/12/2013 09:36, Dimitri Papadopoulos Orfanos wrote:
> I've found a few words on datafeed in the Logilab blog about CubicWeb 3.11:
> http://www.cubicweb.org/blogentry/1512775
>
> A new 'datafeed' source was introduced, inspired by the
> soon to be deprecated datafeed cube. It needs polishing
> but sets the foundation for advanced semantic web
> applications that import content from others site using
> simple http request.
>
> A 'datafeed' source is associated to a parser that
> analyses the imported data and then creates/updates
> entities accordingly. There is currently a single parser
> in the core that imports CubicWeb-generated xml and needs
> to be configured with a mapping information that defines
> how relations are to be followed. It provides a viable
> alternative to 'pyrorql' sources. Other parsers to import
> RDF, RSS, etc should come soon.
>
> From what I gather datafeed can help import data from structured Web
> sites. It doesn't help importing data from files (we have to parse at
> least CSV, XML, DICOM files), does it?
Not really, datafeed really is about importing (part of) an external
database in a CW instance. datafeed can update entities as long as it
can identify a unique identifier for that entity in the remote database
(typically an URI).
> In our application we repeatedly scan a directory for new files and data
> appended to existing files. Can datafeed help in that case? I'm thinking
> here about the "creates/updates entities accordingly" part of datafeed.
At update time, it needs to be able to ask the objects in the remote
database that have been created/modified since last synchronization. All
these stuff you won't (easily) have from files. What you need is a set
of (smart) importation scripts.
Maybe Vincent can help you here.
David
> Le 04/12/2013 09:25, Dimitri Papadopoulos Orfanos a écrit :
>> Dear all,
>>
>> I am interested in datafeed, a recent addition to the core CubicWeb
>> libraries, because we are assigned the task to import data from
>> different sources into our database.
>>
>> There is not much information on datafeed except for the source code. I
>> wonder whether any of you CubicWeb wizards could:
>> * explain the purpose and benefits of datafeed in one or two sentences,
>> * give a few hints on the usage of datafeed.
>>
>
--
David DOUARD LOGILAB
Directeur du département Outils & Systèmes
+33 1 45 32 03 12 david.douard at logilab.fr
+33 1 83 64 25 26 http://www.logilab.fr/id/david.douard
Formations - http://www.logilab.fr/formations
Développements - http://www.logilab.fr/services
Gestion de connaissances - http://www.cubicweb.org/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: david_douard.vcf
Type: text/x-vcard
Size: 302 bytes
Desc: not available
URL: <http://lists.cubicweb.org/pipermail/cubicweb/attachments/20131216/ba2a75ef/attachment-0273.vcf>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 263 bytes
Desc: OpenPGP digital signature
URL: <http://lists.cubicweb.org/pipermail/cubicweb/attachments/20131216/ba2a75ef/attachment-0271.sig>
More information about the Cubicweb
mailing list