[IPT] Publication of big dataset fails, can't find why

Laura Russell larussell at vertnet.org
Wed Mar 2 16:10:01 CET 2016

I¹ve had publishing fail before when there was a line break (or maybe a
vertical tab, can¹t remember which) in a remarks field that then made a new
line causing that new line to have a null occurrenceID.  Any chance
something like that could be causing the failure?

Laura Russell
VertNet Programmer/iDigBio Data Mobilization Specialist

phone: +01 785 813-1496
email: larussell at vertnet.org
Skype: laura.anne.russell
Hangouts: larussell at vertnet.org

url: www.vertnet.org
url: www.idigbio.org

From:  IPT <ipt-bounces at lists.gbif.org> on behalf of Peter Desmet
<peter.desmet at inbo.be>
Date:  Wednesday, March 2, 2016 at 8:59 AM
To:  <ipt at lists.gbif.org>
Cc:  Stijn Van Hoey <stijn.vanhoey at inbo.be>
Subject:  [IPT] Publication of big dataset fails, can't find why


We're trying to publish version 45.4 of this dataset:
http://data.inbo.be/ipt/resource?r=florabank1-occurrences, but the
validation seems to fail. Here's the publication log:

As far as I can tell, the validation fails on "the core ID field
occurrenceID is always present and unique", but we have verified this
in the generated dwca-45.4.zip file, and all records have a unique

Any idea what might be going on? Possible causes:

1. The dataset is quite big (3,5 million records)
2. We've just solved this issue:
https://github.com/LifeWatchINBO/data-publication/issues/104 by
following Kyle Braak's instructions. The latest published version is
now 45.3, the current (to be published) version is 45.4, so everything
seems fine there.
3. Even though the publication failed, the following files are created
in /resource/florabank1-occurrences:


Will those file be overwritten if I try to republish or might they be
causing the publication to fail?


IPT mailing list
IPT at lists.gbif.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gbif.org/pipermail/ipt/attachments/20160302/a01667e1/attachment.html>

More information about the IPT mailing list