<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class="">Dear Quentin, </div><div class=""><br class=""></div><div class="">Thank you for your feedback. </div><div class=""><br class=""></div><div class="">I answer your questions inline below.</div><div class=""><br class=""></div><div class="">Best regards,</div><div class=""><br class=""></div><div class="">Kyle</div><br class=""><div><blockquote type="cite" class=""><div class="">On 11 Nov 2015, at 17:10, Quentin Groom <<a href="mailto:quentin.groom@plantentuinmeise.be" class="">quentin.groom@plantentuinmeise.be</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class="">Having now experimented a little.<div class=""><br class=""></div><div class="">I've had most success with a completely flat file</div><div class=""><br class=""></div><div class="">The validator works OK except for the following 3 errors, which are presumably due to the recent changes to Darwin Core.</div><div class=""><ul style="margin:0px;padding:0px;list-style:none;color:rgb(102,102,102);font-family:arial,helvetica,verdana,sans-serif;font-size:12px;line-height:13.2px" class=""><li style="padding:5px 0px 0px" class=""><span class="" style="color:red">Unknown term</span> <pre style="display:inline" class=""><a href="http://rs.tdwg.org/dwc/terms/organismQuantity" class="">http://rs.tdwg.org/dwc/terms/organismQuantity</a></pre> mapped to column 9</li><li style="padding:5px 0px 0px" class=""><span class="" style="color:red">Unknown term</span> <pre style="display:inline" class=""><a href="http://purl.org/dc/terms/license" class="">http://purl.org/dc/terms/license</a></pre> mapped to column 16</li><li style="padding:5px 0px 0px" class=""><span class="" style="color:red">Unknown term</span> <pre style="display:inline" class=""><a href="http://rs.tdwg.org/dwc/terms/organismQuantityType" class="">http://rs.tdwg.org/dwc/terms/organismQuantityType</a></pre> mapped to column 8</li></ul></div></div></div></blockquote><div><br class=""></div><div>Exactly right. The validator needs to be updated to work with the latest extension versions, which use the latest Darwin Core terms. </div><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class=""><div class=""><font color="#666666" face="arial, helvetica, verdana, sans-serif" class=""><span style="font-size:12px;line-height:13.2px" class=""><br class=""></span></font></div></div><div class=""><font face="arial, helvetica, verdana, sans-serif" class=""><span style="font-size:12px;line-height:13.2px" class=""><font color="#666666" class="">The IPT also seems to accept a flat file and I can get all the fields mapped. However, I'm not clear why it shows zero records in this summary. </font><font color="#ff0000" class="">A problem???</font></span></font></div><div class=""><font color="#666666" face="arial, helvetica, verdana, sans-serif" class=""><span style="font-size:12px;line-height:13.2px" class=""><span id="cid:ii_150f7149778137bf"><image.png></span><br class=""></span></font></div></div></div></blockquote><div><br class=""></div><div>I strongly suspect you are looking at a ‘preview’ page of the unpublished version 1.0. After you publish version 1.0, try viewing its homepage and the number of (core) records will show as you’d expect. </div><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class=""><font color="#666666" face="arial, helvetica, verdana, sans-serif" class=""><span style="font-size:12px;line-height:13.2px" class=""><br class=""></span></font></div><div class=""><font color="#666666" face="arial, helvetica, verdana, sans-serif" class=""><span style="font-size:12px;line-height:13.2px" class="">Using, Event as the core file and occurrence as an extension the validator works OK, but it does create an error "The extension data file contains references to core IDs that do not exist:", but I think this is something to do with where it assumes the core IDs are in the file (i.e. not in the 1st column).</span></font></div></div></div></blockquote><div><br class=""></div><div>The ID field links records from the two sources together. In this case, each occurrence record should link to an event record via its eventID. You can refer to the IPT User Manual's section on mapping [1] for more information. If you get stuck, feel free to write to me for direct assistance. </div><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class=""><font color="#666666" face="arial, helvetica, verdana, sans-serif" class=""><span style="font-size:12px;line-height:13.2px" class=""><br class=""></span></font></div><div class=""><font color="#666666" face="arial, helvetica, verdana, sans-serif" class=""><span style="font-size:12px;line-height:13.2px" class="">I think I've also got it working in the IPT, though I'm not yet seeing the benefit of the star format over the flat format. Are the benefits of the Star Schema format only related to the size of the DWC-A, or are there another benefits. If it does just relate to the size then I think it would be best to recommend the flat file format for all but the big users.</span></font></div></div></div></blockquote><div><br class=""></div><div>There are other benefits. The Star Schema is conceptually cleaner, easier to maintain, less prone to mistakes, etc. </div><div><br class=""></div><div>Please note that you can capture additional measurements or facts using the MeasurementOrFact extension [2]. Using Occurrence as the core it’s only appropriate to capture measurements or fact relating to the species occurrences whereas using Event as the core, you can capture measurements or facts relating to the sampling event (e.g., environmental
measurements like sediment temperature and redox potential (Eh)).</div><div><br class=""></div><div>[1] <a href="https://github.com/gbif/ipt/wiki/IPT2ManualNotes.wiki#darwin-core-mappings" class="">https://github.com/gbif/ipt/wiki/IPT2ManualNotes.wiki#darwin-core-mappings</a></div><div>[2] <a href="http://rs.gbif.org/extension/dwc/measurements_or_facts.xml" class="">http://rs.gbif.org/extension/dwc/measurements_or_facts.xml</a></div><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class=""><font color="#666666" face="arial, helvetica, verdana, sans-serif" class=""><span style="font-size:12px;line-height:13.2px" class="">Regards</span></font></div><div class=""><font color="#666666" face="arial, helvetica, verdana, sans-serif" class=""><span style="font-size:12px;line-height:13.2px" class="">Quentin</span></font></div><div class=""><font color="#666666" face="arial, helvetica, verdana, sans-serif" class=""><span style="font-size:12px;line-height:13.2px" class=""><br class=""></span></font></div><div class=""><font color="#666666" face="arial, helvetica, verdana, sans-serif" class=""><span style="font-size:12px;line-height:13.2px" class=""><br class=""></span></font></div></div><div class="gmail_extra"><br clear="all" class=""><div class=""><div class="gmail_signature"><div dir="ltr" class=""><div class=""><div dir="ltr" class=""><div class=""><br class=""></div><div class=""><br class=""></div><div class="">Dr. Quentin Groom</div><div class="">(Botany and Information Technology)</div><div class=""><br class=""></div><div class="">Botanic Garden Meise</div><div class="">Domein van Bouchout</div><div class="">B-1860 Meise</div><div class="">Belgium</div><div class=""><br class=""></div><div class="">ORCID: 0000-0002-0596-5376</div><div class=""><br class=""></div><div class="">Landline; +32 (0) 226 009 20 ext. 364</div><div class="">FAX: +32 (0) 226 009 45</div><div class=""><br class=""></div><div class="">E-mail: <a href="mailto:quentin.groom@plantentuinmeise.be" target="_blank" class="">quentin.groom@plantentuinmeise.be</a></div><div class="">Skype name: qgroom</div><div class="">Website: <a href="http://www.botanicgarden.be/" target="_blank" class="">www.botanicgarden.be</a></div><div class=""><br class=""></div></div></div></div></div></div>
<br class=""><div class="gmail_quote">On 11 November 2015 at 14:15, Tim Robertson <span dir="ltr" class=""><<a href="mailto:trobertson@gbif.org" target="_blank" class="">trobertson@gbif.org</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class="">Thanks Quentin<div class=""><br class=""></div><div class=""><div class=""><span class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class="">So if I understand correctly, the event file can be used as a Core, just as Taxon and Occurrence can be Core files. </div></div></blockquote><div class=""><br class=""></div></span><div class="">Yes, that’s correct</div><span class=""><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class="">Though as there can only be one Core ID I will still need to keep my taxon information in the Occurrence file.<div class="">Although I don't think this is a problem, it can get a little confusing in the documentation due to the crossover of terms between taxon and occurrence files.</div></div></div></blockquote><div class=""><br class=""></div></span><div class="">I’m afraid that is the kind of limitation I was eluding to about star schemas… You have to denormalise things into a format which flattens what you might otherwise model as 2 tables.</div><div class=""><br class=""></div><div class="">Currently a "Taxon” can’t be used as an extension, so you would need to use Occurrence. Adding Taxon as an option would be technically possible, but that would be completely decoupled from the occurrences. It would however allow you to have:</div><div class=""><br class=""></div><div class="">Core: Rows of Sampling event documenting e.g. a square on the ground sample on a specific period</div><div class=""> Extension taxon: List of species observed within the sampling event</div><div class=""> Extension occurrence: Documented evidence of specimens collected or observed</div><div class=""><br class=""></div><div class="">At the moment though, you would have to express species lists as occurrences, which might make some sense because they are effectively observations.</div><span class=""><br class=""><blockquote type="cite" class=""><div dir="ltr" class=""><div class=""><div class="">I'm happy to be a Guinea pig. I'll experiment with the validator if you think this should work and let you know how I get on.</div></div></div></blockquote><div class=""><br class=""></div></span>Thanks for this,</div><div class=""><br class=""></div><div class="">All the best,</div><div class="">Tim</div><div class=""><div class="h5"><div class=""><br class=""><blockquote type="cite" class=""><div dir="ltr" class=""><div class=""><div class=""><br class=""></div><div class="">Regards</div><div class="">Quentin</div></div></div><div class="gmail_extra"><br clear="all" class=""><div class=""><div class=""><div dir="ltr" class=""><div class=""><div dir="ltr" class=""><div class=""><br class=""></div><div class=""><br class=""></div><div class="">Dr. Quentin Groom</div><div class="">(Botany and Information Technology)</div><div class=""><br class=""></div><div class="">Botanic Garden Meise</div><div class="">Domein van Bouchout</div><div class="">B-1860 Meise</div><div class="">Belgium</div><div class=""><br class=""></div><div class="">ORCID: 0000-0002-0596-5376</div><div class=""><br class=""></div><div class="">Landline; +32 (0) 226 009 20 ext. 364</div><div class="">FAX: +32 (0) 226 009 45</div><div class=""><br class=""></div><div class="">E-mail: <a href="mailto:quentin.groom@plantentuinmeise.be" target="_blank" class="">quentin.groom@plantentuinmeise.be</a></div><div class="">Skype name: qgroom</div><div class="">Website: <a href="http://www.botanicgarden.be/" target="_blank" class="">www.botanicgarden.be</a></div><div class=""><br class=""></div></div></div></div></div></div>
<br class=""><div class="gmail_quote">On 11 November 2015 at 11:58, Hannu Saarenmaa <span dir="ltr" class=""><<a href="mailto:hannu.saarenmaa@helsinki.fi" target="_blank" class="">hannu.saarenmaa@helsinki.fi</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><u class=""></u>
<div text="#000000" bgcolor="#ffffff" class="">
Quentin & Co<br class="">
<br class="">
It depends what you mean by "survey". I would put each visit to a
sampling location (such as a plot) in the event core, and put all
the taxa that are observed in a non-core table. The properties of
the entire survey (project) would go to the EML metadata.<br class="">
<br class="">
Hannu<div class=""><div class=""><br class="">
<br class="">
On 2015-11-11 10:20, Quentin Groom wrote:
</div></div><blockquote type="cite" class=""><div class=""><div class="">
<div dir="ltr" class="">I'm rather confused how the Darwin Core Star Schema
is meant to work for survey data.
<div class=""><br class="">
</div>
<div class="">Darwin Core can have one of two Core files, taxon or
occurrence. The most appropriate for a survey would seem to
be occurrence. So I imagine that in the star schema you could
also have a related event file detailing the date and location
of each survey and a non-core taxon file detailing the taxa
that are observed.</div>
<div class=""><br class="">
</div>
<div class="">However, this does not seem to be possible. The DWC-A
validator (<a href="http://tools.gbif.org/dwca-validator/" target="_blank" class="">http://tools.gbif.org/dwca-validator/</a>),
assumes only on core id in the core file so you can't link an
occurrence both to a taxon and to an event. This is also true
in the Darwin Core Archive Assistant (<a href="http://tools.gbif.org/dwca-assistant/" target="_blank" class="">http://tools.gbif.org/dwca-assistant/</a>).
The solution seems to be to put all the information from the
taxon core file into the occurrence file, but keep the
separate event file linked with the core occurrence id.</div>
<div class=""><br class="">
</div>
<div class="">Is this correct? It seems rather counter intuitive.</div>
<div class=""><br class="">
</div>
<div class="">Regards</div>
<div class="">Quentin<br clear="all" class="">
<div class="">
<div class="">
<div dir="ltr" class="">
<div class="">
<div dir="ltr" class="">
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class="">Dr. Quentin Groom</div>
<div class="">(Botany and Information Technology)</div>
<div class=""><br class="">
</div>
<div class="">Botanic Garden Meise</div>
<div class="">Domein van Bouchout</div>
<div class="">B-1860 Meise</div>
<div class="">Belgium</div>
<div class=""><br class="">
</div>
<div class="">ORCID: 0000-0002-0596-5376</div>
<div class=""><br class="">
</div>
<div class="">Landline; +32 (0) 226 009 20 ext. 364</div>
<div class="">FAX: +32 (0) 226 009 45</div>
<div class=""><br class="">
</div>
<div class="">E-mail: <a href="mailto:quentin.groom@plantentuinmeise.be" target="_blank" class="">quentin.groom@plantentuinmeise.be</a></div>
<div class="">Skype name: qgroom</div>
<div class="">Website: <a href="http://www.botanicgarden.be/" target="_blank" class="">www.botanicgarden.be</a></div>
<div class=""><br class="">
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div></div><pre class=""><fieldset class=""></fieldset>
_______________________________________________
IPT mailing list
<a href="mailto:IPT@lists.gbif.org" target="_blank" class="">IPT@lists.gbif.org</a>
<a href="http://lists.gbif.org/mailman/listinfo/ipt" target="_blank" class="">http://lists.gbif.org/mailman/listinfo/ipt</a>
</pre>
</blockquote>
<br class="">
<pre cols="72" class="">--
Hannu Saarenmaa, Research Director
<a href="mailto:hannu.saarenmaa@uef.fi" target="_blank" class="">hannu.saarenmaa@uef.fi</a>
Mobile <a href="tel:%2B358-50-4479668" value="+358504479668" target="_blank" class="">+358-50-4479668</a>
University of Eastern Finland
Digitarium, SIB Labs, Joensuu Science Park
Länsikatu 15 (P.O. Box 111)
FI-80101 Joensuu
<a href="http://www.digitarium.fi/en" target="_blank" class="">www.digitarium.fi/en</a> - Service Centre for High-Performance Digitisation
<a href="http://www.eubon.eu/" target="_blank" class="">www.eubon.eu</a> - EU BON - GEO BON - Data Integration and Interoperability
</pre>
</div>
</blockquote></div><br class=""></div>
</blockquote></div><br class=""></div></div></div></div></blockquote></div><br class=""></div>
_______________________________________________<br class="">IPT mailing list<br class=""><a href="mailto:IPT@lists.gbif.org" class="">IPT@lists.gbif.org</a><br class="">http://lists.gbif.org/mailman/listinfo/ipt<br class=""></div></blockquote></div><br class=""></body></html>