<div dir="ltr"><div><div>Thanks a lot Dave, especially if you are currently in leave :-)! </div><div><br></div><div>1. This index should be Catalog of Life if I have understood well. Maybe, I should create a new name index (using the nameindexer tool) with the backbone taxonomy list from GBIF. </div><div><br></div><div>2. It works with others codes that contain whitespaces. The only difference that I can see between those codes are punctuation. </div><div><br></div><div>3. Sorry for my first explanation not really helpful ! :-). Actually, I was wrong, it's not a NFE.</div><div>The error takes place before the indexation itself, it happens when I try to create the data resource (using GBIF tool or directly by creating a dataresource and then uploading a ZIP file). </div><div>The DwC is downloaded and directly after, I got the error (see error track below). </div><div>I think that the error come from this function (<a href="https://github.com/AtlasOfLivingAustralia/collectory-plugin/blob/master/grails-app/services/au/org/ala/collectory/GbifService.groovy#L371">https://github.com/AtlasOfLivingAustralia/collectory-plugin/blob/master/grails-app/services/au/org/ala/collectory/GbifService.groovy#L371</a>) so I guess it is when the zip file is unzipped.</div></div><div><br></div><div><div>-----------------------------------------------------</div><div><br></div><div>2016-05-23 16:56:08,179 INFO [DataResourceController] Downloading file: <a href="http://api.gbif.org/v1/occurrence/download/request/0007506-160118175350007.zip">http://api.gbif.org/v1/occurrence/download/request/0007506-160118175350007.zip</a></div><div>2016-05-23 16:56:37,965 INFO [org.jasig.cas.services.DefaultServicesManagerImpl] - <Reloading registered services.></div><div>2016-05-23 16:56:37,976 DEBUG [org.jasig.cas.services.DefaultServicesManagerImpl] - <Adding registered service ^(https?|imaps?)://.*></div><div>2016-05-23 16:56:37,976 INFO [org.jasig.cas.services.DefaultServicesManagerImpl] - <Loaded 1 services.></div><div>2016-05-23 16:57:57,911 INFO [GbifService] dr172 null null</div><div>2016-05-23 16:57:58,155 ERROR [DataResourceController] JSONObject["guid"] not found.</div><div>org.codehaus.groovy.grails.web.json.JSONException: JSONObject["guid"] not found.</div><div><span class="" style="white-space:pre"> </span>at au.org.ala.collectory.GbifService.createOrUpdateGBIFResource(GbifService.groovy:324)</div><div><span class="" style="white-space:pre"> </span>at au.org.ala.collectory.GbifService.createGBIFResourceFromArchiveURL(GbifService.groovy:294)</div><div><span class="" style="white-space:pre"> </span>at au.org.ala.collectory.ProviderGroupController$_closure23.doCall(ProviderGroupController.groovy:557)</div><div><span class="" style="white-space:pre"> </span>at grails.plugin.cache.web.filter.PageFragmentCachingFilter.doFilter(PageFragmentCachingFilter.java:198)</div><div><span class="" style="white-space:pre"> </span>at grails.plugin.cache.web.filter.AbstractFilter.doFilter(AbstractFilter.java:63)</div><div><span class="" style="white-space:pre"> </span>at com.brandseye.cors.CorsFilter.doFilter(CorsFilter.java:82)</div><div><span class="" style="white-space:pre"> </span>at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)</div><div><span class="" style="white-space:pre"> </span>at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)</div><div><span class="" style="white-space:pre"> </span>at java.lang.Thread.run(Thread.java:745)</div></div><div><br></div><div><br></div><div>Cheers,</div><div>Marie</div><div><br></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, May 25, 2016 at 12:26 PM, <span dir="ltr"><<a href="mailto:David.Martin@csiro.au" target="_blank">David.Martin@csiro.au</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr" style="font-size:12pt;color:#000000;background-color:#ffffff;font-family:Calibri,Arial,Helvetica,sans-serif">
<p>Thanks Marie. Just quick answers (im currently on leave)<br>
</p>
<p><br>
</p>
<p>1. BIE isnt required, but there should be an index on the biocache service machine in the usual place (/data/lucence/namematching). This will then be used for taxon resolution.<br>
</p>
<p><br>
</p>
<p>2. Im surprised this causes an issue. Whitespace in those codes can be an issue.<br>
</p>
<p><br>
</p>
<p>3. Can you supply more detail ? A NPE would suggest a bug or bad config. The way we index large datasets is to use the offline method of indexing using the "bulk-processor" option in the command line tool. <br>
</p>
<p><br>
</p>
<p>Dave</p>
<p><br>
</p>
<div style="color:rgb(33,33,33)">
<hr style="display:inline-block;width:98%">
<div dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> Ala-portal <<a href="mailto:ala-portal-bounces@lists.gbif.org" target="_blank">ala-portal-bounces@lists.gbif.org</a>> on behalf of Marie Elise Lecoq <<a href="mailto:melecoq@gbif.fr" target="_blank">melecoq@gbif.fr</a>><br>
<b>Sent:</b> 25 May 2016 03:36<br>
<b>To:</b> <a href="mailto:ala-portal@lists.gbif.org" target="_blank">ala-portal@lists.gbif.org</a><br>
<b>Subject:</b> [Ala-portal] [Indexation] Questions</font>
<div> </div>
</div><div><div class="h5">
<div>
<div dir="ltr">
<div>Hi all !</div>
<div><br>
</div>
<div>I have few questions about the indexation :</div>
<div><br>
</div>
<div>1. It seems that some occurrences are wrongly indexed. For example, if I search "Pica Pica", the three first results will be not relevant (<a href="http://recherche.gbif.fr/occurrences/search?taxa=Pica+pica" target="_blank">http://recherche.gbif.fr/occurrences/search?taxa=Pica+pica</a>).
Do I need to change something on the nameindexer ? I don't have a BIE instance on our system, do I need to install one in order to help ? </div>
<div><br>
</div>
<div>2. We have some provider codes with punctuation (e.g. comma, dot ). It's seems that the link between collection, institution and dataresource is not made due to this. It works with accents. </div>
<div><br>
</div>
<div>3. I try to index a data resource with more than 20 million occurrences and I have a NullPointerException, it's seems that guid is not found. I can upload data resource with much less data inside so I guess the problem comme from the data resource itself
(size ?). Do you have a special way to deal with huge data resource ? </div>
<div><br>
</div>
<div>Thanks in advance for your help :-)!</div>
<div>Cheers,</div>
<div>Marie</div>
<br clear="all">
<div><br>
</div>
-- <br>
<div>
<div dir="ltr"><img src="https://mail.google.com/mail/u/0/?ui=2&ik=f2990c326c&view=fimg&th=143720bd12d267c4&attid=0.1&disp=inline&safe=1&attbid=ANGjdJ-dPPvdDXYTQLEz3sMkSC8MXmmlvNMhRcnZ-5COf76BRKDzNBzALARmrD-ZLTteeCriuqRYcwDCZxnWA6ZjOt8rVgydWnc6h2aRU_hfLDYFdDAPI0uUiC8Do9o&ats=1389188740078&rm=143720bd12d267c4&zw&sz=w1325-h522"><br>
</div>
</div>
</div>
</div>
</div></div></div>
</div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature"><div dir="ltr"><img src="https://mail.google.com/mail/u/0/?ui=2&ik=f2990c326c&view=fimg&th=143720bd12d267c4&attid=0.1&disp=inline&safe=1&attbid=ANGjdJ-dPPvdDXYTQLEz3sMkSC8MXmmlvNMhRcnZ-5COf76BRKDzNBzALARmrD-ZLTteeCriuqRYcwDCZxnWA6ZjOt8rVgydWnc6h2aRU_hfLDYFdDAPI0uUiC8Do9o&ats=1389188740078&rm=143720bd12d267c4&zw&sz=w1325-h522"><br></div></div>
</div></div>