<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
Hello David,
<div class=""><br class="">
</div>
<div class="">thanks for pointing out these faceting issues very clearly. At first glance I agree with all your points and the behaviour is awkward or simply buggy in some cases. We will need some time to investigate further.</div>
<div class=""><br class="">
</div>
<div class="">Many thanks,</div>
<div class="">Markus</div>
<div class=""><br class="">
<div>
<blockquote type="cite" class="">
<div class="">On 10 Feb 2017, at 22:24, Herbario SANT <<a href="mailto:sant.herbarium@gmail.com" class="">sant.herbarium@gmail.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div dir="ltr" class="">Hello.<br class="">
<br class="">
I am trying to play with faceted results from the occurrence api, but returned values are very odd IMHO.
<div class=""><br class="">
Perhaps I am misunderstanding how faceting should work? Or there might be some problem with the indexing of these particular datasets. </div>
<div class="">I am pretty lost. This is what I found:</div>
<div class=""><br class="">
<br class="">
<b class="">(1) RESULTS COUNT NOT MATCHING SUM OF ALL FACET COUNTS</b><br class="">
<br class="">
I put a simple example so everything is returned in one page.<br class="">
<br class="">
<a href="http://api.gbif.org/v1/occurrence/search?facet=ScientificName&limit=10&collectionCode=SANT-Lich&genusKey=2581943" class="">http://api.gbif.org/v1/occurrence/search?facet=ScientificName&limit=10&collectionCode=SANT-Lich&genusKey=2581943</a><br class="">
<br class="">
The count value is 4, the number of results is 4.<br class="">
But the number of facets is 1, and its count is 2. <br class="">
<br class="">
The faceted term ScientificName is a mandatory field, so no null values should happen. I would expect every occurrence having a value for it.<br class="">
And the number of values is short, so everything is returned in one request (no paging needed).
<br class="">
So, in such a case shouldn't the sum of facet counts be equal to the number of results?<br class="">
Why the count of the faceted name is not 4?<br class="">
<br class="">
<br class="">
<b class="">(2) LOWERCASE FACETS (facets values not matching results values):<br class="">
</b><br class="">
<div class="">Look at the same api request above (plant names)<br class="">
<br class="">
results: <br class="">
Scientificname: "Generic_name specific_name (Basionym_Authors) Name_Authors"<br class="">
<br class="">
facets: <br class="">
name: "generic_name specific_name (basionym_authors) name_authors"</div>
<div class=""><br class="">
Why are the facets names always in lowercase? <br class="">
I would say that is an error which shouldn't happen.<br class="">
<br class="">
But I reported it some days ago and got no answer, so I wonder if this is the intended api behaviour.<br class="">
<br class="">
<a href="http://dev.gbif.org/issues/browse/PF-2758" class="">http://dev.gbif.org/issues/browse/PF-2758</a><br class="">
<br class="">
Not only scientific names are lowercased. This also happens to collectionCode in the next question.<br class="">
<br class="">
<br class="">
<b class="">(3) FACETING COLLECTIONCODE VALUES of a single institution fails depending on filtering parameter used to match the institution (code or uuid):</b>
<br class="">
<br class="">
Our institution (uuid= def87a70-0837-11d9-acb2-b8a03c50a862 , institutionCode=SANT) serves datasets from 4 collections, which should sum up more than 100000 records.
<br class="">
<br class="">
Why do I get only 2 of our 4 datasets faceted in the following request, which uses our publishingOrg uuid? (uuid should be the preferred option to do this, as code might not be unique for our institution)<br class="">
<br class="">
<a href="http://api.gbif.org/v1/occurrence/search?facet=collectionCode&limit=0&publishingOrg=def87a70-0837-11d9-acb2-b8a03c50a862" class="">http://api.gbif.org/v1/occurrence/search?facet=collectionCode&limit=0&publishingOrg=def87a70-0837-11d9-acb2-b8a03c50a862</a><br class="">
<br class="">
Why do I got 4 of 4 if I filter the request using institutionCode instead? (fortunately, nobody else uses the same institutionCode yet, so numbers are correct)<br class="">
<br class="">
<a href="http://api.gbif.org/v1/occurrence/search?facet=collectionCode&limit=0&institutionCode=SANT" class="">http://api.gbif.org/v1/occurrence/search?facet=collectionCode&limit=0&institutionCode=SANT</a><br class="">
<br class="">
And why do counts differ for the same facet value (sant-lich) in those two requests?<br class="">
(9960 in the 1st request, 10007 in the 2nd one)<br class="">
<br class="">
Why are facet values lowercase again? ("sant-lich" instead of "SANT-Lich")<br class="">
<br class="">
<br class="">
<b class="">(4) FACETING SCIENTIFICNAME FAILS FOR SOME DATASETS, but works as expected for others:
</b><br class="">
<br class="">
More than 1000 faceted Scientificnames returned for our SANT-Lich and SANT-Algae collections. Both of them look correct results:<br class="">
<br class="">
<a href="http://api.gbif.org/v1/occurrence/search?facet=ScientificName&limit=0&collectionCode=SANT-Lich&ScientificName.facetLimit=50000&ScientificName.facetOffset=0" class="">http://api.gbif.org/v1/occurrence/search?facet=ScientificName&limit=0&collectionCode=SANT-Lich&ScientificName.facetLimit=50000&ScientificName.facetOffset=0</a><br class="">
<br class="">
<a href="http://api.gbif.org/v1/occurrence/search?facet=ScientificName&limit=0&collectionCode=SANT-Algae&ScientificName.facetLimit=50000&ScientificName.facetOffset=0" class="">http://api.gbif.org/v1/occurrence/search?facet=ScientificName&limit=0&collectionCode=SANT-Algae&ScientificName.facetLimit=50000&ScientificName.facetOffset=0</a><br class="">
<br class="">
But no facets returned for SANT-Bryo (which contains several hundred distinct scientificname values):</div>
<div class=""><br class="">
<a href="http://api.gbif.org/v1/occurrence/search?facet=ScientificName&limit=0&collectionCode=SANT-Bryo&ScientificName.facetLimit=50000&ScientificName.facetOffset=0" class="">http://api.gbif.org/v1/occurrence/search?facet=ScientificName&limit=0&collectionCode=SANT-Bryo&ScientificName.facetLimit=50000&ScientificName.facetOffset=0</a><br class="">
<br class="">
And only 7 facets for SANT scientificnames (should be over 10 thousand, as this is by far our largest dataset):</div>
<div class=""><br class="">
<a href="http://api.gbif.org/v1/occurrence/search?facet=ScientificName&limit=0&collectionCode=SANT&ScientificName.facetLimit=50000&ScientificName.facetOffset=0" class="">http://api.gbif.org/v1/occurrence/search?facet=ScientificName&limit=0&collectionCode=SANT&ScientificName.facetLimit=50000&ScientificName.facetOffset=0</a><br class="">
<br class="">
<br class="">
<br class="">
Other than the lowercase facets issue (2), I couldn't reproduce issues 1,3,4 in other institutions datasets. </div>
<div class="">So I wonder if all this is somehow related to a wrong indexing of our IPT.<br class="">
<br class="">
Has anyone else detected these problems?<br class="">
<br class="">
Thanks a lot in advance for your help</div>
<div class=""><br class="">
</div>
<div class="">David</div>
<div class=""><br class="">
</div>
<div class="">-- <br class="">
</div>
<div class="">David García San León<br class="">
Herbario SANT<br class="">
Universidade de Santiago de Compostela<br class="">
<br class="">
</div>
</div>
</div>
_______________________________________________<br class="">
API-users mailing list<br class="">
<a href="mailto:API-users@lists.gbif.org" class="">API-users@lists.gbif.org</a><br class="">
http://lists.gbif.org/mailman/listinfo/api-users<br class="">
</div>
</blockquote>
</div>
<br class="">
</div>
</body>
</html>