- API-users - lists.gbif.org

Using GBIF api for a global insect family website
by Bastiaan Wakkie 14 Jun '17

14 Jun '17

Dear all, >From 1999 onwards I have created and maintained a insect family website about Syrphidae ( http://www.syrphidae.com) Main goal is a centralised meeting point for Syrpidologists around the world. My plan is to use the gbif backbone in the future but have no idea where to start. Basically I used national checklists to update the data in the pastbut never reached the whole world (Europe and Africa where partly done). What is the best option for me? I know in the past there was the gibf-npt which was very suitable for my case but was discontinued. Are there examples I can reusefor my purpose? Would it be best to use only api calls all over my website ot receive speciedata and occurences? kind regards, Bastiaan Wakkie

4 7

Job opening at the GBIF Secretariat
by Federico Mendez 02 Jun '17

02 Jun '17

Hi everyone, the GBIF Secretariat is currently seeking applications for a Software Developer, see complete details here: http://www.gbif.org/newsroom/news/gbif-seeks-software-developer<http://www.gbif.org/newsroom/news/web-developer-job-2017> Deadline for applications is Monday, 17 July 2017. The selected candidate is expected to live in Denmark, and a relocation package is available for internationally recruited staff. Please feel free to share this position with your own networks and contacts. Thanks, Federico Mendez fmendez(a)gbif.org<mailto:fmendez@gbif.org> skype: fede.mendez Lead Software Developer

1 0

Download request content type mismatch
by Doug Palmer 11 May '17

11 May '17

Hi, I've been have some troubles using the groovy http builder to queue a download. I've eventually realised that what is happening is that the GBIF API is returning a download key as a simple text string but setting the content type to application/json. This causes difficulties with the http builder, since by default it tries to be helpful and parse the result as JSON. It then throws an exception, since it tries to interpret something like 0065397-160910150852091 as a number. (The /occurrence/count path also just returns a count, but this is correctly interpreted as just being an integer.) I've worked around this problem, but could future versions of the API always return valid JSON for application/json content? Doug

1 0

Queueing download requests?
by Scott Chamberlain 10 May '17

10 May '17

Hi, Recently was reading this issue http://dev.gbif.org/issues/browse/POR-2734?jql=text%20~%20%22420%20http%22 and wondering if GBIF might be willing to queue requests, and just fire them off when they're allowed to run, instead of the user doing 1 too many then getting 420. I have no knowledge of GBIF backend, so this may be ridiculous. In the meantime I can add docs to the two clients for how to programmatically send of download requests, and occasionally poll until done, then fire off new ones, etc. Best, Scott

1 0

Mismatch in species count
by Srini Anand 07 Apr '17

07 Apr '17

Hello, I'm a student from Indiana University, working towards a Masters in Data Science. As part of a class project, we are working with the team from Global Biointeractions (GloBI) to compare how much their data covers the known species list. To that end we are using GBIF as the reference system. I have worked out the details of navigating your site and extracting information for us to compare against. However, I'm unable to reconcile counts between the download and what I see on the species page. *Mismatch in species count* Using Kingdom = Archaea and Phylum = Crenarchaeota <http://www.gbif.org/species/79>, I navigated to the Occurrences page and downloaded the 3,824 occurrences. When I extract unique Species from this list, there are 59, whereas the main page indicates there should be 68. I see a similar issue with a) Kingdom = Archaea (should be 523, see 485) b) Class = Pinopsida (should be 2,366 species, but see 983) There is probably a small and obvious mistake I'm making, but not sure I know how to identify that. Any guidance would be must appreciated. Regards, *Srini Anand* Masters of Data Science student srianand(a)indiana.edu

2 2

Downloads API predicate format
by Bart Aelterman 28 Mar '17

28 Mar '17

Hi all, I am trying to create a downloads file using the GBIF Occurrence Download API, but I'm not getting the expected result. I want to pass in a dataset key from which I want all occurrences. The predicate I defined is this one: { "creator": "bartaelterman", "notification_address": ["bart.aelterman(a)gmail.com"], "predicate": { "value": "7d5a76b9-819f-4fb8-aab0-664d4089aa54", "key": "DATASET_KEY", "type": "equals" } } I've also set the following headers: Content-Type:'application/json, Accept: application/json Yet, I get a 500 response from the API. Am I doing something wrong in my predicate definition? Thanks, Bart

2 2

Job opening at the GBIF Secretariat
by Federico Mendez 02 Mar '17

02 Mar '17

Hi everyone, A quick announcement, the GBIF Secretariat is currently seeking applications for a Web Developer, see complete details here: http://www.gbif.org/newsroom/news/web-developer-job-2017 Deadline for applications is Monday, 20 March 2017. Please be aware that the successful candidate will be locally recruited (no-relocation involved), according to the GBIF Staff Rules Please feel free to share this position with your own networks and contacts. Thanks, Federico Mendez fmendez(a)gbif.org skype: fede.mendez Lead Software Developer

1 0

API for batch calling GBIF id -> checklist ID
by Yan Wong 22 Feb '17

22 Feb '17

Hi I have a large number of GBIF ids (~ 2 million), for which I wish to know any existing International Plant Names Index IDs (IPNI s). What’s the easiest way to do this? I can use the standard API, e.g. http://api.gbif.org/v1/species/2705290/related for 2705290 = Avena sativa, which gives me an output that contains several IPNIs (in this case, 164949-3, 391732-1, 27200-2, of which the number 391732-1 seems to be the canonical one, although I can’t figure out how to identify it as such from the GBIF API output) It will obviously be tedious (and unwarranted) to make 2 million API calls. What’s the recommended way to do this? Yan

3 3

how to check last date-time dataset was harvested?
by Herbario SANT 19 Feb '17

19 Feb '17

Hi again. Is there any way to use GBIF portal API to know the last date & time a certain resource was harvested by GBIF? (basically, to know if it has the latest info provided by the original source, or it could be outdated) I suspect the info in this url but I don' know how to interpretate them all: http://api.gbif.org/v1/dataset/df3eab30-0837-11d9-acb2-b8a03c50a862 I guess this is the IPT updated time: "modified": "2017-02-16T22:07:41.127+0000" ¿Perhaps this is what I am looking for? "machineTags":[{..."name":"crawl_attempt",..."created":"2017-01-05T12:07:12.525+0000"},...] Thanks a lot David -- David García San León Herbario SANT Universidade de Santiago de Compostela

2 2

Re: [API-users] Is there any NEO4J or graph-based driver for this API ?
by Collins, Matthew 15 Feb '17

15 Feb '17

Jorrit pointed out this thread to us at iDigBio. Downloading and importing data into a relational database will work great, especially if as Jan said you can cut the data size down to a reasonable amount. Another approach we've been working on in a collaboration called GUODA [1] is to build an Apache Spark environment with pre-formatted data frames with common data sets in them for researchers to use. This approach would offer a remote service where you could write arbitrary Spark code, probably in Jupyter notebooks, to iterate over data. Spark does a lot of cool stuff including GraphX which might be of interest. This is definitely pre-alpha at this point and if anyone is interested, I'd like to hear your thoughts. I'll also be at SPNHC talking about this. One thing we've found in working on this is that importing data into a structured data format isn't always easy. If you only want a few columns, it'll be fine. But getting the data typing, format standardization, and column name syntax of the whole width of an iDigBio record right requires some code. I looked to see if EcoData Retriever [2] had a GBIF data source and they have an eBird one that perhaps you might find useful as a starting point if you wanted to try to use someone else's code to download and import data. For other data structures like BHL, we're kind of making stuff up since we're packaging a relational structure and not something nearly as flat as GBIF and DWC stuff. [1] http://guoda.bio/ [2] http://www.ecodataretriever.org/ Matthew Collins Technical Operations Manager Advanced Computing and Information Systems Lab, ECE University of Florida 352-392-5414<callto:352-392-5414> ________________________________ From: jorrit poelen <jhpoelen(a)xs4all.nl> Sent: Monday, May 30, 2016 11:16 AM To: Collins, Matthew; Thompson, Alexander M; Hammock, Jennifer Subject: Fwd: [API-users] Is there any NEO4J or graph-based driver for this API ? Hey y’all: Interesting request below on the GBIF mailing list - sounds like a perfect fit for the GUODA use cases. Would it be too early to jump onto this thread and share our efforts/vision? thx, -jorrit Begin forwarded message: From: Jan Legind <jlegind(a)gbif.org<mailto:jlegind@gbif.org>> Subject: Re: [API-users] Is there any NEO4J or graph-based driver for this API ? Date: May 30, 2016 at 5:48:51 AM PDT To: Mauro Cavalcanti <maurobio(a)gmail.com<mailto:maurobio@gmail.com>>, "Juan M. Escamilla Molgora" <j.escamillamolgora(a)lancaster.ac.uk<mailto:j.escamillamolgora@lancaster.ac.uk>> Cc: "api-users(a)lists.gbif.org<mailto:api-users@lists.gbif.org>" <api-users(a)lists.gbif.org<mailto:api-users@lists.gbif.org>> Dear Juan, Unfortunately we have no tool for creating these kind of SQL like queries to the portal. I am sure you are aware that the filters in the occurrence search pages can be applied in combination in numerous ways. The API can go even further in this regard[1], but it not well suited for retrieving occurrence records since there is a 200.000 records ceiling making it unfit for species exceeding this number. There is going be updates to the pygbif package[2] in the near future that will enable you to launch user downloads programmatically where a whole list of different species can be used as a query parameter as well as adding polygons.[3] In the meantime, Mauro’s suggestion is excellent. If you can narrow your search down until it returns a manageable download (say less than 100 million records), importing this into a database should be doable. From there, you can refine using SQL queries. Best, Jan K. Legind, GBIF Data manager [1] http://www.gbif.org/developer/occurrence#search [2] https://github.com/sckott/pygbif [3] https://github.com/jlegind/GBIF-downloads From: API-users [mailto:api-users-bounces@lists.gbif.org] On Behalf Of Mauro Cavalcanti Sent: 30. maj 2016 14:06 To: Juan M. Escamilla Molgora Cc: api-users(a)lists.gbif.org<mailto:api-users@lists.gbif.org> Subject: Re: [API-users] Is there any NEO4J or graph-based driver for this API ? Hi, One solution I have successfully adopted for this is to download the records (either "manually" via browser or, yet better, using a Python script using the fine pygbif library), storing them into a MySQL or SQLite database and then perform the relational queries. I can provide examples if you are interested. Best regards, 2016-05-30 8:59 GMT-03:00 Juan M. Escamilla Molgora <j.escamillamolgora(a)lancaster.ac.uk<mailto:j.escamillamolgora@lancaster.ac.uk>>: Hola, Is there any API for making relational queries like taxonomy, location or timestamp? Thank you and best wishes Juan _______________________________________________ API-users mailing list API-users(a)lists.gbif.org<mailto:API-users@lists.gbif.org> http://lists.gbif.org/mailman/listinfo/api-users -- Dr. Mauro J. Cavalcanti E-mail: maurobio(a)gmail.com<mailto:maurobio@gmail.com> Web: http://sites.google.com/site/maurobio _______________________________________________ API-users mailing list API-users(a)lists.gbif.org<mailto:API-users@lists.gbif.org> http://lists.gbif.org/mailman/listinfo/api-users

5 11