Subject: | Re: pygbif for occurrence coordinates |
---|---|
Date: | Thu, 2 Jun 2016 09:00:49 -0300 |
From: | Mauro Cavalcanti <maurobio@gmail.com> |
To: | Nils Hempelmann <info@nilshempelmann.de> |
CC: | wps-dev@lists.dkrz.de, Carsten Ehbrecht <ehbrecht@dkrz.de>, Wolfgang.Falk@lwf.bayern.de, Scott Chamberlain <myrmecocystus@gmail.com> |
Nils,
No, there is no other way to get at once a larger number of records using the GBIF API (although one could be able to achieve this by sequentially querying the server using "batches" of 200 or 300 records each). As I said, there are good operational reasons for the GBIF developers to have imposed such limit.
As of your other question, it should better be put to the GBIF developers themselves (in the discussion list, so that we can all benefit from the answers! :-))
With warmest regards,
--
Dr. Mauro J. Cavalcanti
E-mail: maurobio@gmail.com
Web: http://sites.google.com/site/maurobio
Hi Mauro
Oh ...
No way to set it unlimited?
The 'manual via browser' option and parsing the returning csv is the current status.
or any alternative to pygbif?
If I understood it correctly, GBIF database is organized as Web Server, so I gues there should be a way to connect to the birdhouse WPS, am I right?
(put the web-dev list in copy)
Merci
Nils
On 02/06/2016 12:40, Mauro Cavalcanti wrote:
Nils,
That's a limit imposed (for good operational reasons) by the GBIF API. If you want a larger number of records, you'll have to download them "manually" (that is, via browser) and then parse locally the csv file returned from the GBIF server.
Hope this helps.
Best regards,
--
Dr. Mauro J. Cavalcanti
E-mail: maurobio@gmail.com
Web: http://sites.google.com/site/maurobioEm 02/06/2016 04:48, "Nils Hempelmann" <info@nilshempelmann.de> escreveu:
Hi Scott
works fine. Thanks a lot. Just have a question for the search limits:
The maximal records seems to be limited to 300. Is that on purpose?
And requesting more than 200000 gives a 'bad request'
Merci
Nils
In [68]: len( occurrences.search(taxonKey=key, limit=100)['results'])
Out[68]: 100
In [69]: len( occurrences.search(taxonKey=key, limit=300)['results'])
Out[69]: 300
In [70]: len( occurrences.search(taxonKey=key, limit=3000)['results'])
Out[70]: 300
In [71]: len( occurrences.search(taxonKey=key, limit=200000)['results'])
Out[71]: 300
In [72]: len( occurrences.search(taxonKey=key, limit=200001)['results'])
---------------------------------------------------------------------------
HTTPError Traceback (most recent call last)
<ipython-input-72-2f7d7b4ccba0> in <module>()
----> 1 len( occurrences.search(taxonKey=key, limit=200001)['results'])
/home/nils/.conda/envs/birdhouse/lib/python2.7/site-packages/pygbif/occurrences/search.pyc in search(taxonKey, scientificName, country, publishingCountry, hasCoordinate, typeStatus, recordNumber, lastInterpreted, continent, geometry, recordedBy, basisOfRecord, datasetKey, eventDate, catalogNumber, year, month, decimalLatitude, decimalLongitude, elevation, depth, institutionCode, collectionCode, hasGeospatialIssue, issue, q, mediatype, limit, offset, **kwargs)
251 'collectionCode': collectionCode, 'hasGeospatialIssue': hasGeospatialIssue,
252 'issue': issue, 'q': q, 'mediatype': mediatype, 'limit': limit,
--> 253 'offset': offset}, **kwargs)
254 return out
/home/nils/.conda/envs/birdhouse/lib/python2.7/site-packages/pygbif/gbifutils.pyc in gbif_GET(url, args, **kwargs)
17 def gbif_GET(url, args, **kwargs):
18 out = requests.get(url, params=args, headers=make_ua(), **kwargs)
---> 19 out.raise_for_status()
20 stopifnot(out.headers['content-type'])
21 return out.json()
/home/nils/.conda/envs/birdhouse/lib/python2.7/site-packages/requests/models.pyc in raise_for_status(self)
842
843 if http_error_msg:
--> 844 raise HTTPError(http_error_msg, response=self)
845
846 def close(self):
HTTPError: 400 Client Error: Bad Request for url: http://api.gbif.org/v1/occurrence/search?taxonKey=2882316&limit=200001&offset=0
In [73]:
On 02/06/2016 02:22, Scott Chamberlain wrote:
Fixes to docs and fix to download_get are up on Github now. Will push new version to pypi soon. Let me know if some thing still don't work for you after reinstalling from github
S
On Wed, Jun 1, 2016 at 4:51 PM Nils Hempelmann <info@nilshempelmann.de> wrote:
Hi Scott and Mauro
Mauro sended me a snipped of code which worked fine:
https://github.com/bird-house/flyingpigeon/blob/develop/scripts/pygbif_occurence.py
I found the example here:
https://github.com/maurobio/pygbif#occurrence-data
and here:
https://github.com/sckott/pygbif#occurrences-module
Thanks a lot very great. That enables a lot ;-)
I ll keep you posted
merci
Nils
On 02/06/2016 01:45, Scott Chamberlain wrote:
And where are those example from exactly? I don't see those examples searching the repo (which includes all docs).
`pygbif.name_suggest` wouldn't work because `name_suggest` is a method in the `species` module, so you'd have to do pygbif.species.name_suggest, or `from pygbif import species`, then `species.name_suggest`
Looks like occ.get(taxonKey = 252408386) is a documentation bug, that should be `key` instead of `taxonKey`, a copy paste error. will fix that.
The `occ.download_get` call has a small bug, will fix that
All other calls work for me
S
Hi,
What version of pygbif are you on? And what version of Python>
Best, Scott
Hi Mauro and Scott
Was checking out pygbif. seems to be a very useful tool.
Can you help me with the syntax (or forward me to the appropriate person
;-) )
The given snippets of code are outdated.
I am basically just looking for the occurrence coordinates:
here is my first try :
import pygbif
occ = pygbif.occurrences.search(scientificName='Fagus sylvatica')
occ['count']
... and further? ;-)
the examples in the docu are throwing errors:
key= pygbif.name_suggest(q='Helianthus annuus',rank='species')['key']
pygbif.search(taxonKey=key[0]['key'],limit=2)
from pygbif import occurrences as occ occ.search(taxonKey = 3329049)
occ.get(taxonKey = 252408386) occ.count(isGeoreferenced = True)
occ.download_list(user = "sckott", limit = 5)
occ.download_meta(key = "0000099-140929101555934")
occ.download_get("0000099-140929101555934")
Thanks
Nils