[Ala-portal] Populating portal for a country

[Marie Elise Lecoq]

Hi Nico,

For the point 3, I wrote some scripts in order to add data using the GBIF
API. It helps me a lot especially for contacts and institutions.
I will clean up and test a little bit more then I can put it on my github
if that can help.

Cheers from Madagascar !
Marie

On Thu, Jun 11, 2015 at 10:33 AM, Tim Robertson <trobertson at gbif.org> wrote:

> 1) My understanding is the GBIF API has been bombarded recently and they
> have now put in place a limit of the number of concurrent downloads to 3.
> The collectory currently attempts 10 download at a time - hence our
> problem.
> We’ll try and push out an update to fix this very soon.
>
>
> I’d like to just explain why this was rushed out.
> This was a necessary step as GBIF.org was exposed to DOS attacks (often
> inadvertently) which was happening.  In addition the GBIF R library now
> makes it trivial to download from GBIF and users can easily write a loop in
> code and trigger huge numbers of download which would block the system for
> others.
>
> We’ll continue to tune the values by monitoring operations, and probably
> introduce higher per-user limits so we can e.g. increase the concurrency
> for known ALA accounts.
>
> Please contact me if this causes any issue Nico.
>
>
>  On 10 Jun 2015, at 1:59 pm, Nicolas Noé <n.noe at biodiversity.be> wrote:
>
>  Hi all,
>
> I'm trying to set-up an experimental instance of ALA for Belgium. The idea
> is to present all data published by our belgian institutions, in order to
> give those institutations / collections / datasets more visibility.
>
> What's the recommended way to load this data (everything at GBIF published
> by Belgian institutions). I used the "Add all GBIF resource for a country"
> from the Collectory admin interface, but
>
> 1) I encountered some errors. All datasets detected correctly, but only 10
> are loaded - for the rest I get the message "Failed. Please check your
> authentication credentials are valid". The authentication credentials are
> indeed correct, since I only entered once (so, also for the working
> dataset). When looking at my downloads on the GBIF website, I can see no
> track those "failed" datasets. Can I get debugging information ? Should I
> fill a bug report ? In the ala-collectory repository ? Or some other ?
>
> 2) Also, the /collectory/manage/gbifLoadCountry pages states that "This is
> a simple method of bootstrapping an installation with data provided by GBIF
> web services. This is not intended for long-term production use.". So maybe
> I should use another method to load all the data for my country.
>
> 3) How are the (dataset) metadata populated when using this method ? It
> seems the basic metadata is there, but not much more. Should we do it by
> hand ? Should we write a tool to do it ? How about updates?
>
> Thanks a lot!
>  _______________________________________________
> Ala-portal mailing list
> Ala-portal at lists.gbif.org
> http://lists.gbif.org/mailman/listinfo/ala-portal
>
>
>  _______________________________________________
> Ala-portal mailing list
> Ala-portal at lists.gbif.org
> http://lists.gbif.org/mailman/listinfo/ala-portal
>
>
>
> _______________________________________________
> Ala-portal mailing list
> Ala-portal at lists.gbif.org
> http://lists.gbif.org/mailman/listinfo/ala-portal
>
>


--
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gbif.org/pipermail/ala-portal/attachments/20150611/6d94d8cd/attachment.html>