- ALA-Portal - lists.gbif.org

[Spatial] Error during integration of layer
by Marie Elise Lecoq 28 May '16

28 May '16

Hi all, When I integreted a new layer on the spatial portal, I got this error : org.postgresql.util.PSQLException: ERROR: value too long for type character varying(160) at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2102) at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1835) at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257) at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:500) at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:388) at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:381) at au.org.ala.layers.ingestion.contextual.ContextualFromShapefileDatabaseLoader.importShapeFile(ContextualFromShapefileDatabaseLoader.java:293) at au.org.ala.layers.ingestion.contextual.ContextualFromShapefileDatabaseLoader.create(ContextualFromShapefileDatabaseLoader.java:99) at au.org.ala.layers.ingestion.contextual.ContextualFromShapefileDatabaseLoader.main(ContextualFromShapefileDatabaseLoader.java:54) It seems that one of the name is too long. I tried to fixed on the data and I tried to re-run the script sh but I was not possible because the shapefile is already here. How can reload it ? Do I have to remove rows from the postgresql and remove files in /data/ala/data/layers/ready/shape/ ? Thanks for your help, Cheers, Marie --

2 2

[Indexation] Questions
by Marie Elise Lecoq 26 May '16

26 May '16

Hi all ! I have few questions about the indexation : 1. It seems that some occurrences are wrongly indexed. For example, if I search "Pica Pica", the three first results will be not relevant ( http://recherche.gbif.fr/occurrences/search?taxa=Pica+pica). Do I need to change something on the nameindexer ? I don't have a BIE instance on our system, do I need to install one in order to help ? 2. We have some provider codes with punctuation (e.g. comma, dot ). It's seems that the link between collection, institution and dataresource is not made due to this. It works with accents. 3. I try to index a data resource with more than 20 million occurrences and I have a NullPointerException, it's seems that guid is not found. I can upload data resource with much less data inside so I guess the problem comme from the data resource itself (size ?). Do you have a special way to deal with huge data resource ? Thanks in advance for your help :-)! Cheers, Marie --

2 2

[Spatial] Questions
by Marie Elise Lecoq 26 May '16

26 May '16

Hi all (again), I installed the spatial portal with Dave in February, you can find it here <http://spatial.gbif.fr> (not yet in production) and I have also few questions about it : 1. What is http://spatial.ala.org.au/layers ? It is part of the spatial or another tool ? 2. What do you mean by environmental layer in the ALA spatial infrastructure ? Is it layer with temperature, height above sea, etc ? And again, thanks in advance for you help. Cheers, Marie --

2 1

Re: [Ala-portal] Query to get data for a country not published by that country
by Rui Figueira 26 Feb '16

26 Feb '16

Dear Frederico, Many thanks for this clarification. If that sort of query could be implemented, it would greatly facilitate the data management of our national data portal in the future. Regards, Rui On 02/24/2016 12:29 PM, Federico Mendez wrote: > Dear Rui, I¹m afraid that by now the only way of getting those records is > how you described: downloading all the portuguese records and then remove > records coming from portuguese institutions. In the next months we¹ll be > analysing the feasibility of implementing ¹negated¹ expression in out API > to be able to compose queries like: country=PT&publishingCountry=NOT PT. > > Regards, Federico Mendez. > > > > On 23/02/16 15:04, "Mélianie Raymond" <mraymond(a)gbif.org> wrote: >> -----Original Message----- >> From: Rui Figueira [mailto:ruifigueira@isa.ulisboa.pt] >> Sent: Tuesday, February 23, 2016 11:19 AM >> To: Mélianie Raymond; ala-portal(a)lists.gbif.org; Federico Mendez >> Subject: Re: [Ala-portal] Query to get data for a country not published >> by that country >> >> Hi Méliane, >> >> Yes, that's it: >> >>> identify all occurrences in Portugal published by institutions NOT in >>> Portugal. >> If you could create that custom query, it would greatly facilitate the >> management of the national data portal in the future. >> >> But for the time being, I understand that the only option is to download >> all occurrences for Portugal, and afterwards filter it to remove records >> published by Portuguese, right? >> >> Thanks, >> >> Rui >> >> ------------------ >> >> Rui Figueira >> Coordenador do Nó Português do GBIF >> ruifigueira(a)isa.ulisboa.pt >> Instituto Superior de Agronomia >> Herbário >> Tapada da Ajuda, 1349-017 Lisboa, Portugal Tel. +351 213653421 | Fax. >> +351 213653195 http://www.gbif.pt http://www.isa.ulisboa.pt >> >> On 02/23/2016 09:05 AM, Mélianie Raymond wrote: >>> Dear Rui, >>> >>> Thanks for your message. I have discussed this with Federico and we >>> would just like to be sure we have understood the request correctly. >>> >>> Here is an example of a dataset published by the Netherlands about >>> occurrences in Portugal: doi:10.15468/eswvar >>> (http://www.gbif.org/dataset/9cbcf2b3-448f-48ce-9510-39e25c6234a8). >>> >>> However, the datasets published by institutions NOT in Portugal >>> typically tend to be on broader areas, only including some occurrences >>> that fall in Portugal. You can use the API to identify all occurrences >>> in Portugal: http://api.gbif.org/v1/occurrence/search?country=PT. >>> >>> We could make a custom query to identify all occurrences in Portugal >>> published by institutions NOT in Portugal if this is what you need? >>> >>> Thanks! >>> Melianie >>> >>> --------------------------------------- >>> Dr. Mélianie Raymond >>> Senior Programme Officer for Node Development GBIF Secretariat >>> Universitetsparken 15 >>> DK-2100 Copenhagen, Denmark >>> Tel.: +45 35 32 14 96/ Fax: +45 35 32 14 80 >>> E-mail: mraymond(a)gbif.org >>> Website: www.gbif.org >>> >>> >>> >>> >>> -----Original Message----- >>> From: Ala-portal [mailto:ala-portal-bounces@lists.gbif.org] On Behalf >>> Of Rui Figueira >>> Sent: Friday, February 19, 2016 3:57 PM >>> To: ala-portal(a)lists.gbif.org >>> Subject: [Ala-portal] Query to get data for a country not published by >>> that country >>> >>> Hello, >>> >>> We would like to add to our test implementation of ALA one resource >>> with the data for Portugal which IS NOT published by portuguese >>> institutions. >>> Is it possible to define a query in the GBIF data portal to achieve >>> this? >>> >>> Thanks, >>> >>> Rui Figueira >>> >>> _______________________________________________ >>> Ala-portal mailing list >>> Ala-portal(a)lists.gbif.org >>> http://lists.gbif.org/mailman/listinfo/ala-portal

2 2

Query to get data for a country not published by that country
by Rui Figueira 23 Feb '16

23 Feb '16

Hello, We would like to add to our test implementation of ALA one resource with the data for Portugal which IS NOT published by portuguese institutions. Is it possible to define a query in the GBIF data portal to achieve this? Thanks, Rui Figueira

2 2

ALA development updates
by David.Martin＠csiro.au 05 Dec '15

05 Dec '15

Hello, This is email is just to give an update on projects currently working on/with ALA components. These notes are from project representatives: GBIF - "Building on the work of Miguel Carboni during the Canberra workshop (2014) GBIF have completed the basic development to enable crawling of BioCASe providers into the biocache-store. The code is on a private github fork [1] and a pull request [2] has been made for this to be included in the master trunk. It should be noted that some dependency version changes might mean this needs to be carefully considered before it can be applied which could introduce a slight delay in it’s availability. The implementation is a simple wrapper around the GBIF BioCASe crawler [3] and only extracts a few key fields from the ABCD schema at the moment [4]. It would be trivial to complete this mapping should any BioCASe aggregators wish to use the biocache.” - Tim Robertson GBIF Spain - "Work on documentation, particularly about how to include the lang_selector in the in data portal and also I’m working in the updating of several dataset of our data portal. I hope a can do more before the end of the year. Today I did a presentation about the Quality Test that the Data Portal does: https://twitter.com/smr_sama?lang=es and the next week a have another meeting about our image server.” - Santiago de la Riva GBIF France - "I updated the collectory plugin from the 1.3 version to the 1.5.1 version. I have corrected some CSS issues on our collectory theme and I have begun to migrate the collectory module from BS2 to BS3. I have also fixed the issue #11 on github.” - Marie-Elise Lecoq GBIF Germany - “For BGBM, this would be investigating ALA components (biocache, generic hub, image server).” - Jörg Holetschek GBIF Argentina - "I've been working for the national node portal in order to release it. We've tried to install the log module and cas module. Also we've been working on a document to define the actors and roles of the national portal.” - Miguel Carboni GBIF Portugal - "Styling for Portuguese data portal, header a footer done, working on a front search form based on the simple search. To be ready in one week.” - Rui Figueira ICMBio, Brasil - ICMBio have just released a public portal - http://portaldabiodiversidade.icmbio.gov.br/portal/ GBIF Belgium - Currently working on ALA documentation as part of the GBIF Encounter Bay project: http://www.gbif.org/project/2015-ala-internationalization Atlas of Living Scotland - "A Beta version of the Atlas of Living Scotland is up and running at http://www.als.scot. We are currently working on spatial components (http://spatial.als.scot) and a tool for exploring the data by habitat." - Dave Martin Atlas of Living Australia - "Recent development efforts have been focussed on a refreshing of the DigiVol site (http://digivol.ala.org.au), and a tool called BioCollect which we hope to release early next year. BioCollect with be a place for creating citizen science projects.” - Dave Martin Thanks to every for the updates. If you are working on another project using ALA, we’d love to hear from you. Thanks, Dave Martin Atlas of Living Australia

1 0

Mapping between institution and collection and observation data resources
by Marie-Elise Lecoq 14 Sep '15

14 Sep '15

Hi all ! I have a question about the data resource mapping and advanced search. One of our institutions has collection datasets and observation data resources. When you make a search by institutions or collections in the advanced search, only the collection records appear. During the provider map creation, on the group provider box, I can only choose collection datasets but not the others. It is not possible for me to make the mapping between an institution and its observation data resources. My version of biocache-hub is quite old but I have tried with the lastest version of ala-install on a vagrant instance that didn't work either. If it is not possible, I will create a patch to do it because we need this kind of mapping in our data portal. Please, do not hesitate to ask me more questions if you need ! Thanks in advance :-)! Cheers, Marie --

2 1

Hardware infrastructure
by Nicolas Noé 20 Aug '15

20 Aug '15

Hi guys, I'm still evaluating how to deploy an ALA instance for Belgium at the end of this year and was interested in hearing your first hands experience about the infrastructure you run the portal on. I'm especially interested about the experience of countries like France and Spain that did it recently and who have a volume of data of the same order of magnitude as Belgium. The documentation (https://github.com/AtlasOfLivingAustralia/documentation/wiki/Essential-tool…) recommends the following server configuration: - 100Gb SSD - 32GB RAM - 2x CPU My questions are: * Were those requirements sufficient for you? * Did you deploy everything on a single server or did you use a more splitted infrastructure? If so, did you wrote your own ansible scripts to match your configuration? * Did you use physical servers? Or Amazon EC2 instances ? Or another cloud providers? * Is there anything you'd do differently if you had to do it again tomorrow? * Any other comments or remark? Thanks a lot for your collaboration! Nicolas

2 1

Issues with "Add all GBIF resources for a country"
by Nicolas Noé 08 Jul '15

08 Jul '15

Hi, I have some difficulties using this feature to get all the data for Belgium. When I run this option without limits, it generally get stuck for a few specific datasets, I guess for simple reasons like connectivity errors. For example yesterday, 61 datasets (on a total of 71) were successfully downloaded, while the rest (10 datasets) are stuck at either "Downloading", either "Creating GBIF Data Resource". My question is: how can I complete the process by restarting those? - Reloading everything via the "Reload existing resources" checkbox will be very slow, and will likely still fails for a few datasets. - Just relaunching the whole import process without checking this box will not do anything (I guess those stuck resources are considered in progress) - Deleting those few data resources is not possible since they do not appear in the "view all data resources" page. Any idea of what I can do to complete my country import ? Best, Nicolas

2 1

ERROR in biocache-hubs-0-70??
by Santiago Martinez de la Riva 03 Jul '15

03 Jul '15

Hi all, Im trying to generate the war file of generic-hub. I only have change the app-name, and when Im going to create the war I obtain this error: generic-hub-es/target/work/plugins/build-info-1.2.6/grails-app/controllers/grails/plugin/buildinfo/BuildInfoController.groovy: -1: Repetitive method name/signature for method 'java.lang.Object withFormat(groovy.lang.Closure)' in class 'grails.plugin.buildinfo.BuildInfoController'. Thanks in advance, Santi.

1 1