- IPT - lists.gbif.org

Fwd: Auto-discard notification
by "Markus Döring (GBIF)" 15 Sep '10

15 Sep '10

[forwarded to list] Begin forwarded message: > The attached message has been automatically discarded. > From: "Mihail-Constantin Carausu" <MCCarausu(a)snm.ku.dk> > Date: September 15, 2010 12:03:50 GMT+02:00 > To: "Tim Robertson \(GBIF\)" <trobertson(a)gbif.org>, <ipt(a)lists.gbif.org> > Subject: RE: [IPT] Functionality request: ADMIN checking data before GBIFregistration > > > Dear Tim > > I think the development team's mentioned approach is a workable solution > to cover both requirements at this stage. > However, I think Hannu (and me) had in mind a kind of "Basket of > approvals"-alike functionality in the Admin's (owner of the provider) > administration section side: When a Manager has been published a dataset > through the IPT, this will automatically trigger a request for approval > or submits an yes/no event in the Admin's administration section. The > Admin must finally active interfere and approve the dataset publication > (e.g. by checking an "Approved" check box in the basket of approvals > list with events/datasets in the administration section) at the absolute > latest stage (e.g. when GBIF just needs to start to index it, or > something like that). Without this final approval the dataset will still > be published and visible through the IPT but not visible/searchable on > the GBIF data portal. This approach is not necessarily in contradiction > with the Manager's ability to autonomously publish datasets within the > IPT, only it puts this ability always under control from the central > administration section when the dataset has to go to the GBIF data > portal. > I think both solutions/approaches have obvious advantages and > disadvantages while none of them provides a 100% protection against > publishing something odd by a (test) user. > I have a little question regarding the development team's proposed > solutions: is it not possible for the central Admin to enable the > publishing ability for some "trusted" managers and disable this for > others inside the same instance of the IPT. > > Now I saw Hannu's new message just arrived, sorry for eventually > unsynchronized double-crossing messages, but I will send this anyhow. > > Best regards, > Mihail > > ---------------------------- > Mihail Carausu > MSc.Eng., Informatics Manager > Danish Biodiversity Information Facility (DanBIF) > -------------------------------------------- > > > -----Original Message----- > From: ipt-bounces(a)lists.gbif.org [mailto:ipt-bounces@lists.gbif.org] On > Behalf Of Tim Robertson (GBIF) > Sent: 15. september 2010 09:43 > To: ipt(a)lists.gbif.org > Subject: [IPT] Functionality request: ADMIN checking data before > GBIFregistration > > Hi all, > > Hannu has raised a request for the following to be satisfied by the IPT: > "- Publishing a resource must be accepted by the owner of the > provider. It has happened that a test user publishes something odd > which goes all the way to the data portal without nobody controlling > it." > > This is a contradiction to the requests of others, and specifically > those wishing to promote basic "data hosting centers", who request > that a data MANAGER should be able to work autonomously. > > After discussion with the developers the proposal is to implement the > following, which we hope satisfies both requirements: > In the Administration section, an ADMIN can choose to enable or > disable the ability for MANAGERS to register resources with GBIF. By > default MANAGERS can register a resource, but an ADMIN can disable > this through this check box. > > If anyone has any concerns or comments on this approach, please can > you raise them on this list? > > Many thanks, > Tim > > > > > > > _______________________________________________ > IPT mailing list > IPT(a)lists.gbif.org > http://lists.gbif.org/mailman/listinfo/ipt > >

1 0

Re: [IPT] GBIF Integrated Publishing Toolkit
by "Markus Döring (GBIF)" 15 Sep '10

15 Sep '10

[forwarding to list again as the message was rejected] retrieving a full record from the network is a basic often needed feature. But there might be scenarios where people do not want to have a running server at all and prefer to simply publish their dataset to somewhere else. In that case "somewhere else" could be responsible for exposing the records. An IPT used for (national) hosting might be such a solution (given the record resolution is implemented). But one could also think to serve the full, original record by GBIF centrally for those cases. That would require GBIF to cache a full record as it came in of course, e.g. the full ABCD or a DWC with extensions potentially unknown to us. We have that on our wishlist for quite some time already. Is it correct that both solutions, IPT record resolution or central GBIF resolution, would work for the BioCASe portals? Markus On Sep 15, 2010, at 10:48, Guentsch, Anton wrote: > Hi Markus, > > I am not so familiar with the IPT but may be this as a clarification: Some of the "special interest networks" using GBIF/BioCASE technology heavily rely on the ability of provider software to return a full data record (e.g. ABCD and its extensions) for a given ID. This is in particular essential for the DNA-Bank-Network, which uses ABCD plus the DNA-extension. A second example is the emerging GBIF-D soil zoology network. Personally, I think that these networks have a great potential for demonstrating the capabilities of the GBIF infrastructures being based on high-quality data subsets and interfaces which directly serve a certain user community. > > For the IPT I think that at the very least we would need some kind of interface which makes every record addressable. Preferably, the record should be a full record in the sense that DNA-extensions etc. are somehow represented. > > Best regards, > Anton > >> -----Ursprüngliche Nachricht----- >> Von: "Markus Döring (GBIF)" [mailto:mdoering@gbif.org] >> Gesendet: Mittwoch, 15. September 2010 10:24 >> An: =?iso-8859-1?Q?=22Holetschek(a)ns.gbif.org; Holetschek, Jörg >> Cc: Tim Robertson (trobertson(a)gbif.org) ipt(a)lists.gbif.org; Guentsch, >> Anton; Berendsohn, Walter G. >> Betreff: Re: [IPT] GBIF Integrated Publishing Toolkit >> >> Jörg, >> currently it is not planned to provide single record access of any kind >> as we removed any record storage, e.g. the previous H2 database, to >> increase performance and reduce resource requirements. >> But if that is indeed needed we could add it back in of course. A >> simple link to a record in xml or rdf based on its ID would be >> sufficient? >> >> Markus >> >> >> On Sep 15, 2010, at 10:11, Holetschek, Jörg wrote: >> >>> Hi Tim, >>> >>> the removal of the TAPIR interface will cause problems for all of our >> BioCASe portals - for showing the complete record details, they do a >> DiGIR/BioCASe/TAPIR request, which would then fail. >>> >>> Will there be an alternative option to get the DwC XML document, for >> example via a deep link? >>> >>> Cheers from Berlin, >>> Jörg >>> >>> -----Original Message----- >>> From: trobertson(a)gbif.org [mailto:trobertson@gbif.org] >>> Sent: Monday, 13 September, 2010 16:11 >>> To: Berendsohn, Walter G. >>> Subject: GBIF Integrated Publishing Toolkit >>> >>> [FOR INFORMATION] >>> >>> Dear Node Managers and Data Publishers, >>> >>> The objective of this communication is to update you on the current >> situation of the GBIF Integrated Publishing Toolkit (IPT). The IPT is >> a tool providing publishing capabilities for primary biodiversity >> occurrence data, taxonomic checklists and the associated metadata for >> these resource types. >>> >>> Since its first introduction in March 2009, the IPT has been used >> with some success by a limited number of users to publish primary >> biodiversity data into the GBIF network, along with descriptive >> metadata at the dataset level. >>> >>> The GBIF Secretariat (GBIFS) has solicited and received a vast amount >> of feedback during the first year of the IPT use, and would like to >> thank all those who provided this. The feedback received >> overwhelmingly confirms that the concept of the IPT is sound. However, >> feedback has also made it clear that it is over-specified for the >> majority of user needs, is not yet sufficiently robust, is too slow in >> operation, and (due to the excessive functionality) has high server >> requirements creating a barrier for many to adopt and use it. In >> addition, the feedback has made it clear that our communications >> surrounding the status of the IPT releases have been too infrequent and >> unclear concerning which release candidate we were offering and what >> functionalities this release candidate would include or not include, as >> per user needs/expectations. >>> >>> Based upon the extensive feedback GBIFS has revised our planned >> development roadmap for the IPT and are currently performing a major >> refactoring effort to: >>> . Reduce the server requirements significantly >>> . Increase the data import performance by removing the embedded >> database >>> . Remove the dependency on heavy libraries and tools such as >> Geoserver >>> . Remove all data interfaces that are not necessary for data >> publication through GBIF >>> . Improve the robustness of the tool >>> >>> When complete, the refactored IPT will offer through an intuitive >> interface: >>> . Authoring of metadata according to the GBIF metadata profile >>> . Import and mapping of checklist and occurrence data either from >> file upload or by connection of a database >>> . Registration to and publication onto the GBIF network >>> . Importing and mapping of data to the DarwinCore, and DarwinCore >> extensions >>> . Output formats of EML (2.1.0) metadata and DarwinCore Archive >>> . Improved customisation options for the "About this IPT" >>> . Improved GBIF Registry integration >>> . Improved DarwinCore extension and vocabulary organization >>> . Improved management of the organisations to which the IPT and >> resources are related, to enable co-hosting capabilities >>> . Enhanced dataset metadata authoring >>> >>> Whilst this is a reduction in existing functionality this has been >> deemed necessary to ensure the IPT is meeting the core requirements as >> expressed by the user community. The following features will be >> removed and only reintroduced if necessary through consultation with >> the IPT community: >>> . TAPIR interface >>> . TCS output format >>> . Search and browse web interface >>> . OGC web services >>> >>> All IPT development efforts are now focused on this revised roadmap, >> and it is anticipated that early testing of the new codebase will >> commence in October 2010 with the involvement of willing Nodes. The >> release of a fully stable IPT is targeted for the end of 2010. However, >> this will only occur if the testing community is satisfied that the >> product is 'release-ready' together with the required user manual and >> technical documentation. Should that not be the case, as it may take >> more time to implement and test the improvements outlined above, you >> will be informed as soon as possible. >>> >>> Further developments of the IPT beyond this release will only be >> implemented after a further scoping with the GBIF community. An area >> for further IPT discussion will be set up on the GBIF community website >> (http://community.gbif.org) >>> >>> If you have any questions or would like to know more about the GBIF >> IPT and/or revised process, feel free to contact: >>> >>> Tim Robertson >>> GBIF Information Systems Architect >>> trobertson(a)gbif.org >>> >>> With best regards, >>> >>> Tim Robertson >>> GBIF Secretariat >>> >>> >>> >>> _______________________________________________ >>> IPT mailing list >>> IPT(a)lists.gbif.org >>> http://lists.gbif.org/mailman/listinfo/ipt >>> > >

2 1

Re: [IPT] GBIF Integrated Publishing Toolkit
by "Holetschek, Jörg" 15 Sep '10

15 Sep '10

Hi Tim, the removal of the TAPIR interface will cause problems for all of our BioCASe portals - for showing the complete record details, they do a DiGIR/BioCASe/TAPIR request, which would then fail. Will there be an alternative option to get the DwC XML document, for example via a deep link? Cheers from Berlin, Jörg -----Original Message----- From: trobertson(a)gbif.org [mailto:trobertson@gbif.org] Sent: Monday, 13 September, 2010 16:11 To: Berendsohn, Walter G. Subject: GBIF Integrated Publishing Toolkit [FOR INFORMATION] Dear Node Managers and Data Publishers, The objective of this communication is to update you on the current situation of the GBIF Integrated Publishing Toolkit (IPT). The IPT is a tool providing publishing capabilities for primary biodiversity occurrence data, taxonomic checklists and the associated metadata for these resource types. Since its first introduction in March 2009, the IPT has been used with some success by a limited number of users to publish primary biodiversity data into the GBIF network, along with descriptive metadata at the dataset level. The GBIF Secretariat (GBIFS) has solicited and received a vast amount of feedback during the first year of the IPT use, and would like to thank all those who provided this. The feedback received overwhelmingly confirms that the concept of the IPT is sound. However, feedback has also made it clear that it is over-specified for the majority of user needs, is not yet sufficiently robust, is too slow in operation, and (due to the excessive functionality) has high server requirements creating a barrier for many to adopt and use it. In addition, the feedback has made it clear that our communications surrounding the status of the IPT releases have been too infrequent and unclear concerning which release candidate we were offering and what functionalities this release candidate would include or not include, as per user needs/expectations. Based upon the extensive feedback GBIFS has revised our planned development roadmap for the IPT and are currently performing a major refactoring effort to: . Reduce the server requirements significantly . Increase the data import performance by removing the embedded database . Remove the dependency on heavy libraries and tools such as Geoserver . Remove all data interfaces that are not necessary for data publication through GBIF . Improve the robustness of the tool When complete, the refactored IPT will offer through an intuitive interface: . Authoring of metadata according to the GBIF metadata profile . Import and mapping of checklist and occurrence data either from file upload or by connection of a database . Registration to and publication onto the GBIF network . Importing and mapping of data to the DarwinCore, and DarwinCore extensions . Output formats of EML (2.1.0) metadata and DarwinCore Archive . Improved customisation options for the "About this IPT" . Improved GBIF Registry integration . Improved DarwinCore extension and vocabulary organization . Improved management of the organisations to which the IPT and resources are related, to enable co-hosting capabilities . Enhanced dataset metadata authoring Whilst this is a reduction in existing functionality this has been deemed necessary to ensure the IPT is meeting the core requirements as expressed by the user community. The following features will be removed and only reintroduced if necessary through consultation with the IPT community: . TAPIR interface . TCS output format . Search and browse web interface . OGC web services All IPT development efforts are now focused on this revised roadmap, and it is anticipated that early testing of the new codebase will commence in October 2010 with the involvement of willing Nodes. The release of a fully stable IPT is targeted for the end of 2010. However, this will only occur if the testing community is satisfied that the product is 'release-ready' together with the required user manual and technical documentation. Should that not be the case, as it may take more time to implement and test the improvements outlined above, you will be informed as soon as possible. Further developments of the IPT beyond this release will only be implemented after a further scoping with the GBIF community. An area for further IPT discussion will be set up on the GBIF community website (http://community.gbif.org) If you have any questions or would like to know more about the GBIF IPT and/or revised process, feel free to contact: Tim Robertson GBIF Information Systems Architect trobertson(a)gbif.org With best regards, Tim Robertson GBIF Secretariat

2 1

IPT Roadmap
by Tim Robertson (GBIF) 14 Sep '10

14 Sep '10

Dear all, Please accept my apologies for cross posting this, but in case you were not on the previous mailing list, I would like to draw your attention to the communication sent below. We will shortly be revising all the issues in the code site to reflect the status of the revised version. An IPT user group is being set up today on http://community.gbif.org to discuss the IPT, in addition to this mailing list. Please do contact me for any clarifications you need. Many thanks, Tim [FOR INFORMATION] Dear Node Managers and Data Publishers, The objective of this communication is to update you on the current situation of the GBIF Integrated Publishing Toolkit (IPT). The IPT is a tool providing publishing capabilities for primary biodiversity occurrence data, taxonomic checklists and the associated metadata for these resource types. Since its first introduction in March 2009, the IPT has been used with some success by a limited number of users to publish primary biodiversity data into the GBIF network, along with descriptive metadata at the dataset level. The GBIF Secretariat (GBIFS) has solicited and received a vast amount of feedback during the first year of the IPT use, and would like to thank all those who provided this. The feedback received overwhelmingly confirms that the concept of the IPT is sound. However, feedback has also made it clear that it is over-specified for the majority of user needs, is not yet sufficiently robust, is too slow in operation, and (due to the excessive functionality) has high server requirements creating a barrier for many to adopt and use it. In addition, the feedback has made it clear that our communications surrounding the status of the IPT releases have been too infrequent and unclear concerning which release candidate we were offering and what functionalities this release candidate would include or not include, as per user needs/expectations. Based upon the extensive feedback GBIFS has revised our planned development roadmap for the IPT and are currently performing a major refactoring effort to: * Reduce the server requirements significantly * Increase the data import performance by removing the embedded database * Remove the dependency on heavy libraries and tools such as Geoserver * Remove all data interfaces that are not necessary for data publication through GBIF * Improve the robustness of the tool When complete, the refactored IPT will offer through an intuitive interface: * Authoring of metadata according to the GBIF metadata profile * Import and mapping of checklist and occurrence data either from file upload or by connection of a database * Registration to and publication onto the GBIF network * Importing and mapping of data to the DarwinCore, and DarwinCore extensions * Output formats of EML (2.1.0) metadata and DarwinCore Archive * Improved customisation options for the "About this IPT" * Improved GBIF Registry integration * Improved DarwinCore extension and vocabulary organization * Improved management of the organisations to which the IPT and resources are related, to enable co-hosting capabilities * Enhanced dataset metadata authoring Whilst this is a reduction in existing functionality this has been deemed necessary to ensure the IPT is meeting the core requirements as expressed by the user community. The following features will be removed and only reintroduced if necessary through consultation with the IPT community: * TAPIR interface * TCS output format * Search and browse web interface * OGC web services All IPT development efforts are now focused on this revised roadmap, and it is anticipated that early testing of the new codebase will commence in October 2010 with the involvement of willing Nodes. The release of a fully stable IPT is targeted for the end of 2010. However, this will only occur if the testing community is satisfied that the product is 'release-ready' together with the required user manual and technical documentation. Should that not be the case, as it may take more time to implement and test the improvements outlined above, you will be informed as soon as possible. Further developments of the IPT beyond this release will only be implemented after a further scoping with the GBIF community. An area for further IPT discussion will be set up on the GBIF community website (http://community.gbif.org) If you have any questions or would like to know more about the GBIF IPT and/or revised process, feel free to contact: Tim Robertson GBIF Information Systems Architect trobertson(a)gbif.org With best regards, Tim Robertson GBIF Secretariat

1 0

Mapping/publishing problem?
by Jens Rathsman 26 Aug '10

26 Aug '10

Dear GBIF helpdesk, I am publishing some datasets on an IPT server (1.0-RC3-SNAPSHOT) for the Swedish GBIF node. There are some problems with the mapping or publishing the following fields: VerbatimCollectingDate --- EarliestDateCollected --- StartTimeOfDay --- YearSampled --- MonthOfYear --- DayOfMonth --- Collector --- For example catalogNumber 38922 ( puiblished at: http://gbif.se:8080/ipt/occDetail.html?guid=e4ec28b2-128a-49ee-b299-00ced41…) has values for all of the above fields in the datasource (a MySql database) but none of them are presenteded by IPT. When I click "Annotations" for the resource there are no problems shown, and I get no clues from the tomcat log file (catalina.out). This is the data from the data source: catalogNumber collector earliestDateCollected verbatimCollectingDate startDayOfYear endDayOfYear eventTime yearSampled monthOfYear dayOfYear 38922 Åhlander, E et al. 1990-04-04 4 and 12 Apr 1990 94 102 10:00:00 1990 4 4 What could be causing this problem? Best regards, --------------------------------- Jens Rathsman, System developper GBIF Sweden - http://www.gbif.se<http://www.gbif.se/> mailto:jens.rathsman@nrm.se tel: +46 8 5195 5116 Swedish Museum of Natural History - http://www.nrm.se<http://www.nrm.se/> Box 50007 SE-104 05 Stockholm Sweden

3 5

Short presentations about the IPT available in German
by Alberto González-Talaván (GBIF) 24 Aug '10

24 Aug '10

Dear Friends from the IPT list, Thanks to Jörg Holetschek from the Botanischer Garten und Botanisches Museum Berlin-Dahlem (Germany) we have now available the set of 6 short presentations about the IPT available also in German! (Sorry Jörg for having taken some time to upload them!) It is awesome the work that you guys have done with the translation of those presentations into Chinese, Dutch, English, French, German, Portuguese and Spanish!! As usual, the presentations are available in the IPT resources page of the "GBIF IPT helpdesk and training experts" group of the GBIF Community Site: http://community.gbif.org/pg/pages/view/3872/ The GBIF Community Site has been recently officially launched and we have doubled the number of users in a very short time. Have a look at the new aspect of the site and the list of new users or join it if you have not done so yet! Thanks again to all those involved in the translations! With best regards Alberto ---------- Alberto GONZÁLEZ TALAVÁN Training Officer Global Biodiversity Information Facility Secretariat Universitetsparken 15 DK-2100 Copenhagen, Denmark email: <mailto:atalavan@gbif.org> atalavan(a)gbif.org web: <http://www.gbif.org/> http://www.gbif.org/ T: +45 35 32 14 83 F: +45 35 32 14 80

1 0

Re: [IPT] IPT configuration question
by Tim Robertson (GBIF) 17 Aug '10

17 Aug '10

Dear Jens and Jose, Could you please liaise to make sure that at the time of registration there are no outages and perhaps debug this issue? I suspect that there is some firewall issue coming in to play here from the client (it's port 80 though), or perhaps there is some proxy involved that we have not seen before. May I also please ask Jens - which version of the IPT is being installed? Many thanks, Tim On Aug 17, 2010, at 1:36 PM, Jens Rathsman wrote: > Dear GBIF developers, > > I work at the Swedish GBIF node and have a question about IPT- > organization setup. I write to you as system architect because I > didn’t know who else to contact. Maybe you can redirect me if you > are not the one to answer those questions? > > Some months ago I installed and tried out IPT on some datasets here > at the museum. Maps and embedded tapir worked nicely. Now one of our > data contributors is installing IPT for automatic data delivery to > us, and he had problems. I verified at our installation, that was > successfully setup earlier and saw the same problem. > > When configuring “Admin > Configuration > Organization” IPT seems > to get stuck at “loading from repository” while looking up “Title” > and “GIBF Node”. I don’t remember seeing this problem before. What > do you think is the reason? I have two guesses, > 1. could it be that the lookup is made on some port that is > blocked? (We have a new firewall since installing in march.) If so, > what port is used? > 2. Does gbif.org have a problem with the lookup service? > > Is the organization configuration needed for data contributors, or > only for Gbif-nodes? > > Best regards, > Jens Rathsman > > --------------------------------- > Jens Rathsman, System developper > GBIF Sweden - http://www.gbif.se > mailto:jens.rathsman@nrm.se > tel: +46 8 5195 5116 > Swedish Museum of Natural History - http://www.nrm.se Box 50007 > SE-104 05 Stockholm > Sweden >

3 2

Geoserver and Tomcat 6.0.26
by Burke Chih-Jen Ko 16 May '10

16 May '10

Dear IPT friends, While upgrading my IPT instance to 1.0RC3, I also upgraded Tomcat 6.0.26. IPT 1.0RC3 worked, but either bundled or original geoserver 1.7.7 didn't startup. Downgrading Tomcat back to 6.0.24 solves this issue. My Tomcat runs on Mac OS X 10.6.3 with JDK 1.6. For information only. Cheers, Burke Taiwan Biodiversity Information Facility, TaiBIF Biodiversity Research Center, Academia Sinica 128 Sec 2, Academia Rd., Nangang District Taipei 115, Taiwan Tel: +886 2 27899621 ext. 220

1 0

problem with charts/graphs
by Andrea Hahn (GBIF) 04 May '10

04 May '10

(forwarding for Kyle) Hi, I'm on an internet connection going through an HTTP proxy that is password protected. Unfortunately, I lose the charts/graphs in my IPT and was wondering why that is. Charts/graphs displayed from the same default resource 'Pontaurus' on ipt.gbif.org for example display fine. Any idea what the problem might be? If the problem is not related to connecting via a password protected proxy, what other roots might there be to this problem? Thanks for your time and help in resolving this problem, Kyle

2 1

OAI-PMH on IPT
by Hannu Saarenmaa 15 Apr '10

15 Apr '10

Hello How can I find the OAI-PMH access point of an IPT resource or of the entire server? It is not listed among the other webservices & resources. Is this documented somewhere? Thanks, Hannu

3 3