- IPT - lists.gbif.org

Re: [IPT] IPT v2 Snapshot frequency
by "Markus Döring (GBIF)" 15 Oct '10

15 Oct '10

when setting up a data directory for the IPT a small file called "datadir.location" is created in the WEB-INF folder that points to the actual data dir. If you simply remove this file you force the IPT to the setup page. You can enter an existing data dir if you want and all its configuration is loaded - or create a new, empty one. For jetty when running "mvn" this file is kept with the source files. To remove it simply do: $ cd gbif-ipt $ rm src/main/webapp/WEB-INF/datadir.location Markus On Oct 15, 2010, at 14:12, Nicolas Noé wrote: > One last question, when playing with it (without building the war, letting mvn launch jetty), how can I restore it to its original state (to lose all configurations and send me back on the first (select datat dir) page ? > > Thanks, > > Nicolas > > On 10/15/2010 12:07 PM, "Markus Döring (GBIF)" wrote: >> Nicolas, >> that is indeed a good idea. We will consider some regular build of the war, maybe even on a daily basis. I will upload a new version later today once the serious bugs are fixed that Ive introduced when refactoring the http/registry communication. >> >> If you ever want to try the latest trunk from the svn its rather simple if you have java, maven and svn installed (OSX comes with all of these): >> >> $ svn checkout http://gbif-providertoolkit.googlecode.com/svn/trunk/gbif-ipt >> $ cd gbif-ipt >> $ mvn >> >> The last "mvn" command should start up the IPT with a local jetty instance on port 7001 and you can go into your browser of choice with this url: >> http://localhost:7001/ipt >> >> If you want to build a war instead you need to issue a different maven command: >> >> $ mvn install >> >> and then you should find the war in this folder (example revision 2396): >> >> $ cp target/ipt-2.0-RC1-SNAPSHOT-r2396.war >> >> >> When you do the maven command for the very first time it will download quite a few libraries, so you need some patience at that point. But when done once it will be fairly fast afterwards, even if you update to the latest trunk again. That can be done like this by the way: >> >> $ cd gbif-ipt >> $ svn up >> >> Markus >> >> >> On Oct 15, 2010, at 11:48, Nicolas Noé wrote: >> >>> Hi, >>> >>> With the current move to the v2 of the IPT and the very fast development of the last few weeks, I think it could be great to provide development snapshots (similar to the one Tim uploaded a few days ago) on a regular and very frequent basis. >>> >>> For example, I've reported a bug 2 days ago that is supposed to be fixed, and I'd like to re-test this and investigate deeper the IPT (hat bug make me stuck at the beginning), so I could follow the development day after day and provide immediate feedback to the development team. I think this transparency and rapid feedback mechanism is important if we want a very fast development rate and at the same time keep a good stability. It could also lead to a better communication between IPT developers and end users (for example to keep them aware of which features/bugs should be on the top-priority list). >>> >>> I see 3 ways to make these builds for "faster feedback" possible : >>> >>> 1) The GBIF manually creates "ipt-snapshot.war" on a regular basis (i.e. weekly) >>> 2) Putting an automatic mechanism to create these builds, the frequence could so be increased (nightly builds). >>> 3) Update (and detail) the documentation of the build process on the wiki (available for v1). The java build process/ build tools and the quirks that can be encountered at that step can be, to say the least, quite intimidating for someone who isn't a Java developer. >>> >>> What do you think of this ? >>> >>> Thanks, Nicolas. >>> <niconoe.vcf>_______________________________________________ >>> IPT mailing list >>> IPT(a)lists.gbif.org >>> http://lists.gbif.org/mailman/listinfo/ipt > > <niconoe.vcf>

1 0

IPT v2 Snapshot frequency
by Nicolas Noé 15 Oct '10

15 Oct '10

Hi, With the current move to the v2 of the IPT and the very fast development of the last few weeks, I think it could be great to provide development snapshots (similar to the one Tim uploaded a few days ago) on a regular and very frequent basis. For example, I've reported a bug 2 days ago that is supposed to be fixed, and I'd like to re-test this and investigate deeper the IPT (hat bug make me stuck at the beginning), so I could follow the development day after day and provide immediate feedback to the development team. I think this transparency and rapid feedback mechanism is important if we want a very fast development rate and at the same time keep a good stability. It could also lead to a better communication between IPT developers and end users (for example to keep them aware of which features/bugs should be on the top-priority list). I see 3 ways to make these builds for "faster feedback" possible : 1) The GBIF manually creates "ipt-snapshot.war" on a regular basis (i.e. weekly) 2) Putting an automatic mechanism to create these builds, the frequence could so be increased (nightly builds). 3) Update (and detail) the documentation of the build process on the wiki (available for v1). The java build process/ build tools and the quirks that can be encountered at that step can be, to say the least, quite intimidating for someone who isn't a Java developer. What do you think of this ? Thanks, Nicolas.

2 1

data base wont publish
by Fabian Haas 01 Oct '10

01 Oct '10

Dear All, would need help with some IPT issue (1.0-RC3-SNAPSHOT). I submitted two data bases to GBIF and they work fine, no I added another one with just about 155 data sets. checking on the contents with the explore tools the data have read in properly. Now it say "modified, not publish", clicking on "publish" gets at best a time out error, but nothing happens. neither on my side, changing to published, nor gets Andrea Hahn a message. I have tried several times on different days to exclude Internet issues to no effect. One more thing, which does not bother me, but might be a hint. Some of the meta data like Geographic Coverage, do nort save but get lost, when I leave the toolkit http://kupe.icipe.org:8080/ipt/index.html Any ideas what it could be? best Fabian -- ********************************************************** fhaas(a)icipe.org, Extension -2052 The African Insect Taxonomy Toolkit AITT http://taxonomy.icipe.org Dr. Fabian Haas Insect Taxonomist and ABS Specialist ICIPE - African Insect Science for Food and Health Duduville Campus, Kasarani P.O. Box 30772 - 00100 N A I R O B I Kenya Telephone No. +254 (0)20 8632000 Fax No. +254 (0)20 8632001 Cell Phone +254 (0)728 132868 This position is supported by CIM, the Centre for International Migration www.cimonline.de

1 0

Re: [IPT] GBIF Integrated Publishing Toolkit
by Tim Robertson (GBIF) 20 Sep '10

20 Sep '10

Dear Jonas, Thank you for your questions. Answers given inline: > Dear Tim > > Thanks for the information, and the great work on the IPT. I think > it will be a good move, just seen from my perspective, to > concentrate on the core functionality for better performance. Thank you for this feedback - this is important to help steer the IPT process. > After having done some promotion and held some training and > installation sessions of the IPT under the GBIF - NordGen evaluation > project I see that the general interest in the genebank > documentation community is quite strong and the installation and > configuration is very user-friendly. We just this week had a > workshop in Estonia introducing the IPT and the GBIF network to > people from genebanks in Estonia, Latvia and Lithuania and the > response was very good. > > To be able to make the most out of these training sessions, there > will be a few more during the remainder of this year, I have a > couple of questions about the coming version: > > if the GUI for mapping imported dataset fields to the DwC / DwC > extension fields will be significantly different? It will be slightly different, but familiar enough for people who understand the older version to pick it up easily. We received feedback during the IPT experts training event last year, and have implemented their recommendations. In short, the terms are simply provided in a long list. I attach a screenshot (dwc-mapping.png) showing the mapping page for occurrences. Please remember that we will initiate a testing phase in October, and there will be the ability to comment on this and have it revised if there are problems. > > if the mapping through SQL / direct database connection will work > significantly different? No changes here. You create the SQL statement and then do a mapping the same, regardless of whether it is a SQL source or text file. > if it will be possible to host multiple resources registered under > different GBIF nodes on the same IPT webapp or if all resources on > one IPT webapp must be under the same GBIF node? The revised version will handle this: The IPT instance is registered with the Institution/Organisation/Node to which it is associated. The ADMIN user can then associate further Institution/Organisation/ Nodes from which data MANAGERs can choose when registering a data RESOURCE. Therefore the following could be handled: - ADMIN in Node1 installs the IPT - ADMIN enables Node2 and Institution3 as available entities for MANAGERs to use - ADMIN creates accounts for MANAGERs - MANAGERs can log in and create resources - when a MANAGER creates a resource they see a choice of Node1, Node2 or Institution3 to associate the resource with In addition to this, the ADMIN can provide an alias to each Organisation/Node/Institution, so that MANAGERs see something familiar. For example the GBIF Registry might have "The Botanical holdings of the Harvard Herbarium Museum" but internally the ADMIN and MANAGERs all call this "HUH". The ADMIN will be able to declare this and MANAGERs will simply see "HUH" in the choice of Institutions to associate the resource with. I hope this helps, and thanks again for the feedback, Tim > > Best regards > > Jonas Nordling, NordGen > From: trobertson(a)gbif.org > To: jonas.nordling(a)nordgen.org > Sent: Mon, 13 Sep 2010 16:10:36 +0200 > Subject: GBIF Integrated Publishing Toolkit > > [FOR INFORMATION] > > Dear Node Managers and Data Publishers, > > The objective of this communication is to update you on the current > situation of the GBIF Integrated Publishing Toolkit (IPT). The IPT > is a tool providing publishing capabilities for primary biodiversity > occurrence data, taxonomic checklists and the associated metadata > for these resource types. > > Since its first introduction in March 2009, the IPT has been used > with some success by a limited number of users to publish primary > biodiversity data into the GBIF network, along with descriptive > metadata at the dataset level. > > The GBIF Secretariat (GBIFS) has solicited and received a vast > amount of feedback during the first year of the IPT use, and would > like to thank all those who provided this. The feedback received > overwhelmingly confirms that the concept of the IPT is sound. > However, feedback has also made it clear that it is over-specified > for the majority of user needs, is not yet sufficiently robust, is > too slow in operation, and (due to the excessive functionality) has > high server requirements creating a barrier for many to adopt and > use it. In addition, the feedback has made it clear that our > communications surrounding the status of the IPT releases have been > too infrequent and unclear concerning which release candidate we > were offering and what functionalities this release candidate would > include or not include, as per user needs/expectations. > > Based upon the extensive feedback GBIFS has revised our planned > development roadmap for the IPT and are currently performing a major > refactoring effort to: > • Reduce the server requirements significantly > • Increase the data import performance by removing the embedded > database > • Remove the dependency on heavy libraries and tools such as Geoserver > • Remove all data interfaces that are not necessary for data > publication through GBIF > • Improve the robustness of the tool > > When complete, the refactored IPT will offer through an intuitive > interface: > • Authoring of metadata according to the GBIF metadata profile > • Import and mapping of checklist and occurrence data either from > file upload or by connection of a database > • Registration to and publication onto the GBIF network > • Importing and mapping of data to the DarwinCore, and DarwinCore > extensions > • Output formats of EML (2.1.0) metadata and DarwinCore Archive > • Improved customisation options for the “About this IPT” > • Improved GBIF Registry integration > • Improved DarwinCore extension and vocabulary organization > • Improved management of the organisations to which the IPT and > resources are related, to enable co-hosting capabilities > • Enhanced dataset metadata authoring > > Whilst this is a reduction in existing functionality this has been > deemed necessary to ensure the IPT is meeting the core requirements > as expressed by the user community. The following features will be > removed and only reintroduced if necessary through consultation with > the IPT community: > • TAPIR interface > • TCS output format > • Search and browse web interface > • OGC web services > > All IPT development efforts are now focused on this revised roadmap, > and it is anticipated that early testing of the new codebase will > commence in October 2010 with the involvement of willing Nodes. The > release of a fully stable IPT is targeted for the end of 2010. > However, this will only occur if the testing community is satisfied > that the product is ‘release-ready’ together with the required user > manual and technical documentation. Should that not be the case, as > it may take more time to implement and test the improvements > outlined above, you will be informed as soon as possible. > > Further developments of the IPT beyond this release will only be > implemented after a further scoping with the GBIF community. An area > for further IPT discussion will be set up on the GBIF community > website (http://community.gbif.org) > > If you have any questions or would like to know more about the GBIF > IPT and/or revised process, feel free to contact: > > Tim Robertson > GBIF Information Systems Architect > trobertson(a)gbif.org > > With best regards, > > Tim Robertson > GBIF Secretariat > > >

1 0

Functionality request: ADMIN checking data before GBIF registration
by Tim Robertson (GBIF) 16 Sep '10

16 Sep '10

Hi all, Hannu has raised a request for the following to be satisfied by the IPT: "- Publishing a resource must be accepted by the owner of the provider. It has happened that a test user publishes something odd which goes all the way to the data portal without nobody controlling it." This is a contradiction to the requests of others, and specifically those wishing to promote basic "data hosting centers", who request that a data MANAGER should be able to work autonomously. After discussion with the developers the proposal is to implement the following, which we hope satisfies both requirements: In the Administration section, an ADMIN can choose to enable or disable the ability for MANAGERS to register resources with GBIF. By default MANAGERS can register a resource, but an ADMIN can disable this through this check box. If anyone has any concerns or comments on this approach, please can you raise them on this list? Many thanks, Tim

4 5

Re: [IPT] Functionality request: ADMIN checking data before GBIFregistration
by Tim Robertson (GBIF) 15 Sep '10

15 Sep '10

Thanks all for the comments, which I will try and collate here with some resolutions: Propose resolved: a) The preference would be to allow the ADMIN to provide Registration privileges to individual MANAGERS Outstanding issues: b) Visualisations are important, and will need discussion and potentially further IPT modules developed, or deployment of external services. I know of one group considering the development of DwC-A visualisations already, and technologies like Google Fusion tables makes this kind of thing trivial. c) record resolution is something that has been indicated as important. This can be achieved in one of 2 ways: > implementation in the IPT, and requires research into technologies to perform satisfactorily Potential technologies might be - Berkeley DB - A relational database, such as Mysql or H2 - Lucene indexing > reliance on a "stable cache" for record serving The first release (for test purposes) of the revised IPT software will not have record level serving, but while this is being developed, I would like to ask people to start discussing what kind of record level serving is truly a requirement in the IPT, as opposed to a "nice to have". We support DwC-A in the GBIF portal, and the intention is to simply reserve the record that came from the DwC-A directly, unless the record indicates there is further information on a URL (e.g. if the record identifier is an LSID). Would this strategy not be suitable for the likes of the BioCASe portals as often there is no further information to redirect to? in the case of the IPT, there is no extra information, and I propose should the source be a DwC-A, that individual records be cached in the harvesting portal. With this approach, there would be no individual record serving needs in the IPT. Ultimately, we might consider aiming for data owners offering single records on a resolvable URL, and conforming to Linked/Open Data requirements, along with a DwC-A effectively providing a single "index" view of the dataset. The DwC-A records would reference the originals by resolvable ID so any search system would always be able to point back to the authoritative source. This would effectively be distributed indexing, and not dissimilar to the sharing of sitemaps, but with extra information to enable better discovery. Thank you all for this feedback, and please correct any misunderstandings on my part Tim On Sep 15, 2010, at 12:03 PM, Mihail-Constantin Carausu wrote: > Dear Tim > > I think the development team's mentioned approach is a workable > solution > to cover both requirements at this stage. > However, I think Hannu (and me) had in mind a kind of "Basket of > approvals"-alike functionality in the Admin's (owner of the provider) > administration section side: When a Manager has been published a > dataset > through the IPT, this will automatically trigger a request for > approval > or submits an yes/no event in the Admin's administration section. The > Admin must finally active interfere and approve the dataset > publication > (e.g. by checking an "Approved" check box in the basket of approvals > list with events/datasets in the administration section) at the > absolute > latest stage (e.g. when GBIF just needs to start to index it, or > something like that). Without this final approval the dataset will > still > be published and visible through the IPT but not visible/searchable on > the GBIF data portal. This approach is not necessarily in > contradiction > with the Manager's ability to autonomously publish datasets within the > IPT, only it puts this ability always under control from the central > administration section when the dataset has to go to the GBIF data > portal. > I think both solutions/approaches have obvious advantages and > disadvantages while none of them provides a 100% protection against > publishing something odd by a (test) user. > I have a little question regarding the development team's proposed > solutions: is it not possible for the central Admin to enable the > publishing ability for some "trusted" managers and disable this for > others inside the same instance of the IPT. > > Now I saw Hannu's new message just arrived, sorry for eventually > unsynchronized double-crossing messages, but I will send this anyhow. > > Best regards, > Mihail > > ---------------------------- > Mihail Carausu > MSc.Eng., Informatics Manager > Danish Biodiversity Information Facility (DanBIF) > -------------------------------------------- > > > -----Original Message----- > From: ipt-bounces(a)lists.gbif.org [mailto:ipt-bounces@lists.gbif.org] > On > Behalf Of Tim Robertson (GBIF) > Sent: 15. september 2010 09:43 > To: ipt(a)lists.gbif.org > Subject: [IPT] Functionality request: ADMIN checking data before > GBIFregistration > > Hi all, > > Hannu has raised a request for the following to be satisfied by the > IPT: > "- Publishing a resource must be accepted by the owner of the > provider. It has happened that a test user publishes something odd > which goes all the way to the data portal without nobody controlling > it." > > This is a contradiction to the requests of others, and specifically > those wishing to promote basic "data hosting centers", who request > that a data MANAGER should be able to work autonomously. > > After discussion with the developers the proposal is to implement the > following, which we hope satisfies both requirements: > In the Administration section, an ADMIN can choose to enable or > disable the ability for MANAGERS to register resources with GBIF. By > default MANAGERS can register a resource, but an ADMIN can disable > this through this check box. > > If anyone has any concerns or comments on this approach, please can > you raise them on this list? > > Many thanks, > Tim > > > > > > > _______________________________________________ > IPT mailing list > IPT(a)lists.gbif.org > http://lists.gbif.org/mailman/listinfo/ipt >

2 2

Fwd: Auto-discard notification
by "Markus Döring (GBIF)" 15 Sep '10

15 Sep '10

[forwarded to list] Begin forwarded message: > The attached message has been automatically discarded. > From: "Mihail-Constantin Carausu" <MCCarausu(a)snm.ku.dk> > Date: September 15, 2010 12:03:50 GMT+02:00 > To: "Tim Robertson $GBIF$" <trobertson(a)gbif.org>, <ipt(a)lists.gbif.org> > Subject: RE: [IPT] Functionality request: ADMIN checking data before GBIFregistration > > > Dear Tim > > I think the development team's mentioned approach is a workable solution > to cover both requirements at this stage. > However, I think Hannu (and me) had in mind a kind of "Basket of > approvals"-alike functionality in the Admin's (owner of the provider) > administration section side: When a Manager has been published a dataset > through the IPT, this will automatically trigger a request for approval > or submits an yes/no event in the Admin's administration section. The > Admin must finally active interfere and approve the dataset publication > (e.g. by checking an "Approved" check box in the basket of approvals > list with events/datasets in the administration section) at the absolute > latest stage (e.g. when GBIF just needs to start to index it, or > something like that). Without this final approval the dataset will still > be published and visible through the IPT but not visible/searchable on > the GBIF data portal. This approach is not necessarily in contradiction > with the Manager's ability to autonomously publish datasets within the > IPT, only it puts this ability always under control from the central > administration section when the dataset has to go to the GBIF data > portal. > I think both solutions/approaches have obvious advantages and > disadvantages while none of them provides a 100% protection against > publishing something odd by a (test) user. > I have a little question regarding the development team's proposed > solutions: is it not possible for the central Admin to enable the > publishing ability for some "trusted" managers and disable this for > others inside the same instance of the IPT. > > Now I saw Hannu's new message just arrived, sorry for eventually > unsynchronized double-crossing messages, but I will send this anyhow. > > Best regards, > Mihail > > ---------------------------- > Mihail Carausu > MSc.Eng., Informatics Manager > Danish Biodiversity Information Facility (DanBIF) > -------------------------------------------- > > > -----Original Message----- > From: ipt-bounces(a)lists.gbif.org [mailto:ipt-bounces@lists.gbif.org] On > Behalf Of Tim Robertson (GBIF) > Sent: 15. september 2010 09:43 > To: ipt(a)lists.gbif.org > Subject: [IPT] Functionality request: ADMIN checking data before > GBIFregistration > > Hi all, > > Hannu has raised a request for the following to be satisfied by the IPT: > "- Publishing a resource must be accepted by the owner of the > provider. It has happened that a test user publishes something odd > which goes all the way to the data portal without nobody controlling > it." > > This is a contradiction to the requests of others, and specifically > those wishing to promote basic "data hosting centers", who request > that a data MANAGER should be able to work autonomously. > > After discussion with the developers the proposal is to implement the > following, which we hope satisfies both requirements: > In the Administration section, an ADMIN can choose to enable or > disable the ability for MANAGERS to register resources with GBIF. By > default MANAGERS can register a resource, but an ADMIN can disable > this through this check box. > > If anyone has any concerns or comments on this approach, please can > you raise them on this list? > > Many thanks, > Tim > > > > > > > _______________________________________________ > IPT mailing list > IPT(a)lists.gbif.org > http://lists.gbif.org/mailman/listinfo/ipt > >

1 0

Re: [IPT] GBIF Integrated Publishing Toolkit
by "Markus Döring (GBIF)" 15 Sep '10

15 Sep '10

[forwarding to list again as the message was rejected] retrieving a full record from the network is a basic often needed feature. But there might be scenarios where people do not want to have a running server at all and prefer to simply publish their dataset to somewhere else. In that case "somewhere else" could be responsible for exposing the records. An IPT used for (national) hosting might be such a solution (given the record resolution is implemented). But one could also think to serve the full, original record by GBIF centrally for those cases. That would require GBIF to cache a full record as it came in of course, e.g. the full ABCD or a DWC with extensions potentially unknown to us. We have that on our wishlist for quite some time already. Is it correct that both solutions, IPT record resolution or central GBIF resolution, would work for the BioCASe portals? Markus On Sep 15, 2010, at 10:48, Guentsch, Anton wrote: > Hi Markus, > > I am not so familiar with the IPT but may be this as a clarification: Some of the "special interest networks" using GBIF/BioCASE technology heavily rely on the ability of provider software to return a full data record (e.g. ABCD and its extensions) for a given ID. This is in particular essential for the DNA-Bank-Network, which uses ABCD plus the DNA-extension. A second example is the emerging GBIF-D soil zoology network. Personally, I think that these networks have a great potential for demonstrating the capabilities of the GBIF infrastructures being based on high-quality data subsets and interfaces which directly serve a certain user community. > > For the IPT I think that at the very least we would need some kind of interface which makes every record addressable. Preferably, the record should be a full record in the sense that DNA-extensions etc. are somehow represented. > > Best regards, > Anton > >> -----Ursprüngliche Nachricht----- >> Von: "Markus Döring (GBIF)" [mailto:mdoering@gbif.org] >> Gesendet: Mittwoch, 15. September 2010 10:24 >> An: =?iso-8859-1?Q?=22Holetschek(a)ns.gbif.org; Holetschek, Jörg >> Cc: Tim Robertson (trobertson(a)gbif.org) ipt(a)lists.gbif.org; Guentsch, >> Anton; Berendsohn, Walter G. >> Betreff: Re: [IPT] GBIF Integrated Publishing Toolkit >> >> Jörg, >> currently it is not planned to provide single record access of any kind >> as we removed any record storage, e.g. the previous H2 database, to >> increase performance and reduce resource requirements. >> But if that is indeed needed we could add it back in of course. A >> simple link to a record in xml or rdf based on its ID would be >> sufficient? >> >> Markus >> >> >> On Sep 15, 2010, at 10:11, Holetschek, Jörg wrote: >> >>> Hi Tim, >>> >>> the removal of the TAPIR interface will cause problems for all of our >> BioCASe portals - for showing the complete record details, they do a >> DiGIR/BioCASe/TAPIR request, which would then fail. >>> >>> Will there be an alternative option to get the DwC XML document, for >> example via a deep link? >>> >>> Cheers from Berlin, >>> Jörg >>> >>> -----Original Message----- >>> From: trobertson(a)gbif.org [mailto:trobertson@gbif.org] >>> Sent: Monday, 13 September, 2010 16:11 >>> To: Berendsohn, Walter G. >>> Subject: GBIF Integrated Publishing Toolkit >>> >>> [FOR INFORMATION] >>> >>> Dear Node Managers and Data Publishers, >>> >>> The objective of this communication is to update you on the current >> situation of the GBIF Integrated Publishing Toolkit (IPT). The IPT is >> a tool providing publishing capabilities for primary biodiversity >> occurrence data, taxonomic checklists and the associated metadata for >> these resource types. >>> >>> Since its first introduction in March 2009, the IPT has been used >> with some success by a limited number of users to publish primary >> biodiversity data into the GBIF network, along with descriptive >> metadata at the dataset level. >>> >>> The GBIF Secretariat (GBIFS) has solicited and received a vast amount >> of feedback during the first year of the IPT use, and would like to >> thank all those who provided this. The feedback received >> overwhelmingly confirms that the concept of the IPT is sound. However, >> feedback has also made it clear that it is over-specified for the >> majority of user needs, is not yet sufficiently robust, is too slow in >> operation, and (due to the excessive functionality) has high server >> requirements creating a barrier for many to adopt and use it. In >> addition, the feedback has made it clear that our communications >> surrounding the status of the IPT releases have been too infrequent and >> unclear concerning which release candidate we were offering and what >> functionalities this release candidate would include or not include, as >> per user needs/expectations. >>> >>> Based upon the extensive feedback GBIFS has revised our planned >> development roadmap for the IPT and are currently performing a major >> refactoring effort to: >>> . Reduce the server requirements significantly >>> . Increase the data import performance by removing the embedded >> database >>> . Remove the dependency on heavy libraries and tools such as >> Geoserver >>> . Remove all data interfaces that are not necessary for data >> publication through GBIF >>> . Improve the robustness of the tool >>> >>> When complete, the refactored IPT will offer through an intuitive >> interface: >>> . Authoring of metadata according to the GBIF metadata profile >>> . Import and mapping of checklist and occurrence data either from >> file upload or by connection of a database >>> . Registration to and publication onto the GBIF network >>> . Importing and mapping of data to the DarwinCore, and DarwinCore >> extensions >>> . Output formats of EML (2.1.0) metadata and DarwinCore Archive >>> . Improved customisation options for the "About this IPT" >>> . Improved GBIF Registry integration >>> . Improved DarwinCore extension and vocabulary organization >>> . Improved management of the organisations to which the IPT and >> resources are related, to enable co-hosting capabilities >>> . Enhanced dataset metadata authoring >>> >>> Whilst this is a reduction in existing functionality this has been >> deemed necessary to ensure the IPT is meeting the core requirements as >> expressed by the user community. The following features will be >> removed and only reintroduced if necessary through consultation with >> the IPT community: >>> . TAPIR interface >>> . TCS output format >>> . Search and browse web interface >>> . OGC web services >>> >>> All IPT development efforts are now focused on this revised roadmap, >> and it is anticipated that early testing of the new codebase will >> commence in October 2010 with the involvement of willing Nodes. The >> release of a fully stable IPT is targeted for the end of 2010. However, >> this will only occur if the testing community is satisfied that the >> product is 'release-ready' together with the required user manual and >> technical documentation. Should that not be the case, as it may take >> more time to implement and test the improvements outlined above, you >> will be informed as soon as possible. >>> >>> Further developments of the IPT beyond this release will only be >> implemented after a further scoping with the GBIF community. An area >> for further IPT discussion will be set up on the GBIF community website >> (http://community.gbif.org) >>> >>> If you have any questions or would like to know more about the GBIF >> IPT and/or revised process, feel free to contact: >>> >>> Tim Robertson >>> GBIF Information Systems Architect >>> trobertson(a)gbif.org >>> >>> With best regards, >>> >>> Tim Robertson >>> GBIF Secretariat >>> >>> >>> >>> _______________________________________________ >>> IPT mailing list >>> IPT(a)lists.gbif.org >>> http://lists.gbif.org/mailman/listinfo/ipt >>> > >

2 1

Re: [IPT] GBIF Integrated Publishing Toolkit
by "Holetschek, Jörg" 15 Sep '10

15 Sep '10

Hi Tim, the removal of the TAPIR interface will cause problems for all of our BioCASe portals - for showing the complete record details, they do a DiGIR/BioCASe/TAPIR request, which would then fail. Will there be an alternative option to get the DwC XML document, for example via a deep link? Cheers from Berlin, Jörg -----Original Message----- From: trobertson(a)gbif.org [mailto:trobertson@gbif.org] Sent: Monday, 13 September, 2010 16:11 To: Berendsohn, Walter G. Subject: GBIF Integrated Publishing Toolkit [FOR INFORMATION] Dear Node Managers and Data Publishers, The objective of this communication is to update you on the current situation of the GBIF Integrated Publishing Toolkit (IPT). The IPT is a tool providing publishing capabilities for primary biodiversity occurrence data, taxonomic checklists and the associated metadata for these resource types. Since its first introduction in March 2009, the IPT has been used with some success by a limited number of users to publish primary biodiversity data into the GBIF network, along with descriptive metadata at the dataset level. The GBIF Secretariat (GBIFS) has solicited and received a vast amount of feedback during the first year of the IPT use, and would like to thank all those who provided this. The feedback received overwhelmingly confirms that the concept of the IPT is sound. However, feedback has also made it clear that it is over-specified for the majority of user needs, is not yet sufficiently robust, is too slow in operation, and (due to the excessive functionality) has high server requirements creating a barrier for many to adopt and use it. In addition, the feedback has made it clear that our communications surrounding the status of the IPT releases have been too infrequent and unclear concerning which release candidate we were offering and what functionalities this release candidate would include or not include, as per user needs/expectations. Based upon the extensive feedback GBIFS has revised our planned development roadmap for the IPT and are currently performing a major refactoring effort to: . Reduce the server requirements significantly . Increase the data import performance by removing the embedded database . Remove the dependency on heavy libraries and tools such as Geoserver . Remove all data interfaces that are not necessary for data publication through GBIF . Improve the robustness of the tool When complete, the refactored IPT will offer through an intuitive interface: . Authoring of metadata according to the GBIF metadata profile . Import and mapping of checklist and occurrence data either from file upload or by connection of a database . Registration to and publication onto the GBIF network . Importing and mapping of data to the DarwinCore, and DarwinCore extensions . Output formats of EML (2.1.0) metadata and DarwinCore Archive . Improved customisation options for the "About this IPT" . Improved GBIF Registry integration . Improved DarwinCore extension and vocabulary organization . Improved management of the organisations to which the IPT and resources are related, to enable co-hosting capabilities . Enhanced dataset metadata authoring Whilst this is a reduction in existing functionality this has been deemed necessary to ensure the IPT is meeting the core requirements as expressed by the user community. The following features will be removed and only reintroduced if necessary through consultation with the IPT community: . TAPIR interface . TCS output format . Search and browse web interface . OGC web services All IPT development efforts are now focused on this revised roadmap, and it is anticipated that early testing of the new codebase will commence in October 2010 with the involvement of willing Nodes. The release of a fully stable IPT is targeted for the end of 2010. However, this will only occur if the testing community is satisfied that the product is 'release-ready' together with the required user manual and technical documentation. Should that not be the case, as it may take more time to implement and test the improvements outlined above, you will be informed as soon as possible. Further developments of the IPT beyond this release will only be implemented after a further scoping with the GBIF community. An area for further IPT discussion will be set up on the GBIF community website (http://community.gbif.org) If you have any questions or would like to know more about the GBIF IPT and/or revised process, feel free to contact: Tim Robertson GBIF Information Systems Architect trobertson(a)gbif.org With best regards, Tim Robertson GBIF Secretariat

2 1

IPT Roadmap
by Tim Robertson (GBIF) 14 Sep '10

14 Sep '10

Dear all, Please accept my apologies for cross posting this, but in case you were not on the previous mailing list, I would like to draw your attention to the communication sent below. We will shortly be revising all the issues in the code site to reflect the status of the revised version. An IPT user group is being set up today on http://community.gbif.org to discuss the IPT, in addition to this mailing list. Please do contact me for any clarifications you need. Many thanks, Tim [FOR INFORMATION] Dear Node Managers and Data Publishers, The objective of this communication is to update you on the current situation of the GBIF Integrated Publishing Toolkit (IPT). The IPT is a tool providing publishing capabilities for primary biodiversity occurrence data, taxonomic checklists and the associated metadata for these resource types. Since its first introduction in March 2009, the IPT has been used with some success by a limited number of users to publish primary biodiversity data into the GBIF network, along with descriptive metadata at the dataset level. The GBIF Secretariat (GBIFS) has solicited and received a vast amount of feedback during the first year of the IPT use, and would like to thank all those who provided this. The feedback received overwhelmingly confirms that the concept of the IPT is sound. However, feedback has also made it clear that it is over-specified for the majority of user needs, is not yet sufficiently robust, is too slow in operation, and (due to the excessive functionality) has high server requirements creating a barrier for many to adopt and use it. In addition, the feedback has made it clear that our communications surrounding the status of the IPT releases have been too infrequent and unclear concerning which release candidate we were offering and what functionalities this release candidate would include or not include, as per user needs/expectations. Based upon the extensive feedback GBIFS has revised our planned development roadmap for the IPT and are currently performing a major refactoring effort to: * Reduce the server requirements significantly * Increase the data import performance by removing the embedded database * Remove the dependency on heavy libraries and tools such as Geoserver * Remove all data interfaces that are not necessary for data publication through GBIF * Improve the robustness of the tool When complete, the refactored IPT will offer through an intuitive interface: * Authoring of metadata according to the GBIF metadata profile * Import and mapping of checklist and occurrence data either from file upload or by connection of a database * Registration to and publication onto the GBIF network * Importing and mapping of data to the DarwinCore, and DarwinCore extensions * Output formats of EML (2.1.0) metadata and DarwinCore Archive * Improved customisation options for the "About this IPT" * Improved GBIF Registry integration * Improved DarwinCore extension and vocabulary organization * Improved management of the organisations to which the IPT and resources are related, to enable co-hosting capabilities * Enhanced dataset metadata authoring Whilst this is a reduction in existing functionality this has been deemed necessary to ensure the IPT is meeting the core requirements as expressed by the user community. The following features will be removed and only reintroduced if necessary through consultation with the IPT community: * TAPIR interface * TCS output format * Search and browse web interface * OGC web services All IPT development efforts are now focused on this revised roadmap, and it is anticipated that early testing of the new codebase will commence in October 2010 with the involvement of willing Nodes. The release of a fully stable IPT is targeted for the end of 2010. However, this will only occur if the testing community is satisfied that the product is 'release-ready' together with the required user manual and technical documentation. Should that not be the case, as it may take more time to implement and test the improvements outlined above, you will be informed as soon as possible. Further developments of the IPT beyond this release will only be implemented after a further scoping with the GBIF community. An area for further IPT discussion will be set up on the GBIF community website (http://community.gbif.org) If you have any questions or would like to know more about the GBIF IPT and/or revised process, feel free to contact: Tim Robertson GBIF Information Systems Architect trobertson(a)gbif.org With best regards, Tim Robertson GBIF Secretariat

1 0