[IPT] Adding an extension for germplasm to IPT

Wed May 27 08:43:22 CEST 2009

Dear Eli,

The instructions I received from Markus is actually already included here below as included in your email. I have also added the instructions from Tim, as also posted to the IPT mailing list.

Basically what (I think) the IPT does is to contact the registry as defined in "ipt.properties". By default the GBIF registry in Copenhagen is defined here, but you may modify this to point to your own registry. I did not mimic the complete features of the registry - ONLY the list of extensions "extensions.json" to add the new draft germplasm extension I made. You may want to add your own thesaurus "thesauri.json" of controlled vocabulary or more features in the similar way. After loading my new extension to my IPT at NordGen, I modified the "properties.ipt" back to point to the GBIF registry in Copenhagen again.

To mimic the registry feature for adding the extension, I made a copy of the file from "http://gbrds.gbif.org/registry/ipt/extensions.json" and added my application schema from the extension as you will see at "http://wwwdev.ngb.se/portal/rs/ipt/extensions.json".

The application schema (as defined in "extensions.json") and online from: "http://wwwdev.ngb.se/portal/rs/ipt_germplasm.xml", creates a namespace (for the use by IPT) as well as inclides the URL for the actual draft germplasm extension I made:

  "http://ipt.gbif.org/extension/germplasm/ http://wwwdev.ngb.se/portal/rs/dwc_germplasm.xsd"

The application schema itself (ipt_germplasm.xml) only lists the terms I want to use in IPT, with the property name to be displayed in the IPT mapping interface. To create this file I started with one of the other extensions "http://gbrds.gbif.org/resources/extensions/vernacularName.xml". I just made a copy of this file and replaced the names and terms for vernacular name extension with the terms I wanted from the draft germplasm extension.

I hope that my description of what I think it was I did is more or less correct. I copy the IPT mailing list so that Markus and Tim may have a chance to correct me or to add more information to the procedure.

Cheers
Dag Terje

  _____  

From: Eli Agbayani [mailto:eagbayani at eol.org]
To: Dag Terje Filip Endresen [mailto:dag.endresen at nordgen.org]
Sent: Tue, 26 May 2009 22:52:35 +0200
Subject: Re: Adding an extension for germplasm to IPT

Dear Tag,
  Thanks for your email. Can you please forward to me the instructions you said you followed in testing your own extension. That is the instructions the GBIF guys gave you.
  Thanks.
  Eli

  ----- Original Message -----
  From: "Dag Terje Filip Endresen" <dag.endresen at nordgen.org>
  To: "\"Markus Döring (GBIF)\"" <mdoering at gbif.org>, "Eli Agbayani" <eagbayani at eol.org>
  Cc: ipt at lists.gbif.org
  Sent: Friday, May 15, 2009 9:07:29 AM (GMT-0500) America/New_York
  Subject: Adding an extension for germplasm to IPT

  Hi Markus and Tim, 

  Thanks a lot for your help! I followed your instructions and successfully added my DRAFT version of a DarwinCore extension for germplasm to the IPT. I have listed a brief summary of each of the steps I followed below. 

  If you are interested, you may visit the IPT from NordGen with a DEMO dataset mapped to terms from this DRAFT Darwin Core extension for germplasm at: 
  http://atlas.nordgen.org/ipt/occResource.html?resource_id=77 

  1. Schema 
  I built a suggestion to a DRAFT Darwin Core extansion for germplasm 
  http://code.google.com/p/darwincore/source/browse/trunk/xsd/profiles/germplasm/dwc_germplasm.xsd 
  http://wwwdev.ngb.se/portal/rs/dwc_germplasm.xsd 

  2. XML definition for IPT 
  Listing the terms to use from the suggested DRAFT DarwinCore extension 
  http://wwwdev.ngb.se/portal/rs/ipt_germplasm.xml 

  3. Add this XML definition to a copy of extensions.json 
  Starting with " http://gbrds.gbif.org/registry/ipt/extensions.json " 
  Add text {"title":"Germplasm","url":" http://wwwdev.ngb.se/portal/rs/ipt_germplasm.xml ","id":9} 
  Saved as: http://wwwdev.ngb.se/portal/rs/ipt/extensions.json 

  4. Modify ipt.properties 
  PATH_TO_TOMCAT_WEBAPPS/ipt/WEB-INF/classes/ipt.properties 
  Line 7: "registryUrl= http://gbrds.gbif.org/registry " modified to 
  "registryUrl= http://wwwdev.ngb.se/portal/rs " 

  5. Restart Tomcat (reload ipt) 
  The new ipt.properties settings does not seem to be loaded without restart of tomcat 

  6. Add/load the new extension to IPT 
  Login with admin permissions 
  http://your_server_name/ipt/admin/extensions.html 
  Click "Check for all available extensions" 
  Select the name of the new extension 
  Select "install" and "Done" 

  7. Mapping of terms to your data 
  Select the new extension under "Add New Mapping" 
  Click "Add" button to the right 
  Select and map terms 

  I selected the "publish" button and hope that this will not cause this dataset to be registered at the GBIF registry...? Without selecting the "publish" button the dataset is not listed from the menu item "explore". I was missing the option to publish a dataset under this "explore", but not not to register with the GBIF registry...? Should there not be one more step to this procedure? 

  Cheers 
  Dag Terje 

  PS. I have noticed that the right side context menu is very sensitive to missing data (Navigate Taxonomy, Navigate Geography). If some of the higher taxonomic levels is missing I get an error message instead of the context menu. For example I did not have phylum for the genebank dataset and the error message on "Navigate Taxonomy" untill I added data for this term. Similar I did not have continent causing the "Navigate Geography" context menu to start at country. When continent and/or country is missing, the "Navigate Geography" context menu creates a top node for the lowest level available - sometimes even the coordinate is displayed as the top node here. Would be cute if the IPT could default to "unknown continent", "unknown country"; "unknown phylum", ... or something similar. 

  PPS. NB! This DRAFT extension for germplasm is still under development and I am sure that I still duplicate some terms from DarwinCore or other extensions! I did not yet work much with the terms for trait observations on germplasm, but added most of the terms from the new proposed draft standard for these data for the European genebank community. 

  From: "Markus Döring (GBIF)" [mailto: mdoering at gbif.org ] 
  To: Dag Terje Filip Endresen [mailto: dag.endresen at nordgen.org ], Eli Agbayani [mailto: eagbayani at eol.org ] 
  Cc: ipt at lists.gbif.org 
  Sent: Wed, 13 May 2009 00:05:33 +0200 
  Subject: Re: [IPT] IPT connection problem 

  Dag, Eli, 
  I've checked the IPT code again for the extension registry URL and I was actually wrong. 
  The current version *does* make use of the ipt.properties settings. 

  The standard GBIF URL that is used in the ipt.properties should be: 
  registryUrl= http://gbrds.gbif.org/registry 

  The registry then returns a JSON list of all known extensions located at: 
  http://gbrds.gbif.org/registry /ipt/extensions.json 

  So to test your own extensions you can modify this json list to include entries for your own (pointing to your extension xml definitions hosted on some URL), upload that json file to some http server and modify the ipt.properties setting to point to your registry, e.g.: 

  registryUrl= http://dev.eol.org/ipt 

  that will finally point to the file 
  http://dev.eol.org/ipt/ipt/extensions.json 

  To get everything working this registry base URL should also deal with other things, e.g. the registry of resources, organisations, services, etc. 
  But in order to test the extensions this should still work. 

  If you give it a try, please report your findings back to this list. 

  cheers, 
  Markus 

  On May 12, 2009 , at 6:05 PM, Markus Döring (GBIF) wrote: 

  PS. I agree with Jörg that a convenient way to keep the old data directory when upgrading the IPT would be cute. 

  This is indeed the case. We will aim wherever possible to make upgrading as simple as it can be. 
  For bug fix releases, such as this snapshot, with no structural change the data directory can be migrated directly. In future releases, we might require a "one time" migration task be run. I anticipate the first one being required when the DwC standard is ratified. 

  if you look into your ipt.properties file you can modify the following settings: 

  dataDir= ${ipt.datadir} 
  registryUrl= ${ipt.registry.url} 

  the dataDir should be an absolute path pointing to your data dir (defaults to the ipt webapp data subfolder). 
  the registry URL can point to a development registry - unfortunately the extensions and vocabs currently dont use this setting, only the UDDI style registration of resources and organisations. We will make sure that the IPT uses this setting also for the extension registration, so you can easily point it to an alternative extension site for testing. 

  markus 

---------------------------------------------------------------------
-------- Some more useful instructions from Tim -----------
---------------------------------------------------------------------

  _____  

From: Tim Robertson [mailto:trobertson at gbif.org]
To: Dag Terje Filip Endresen [mailto:dag.endresen at nordgen.org]
Cc: ipt at lists.gbif.org
Sent: Tue, 12 May 2009 15:11:10 +0200
Subject: Re: [IPT] IPT connection problem

Hi all, 

PS. I agree with Jörg that a convenient way to keep the old data directory when upgrading the IPT would be cute.

This is indeed the case.  We will aim wherever possible to make upgrading as simple as it can be.  
For  bug fix releases, such as this snapshot, with no structural change the  data directory can be migrated directly.   In future releases, we might  require a "one time" migration task be run.  I anticipate the first one  being required when the DwC standard is ratified.

PPS.  And a convinient way to add a thematic extension to the DwC (for  testing and demonstration) would also be very useful. I am thinking of  the draft DwC germplasm extension - as well as a new draft for  germplasm trait observation data I have started to work on. I assume  this perhaps could be done as an addition in the IPT code somewhere  around "WEB-INF/classes/org/gbif/provider/model/" ...?

A good suggestion.  We should look at setting up some sandbox repository for exactly this purpose.

Dag,  for testing and demonstration you could do the following, but please be  very careful not to register these datasets with GBIF.
- The IPT will call (http://gbrds.gbif.org/registry/ipt/extensions.json and http://gbrds.gbif.org/registry/ipt/thesauri.json) to get the extensions and the thesauri.  
-  The response is fairly self explanatory, and for a test, you could  override gbrds.gbif.org in you host file, and put up a different  version locally for example referencing your test extensions.  
- Then simply update your extensions through the IPT.
-  Sorry, not the easiest work around, but might get you able to  demonstrate how you can start to build your community without waiting  for the sandbox repository.

Best wishes,

Tim

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.gbif.org/pipermail/ipt/attachments/20090527/a591a755/attachment.html 