- IPT - lists.gbif.org

Can you get Metadata For resources that are alredy being served live with Tapir.
by Khwezi Flatela 14 Apr '11

14 Apr '11

HI I Would like to find out if I can get my metadata information on a tapir instance that we have running to feed into the IPT instance and also include the resources listed on that tapir instance. Thank you Khwezi Flatela South African Biodiversity Information Facility (SABIF) SABIF Developer Biodiversity Information Management (BIM) South African National Biodiversity Institute (SANBI) ph: +27 21 799 8699 fax: +27 86 520 6990 cell: +27 82 224 5905 http://www.sanbi.org http://www.sabif.ac.za k.flatela(a)sanbi.org.za<mailto:flatela@sanbi.org> ________________________________ VISIT OUR NEW WEBSITE ----BIODIVERSITY ADVISOR---- ON HTTP://BIODIVERSITYADVISOR.SANBI.ORG/ Think before you print. Please consider the environment before printing this email. NOTE: This e-mail message and any attachments are intended for the addressee only, and contain confidential information that may be legally privileged and/or the subject of copyright that is protected by law. Any unauthorised usage, disclosure, alteration or dissemination is prohibited. SANBI accepts no responsibility for loss, data corruption or mail that fails to reach its intended destination. Furthermore, SANBI cannot assure the integrity of this communication nor guarantee that it is free of errors, viruses, interception or interference. No liability, whether direct or indirect, is accepted by SANBI or the sender. Any view or opinion expressed in this message may not necessarily be that of SANBI or SANBI Management. SANBI reserves the right to monitor all e-mail communication. The disclaimer is located at http://www.sanbi.org/index.php?option=com_content&view=article&id=916:-sanb…

2 1

IPT page title
by Deependra Tandukar 13 Apr '11

13 Apr '11

Hello, I have configured IPT in Linux and running. I want to customize the page and also change the title of the page - right now it is IPT, how do I do this? Best regards, Deependra

2 1

Re: [IPT] IPT feedback notes
by Tim Robertson (GBIF) 12 Apr '11

12 Apr '11

Dear David, Thank you very much for this thorough feedback, which will be logged as issues and addressed to improve the IPT. We are in a testing phase for a 2.0.2 release which has minor enhancements, so I would anticipate the majority of these being fixed for the 2.0.3 release. It is reassuring to hear that you feel the IPT was easy to use and you don't consider these serious issues. Thanks again, Tim On Apr 12, 2011, at 8:52 AM, David Remsen (GBIF) wrote: > Guys - I received this message from one of the checklist award > recipients. Fairly detailed set of IPT reviews. Some of these sure > worthy of new issues. > > DR > > Begin forwarded message: > >> From: David Eades <dceades(a)illinois.edu> >> Date: April 11, 2011 9:21:38 PM GMT+02:00 >> To: "'David Remsen \(GBIF\)'" <dremsen(a)gbif.org> >> Subject: IPT feedback notes >> Reply-To: <dceades(a)illinois.edu> >> >> Dear David, >> >> This message is sent in response to the request for feedback about >> our use >> of IPT. >> >> Overall, the IPT 2.0.1-r3048 is an easy to use and intuitive >> interface, it >> does a good job at processing our tarball datasets and comes with a >> handy >> interface to fill out the EML metadata. The installation was done >> independently by two persons in two countries. The following is a >> list of >> the issues mentioned. None of the comments indicate serious >> problems with a >> good product. This information is about three weeks old. Some >> issues may >> have already been fixed. >> >> 1. Updating resources from archives: We couldn't find a way in >> which an >> updated archive dataset can be used to replace the existing resource >> data. The only available method using the IPT web interfaces seems to >> be by uploading updated versions of the TXT files one by one (using >> exactly the same file names that were used when creating the resource >> for the first time), and then press "publish". One alternative is >> deleting the resource and re-create it from the updated archive, but >> this option isn't convenient for registered resources. The other >> alternative is to just replace the TXT files by writing to the file >> system directly ([data_directory]/resources/[resource_name]/sources/ >> *.txt) >> and then >> press "publish", but is this a supported method? >> >> 2. Unstable "main" administrator: When the IPT is set up for the >> first >> time, there is an administrator user created and it is the one who >> appears in messages like "If you don't have an account yet, please >> ask >> your IPT administrator to create one for you.: admin_name >> <mail_address>". However, if later a new user is added with (or an >> existing user promoted to) an administration role and its mail >> address >> alphabetically precedes the current "main" administrator, then this >> new administrator starts to appear in the aforementioned messages >> instead of the original one. >> >> 3. Failed processing archives containing some empty TXT files: When >> an >> archive is uploaded at the resource creation stage, the IPT >> immediately >> begins processing it. However, as soon as an empty file is found, the >> process stops without finish processing the remaining TXT files (but >> creating the resource with the files it was able to process), and >> suggesting to run the archive through the validator (which process >> such files without complaint.) The workaround for this is to have >> such files contain at least a newline character, but when the >> resource >> is published, those files are generated completely empty once more, >> and when the generated archive is submitted back to the IPT for >> resource creation, it fails again (despite its generation by the IPT >> itself.) We think such files should be accepted by the IPT, as them >> should be considered empty (0 rows) tables. The reason we have some >> of >> our datasets with empty extension files is because all of them are >> generated from equally-capable databases all maintained by the same >> software (SpeciesFile), but some of them don't have records for all >> kinds of information yet (like common names for instance), however in >> future updates such records may begin to appear. >> >> 4. Sometimes the file sizes are incorrectly reported: When managing a >> resource ( /manage/resource.do?r=resource_name ), some file sizes are >> incorrectly reported. Example: "vernacular [file] 0 bytes, 1 rows, 3 >> columns." The file contains this one line only, terminated by a >> newline character: "1";"webspinners";"English". This doesn't seem to >> be really a problem, but we make the comment just in case it's >> actually more than a "cosmetic" issue. (The published archive the IPT >> generates still contains the example line.) >> >> 5. Validator (http://tools.gbif.org/dwca-validator/ ) and IPT out of >> sync: This issue occurred when migrating from the release candidate 3 >> to the 2.0.1-r3048 version. Previously, the types and specimens >> extension was of rowtype "http://rs.gbif.org/terms/1.0/Specimen", >> however now it is no longer recognized by the IPT and >> "http://rs.gbif.org/terms/1.0/TypesAndSpecimen" must be used instead >> (which in turn is not recognized by the validator and this new >> rowtype doesn't have the identificationRemarks term.) Also, with the >> species profile extension we are using the livingPeriod term, but >> this >> one is not recognized by the validator while it is accepted by the >> IPT. >> >> 6. Explicitly set vocabularies in meta.xml are not preserved by the >> IPT: For several of our columns we are using the vocabularies the IPT >> comes with and we explicitly advertise that fact in our source >> meta.xml (for example: <field index="6" >> term="http://rs.tdwg.org/dwc/terms/taxonRank" >> vocabulary="http://rs.gbif.org/vocabulary/gbif/rank.xml"/>.) However, >> when the resource is published, the archive generated by the IPT >> removes this information. We think that perhaps the IPT should not >> remove the vocabularies from meta.xml and maybe for those >> vocabularies >> the IPT is aware of it should either complain when a row violates the >> vocabulary or else set the column value to NULL (like the automap >> option >> of the value translation page does?) >> >> 7. When the id and a term in the core file are set to the same column >> index the IPT generates a separate column with a duplicated value: >> Our >> meta.xml defines the id and taxonID in the same way as in the example >> at http://rs.tdwg.org/dwc/terms/guides/text/index.htm#implement (id >> and taxonID both mapped to column 0.) However, when we publish the >> resource, the IPT keeps the id at column 0, but also creates a new >> column for taxonID containing the same value column 0 has. This is >> not >> much of a problem, but in doing so it takes more space than >> necessary. >> >> 8. The documentation is heavily biased toward Linux >> (IPTServerPreparation). >> >> No support of Windows Server environments was apparent. We elected >> to >> implement >> a standalone Linux server, rather than try to use Microsoft >> interoperability >> tools >> under Windows Server. >> >> It seems that the presumption is that the installer knows the >> names and locations of the relevant files, which are often not >> explicitly >> stated. >> >> No step-by-step instruction is available. The section related >> to Tomcat comprises about 9 sentences. The time it required to >> research and implement the actual steps was several hours, shortened >> by liberal use of a Linux-familiar collaborator. >> >> 9. No criteria for selection of the server infrastructure are >> provided. This appears to be oriented toward making a given >> pre-existing web server installation adapt to the IPT. Since in our >> case, we created a server from the ground up, we selected an >> Ubuntu/Tomcat approach for no better reason than we had some minimal >> familiarity with them, and access to greater expertise. >> >> 10. Maintenance/version upgrade process (Starting Over) is >> similarly sparsely detailed, and described in general terms. Since >> in-house expertise is oriented toward WS2008 and IIS 7, the >> management >> and maintenance of this web application in the selected environment >> is >> by no means obvious. The initial setup eventually worked, but the >> structure of the web server and relationships among the programs and >> files was essentially mysterious, without reference material like: >> how >> web applications are managed under Tomcat. >> >> 11. Pictorial information is limited to post-installation data >> management user operations. Installation-related graphics would have >> been useful. A disclaimer emphasizing the value of Linux expertise >> would prevent the mistaken assumption that this is a cookbook >> installation. This section of the documentation appears to be where >> the effort was concentrated, but this was not the area of my activity >> (setting up the server and site). >> >> Hope this information is useful. >> >> >

1 0

structured text files
by Wolfgang Lorenz 11 Apr '11

11 Apr '11

Dear All, what everyone can do easily (no matter how data are originally entered in excel or whatsoever format) is export datasets as simple structured text files. Then we just need a central server with IPT installed (e.g., at country nodes?) where the 'little data holders' can submit text files for publication via GBIF. Are there plans to offer such things? Just thinking of those who cannot afford or don't know (like myself) how to set up the IPtoolkit on a server. Cheers, Wolfgang -------------------------------- Wolfgang Lorenz, Tutzing, Germany

2 1

can IPT2 handle big datasets?
by Kyle Braak (GBIF) 10 Apr '11

10 Apr '11

Dear IPT mailing list, This afternoon we conducted a little test to see whether the IPT2 can handle publishing a big dataset from a database. In the test we used a MySQL database, and successfully generated an archive with 24.000.000 records in about 50 minutes! This was run on a Tomcat server with 256MB memory. http://ipt.gbif.org/resource.do?r=bigdbtest Previously IPT1 had serious problems with such large datasets, but during IPT2 development special care was taken to be ensure that they could be handled gracefully. The way it is done now, is that the result sets from the database are streamed to the file system where they are written (about a 1000 records per result set) so there is no memory burden at all. This is one of the reasons why the IPT2 is not as feature rich as the IPT1 was. Best wishes, Kyle Braak Programmer Global Biodiversity Information Facility Secretariat Universitetsparken 15, DK-2100 Copenhagen, Denmark Tel: +45-35321479 Fax: +45-35321480 http://community.gbif.org/pg/profile/kbraak URL: http://www.gbif.org

4 3

Re: [IPT] IPT issue
by Tim Robertson (GBIF) 18 Mar '11

18 Mar '11

Dear Dimi, Where did you place your data directory? It sounds like you may have placed it in a location that the server deletes (like /tmp for example). Please can you use the ipt users lists for this kind of discussion to ensure everyone benefits. Thanks, Tim On Mar 18, 2011, at 2:34 PM, BROSENS, Dimitri wrote: > Hi Tim, > > We are trying to have an IPT up and running in the institute. When > we reboot the server we lose everything what we entered in IPT and > we need to go through the setup process again. Are you aware of > problems like this, is there a solution? > > Thnx, > Dimi > > Update Your Profile http://biobel.biodiversity.be/user/login > Dimitri BROSENS > Belgian Biodiversity Platform > Research Institute for Nature and Forest > Kliniekstraat 25 > B-1070 Brussels > Belgium > gsm +32 497 35 47 96 > tel +32 (0)2 525 02 17 > fax +32 (0)2 558 18 05 > dimitri.brosens(a)inbo.be > www.biodiversity.be -- http://belgianbiodiversity.tumblr.com/ > www.inbo.be >

3 3

digir/tapirlink useful features ... Howto in IPT?
by Herbario SANT 17 Feb '11

17 Feb '11

Thanks for your answer Markus. What I am trying to figure out is the best way to reconfigure an IPT when for some reason the server stops working (we buy a new computer, we migrate from/to windows to/from linux, or something alike). How to restore everything fastly and safely? I used to keep copies of some xml files when using digir/tapirlink, and this was a reasonably easy task to do. But what about IPT? One more question regarding digir/tapirlink versus IPT: It was possible to launch direct xml queries to them, so I could construct a web server to show inventories or specific searches of an specific data provider. How can I do this with IPT? I haven't looked in detail but I guess the only way to get data from IPT is downloading full datasets Nobody else has requested this feature? Would it be possible to have something like this built in IPT, without reducing IPT performance a lot? (even with a different technic but similar client interface services): http://tapirlink.berkeley.edu/tapir_client.php http://algae.manoa.hawaii.edu/database/lib/tapirlink/www/tapir_client.php Thanks for your answers David -- David García San León Herbario SANT Facultade de Farmacia - Laboratorio de Botánica Universidade de Santiago de Compostela 15782 - Galicia (Spain) http://www.usc.es/herbario Tel. +34 881815022 Fax +34 981594912

9 9

IPT v2.0 is ready
by Oliver Meyn 17 Feb '11

17 Feb '11

Dear IPT users, The GBIF Secretariat are pleased to announce that following three rounds of formal testing, a stable release of the Integrated Publishing Toolkit Version 2.0 (IPT v2.0) is now available for download from http://code.google.com/p/gbif-providertoolkit/. The IPT v2.0 enables simple biodiversity data publishing to the internet according to global standards. We would like to encourage early adopters to install/upgrade this latest release, particularly if you have an existing IPT installation. Updated information about this version of the IPT and instructions on how to join the IPT mailing lists is available on the GBIF Website: http://www.gbif.org/informatics/infrastructure/publishing/. In addition: - The live user manual can be found online [http://code.google.com/p/gbif-providertoolkit/wiki/IPT2ManualNotes?tm=6] All users are invited to contribute and update this manual. We will soon start a process to ensure that versions are available in some of the major languages. - We kindly ask those of you that will install and use this new version to report any issues on the project site [http://code.google.com/p/gbif-providertoolkit/issues/list] and address questions to the IPT-users mailing list [http://lists.gbif.org/mailman/listinfo/ipt] - The project roadmap outlining the technical developments of the software [http://code.google.com/p/gbif-providertoolkit/wiki/Roadmap] will provide a schedule of any minor version releases planned for the future. This new version of the IPT will be officially launched in October 2011 at the 18th GBIF Governing Board Meeting in Buenos Aires, Argentina. The GBIF Secretariat will work closely with the Nodes Strategy Group and key IPT users to ensure that a comprehensive uptake plan is ready for the IPT v2.0 launch. This plan will include a technology transition guide, a training programme, final user manuals and technical documentation, an expert network and a distributed helpdesk. Thank you once again for all your help in testing and contributing to the IPT! With best regards, Oliver, on behalf of the IPT development team and the GBIF Secretariat -- Oliver Meyn Software Developer Global Biodiversity Information Facility (GBIF) +45 35 32 15 12 http://www.gbif.org

1 0

IPT Linux configuration
by Deependra Tandukar 04 Feb '11

04 Feb '11

Hello, Please let me know the requirement/configuration for IPT in linux - space, dependent packages, memory, etc. Best regards, [cid:image001.jpg@01CBC4AD.A3A7CAA0]<http://www.icimod.org/> Deependra Tandukar Knowledge Management and Web Specialist, Knowledge Management International Centre for Integrated Mountain Development GPO Box 3226, Khumaltar, Lalitpur, Kathmandu, Nepal. Tel +977-1-5003222 Ext 154 Fax +977-1-5003277 Web www.icimod.org<http://www.icimod.org/> Connect to ICIMOD: [cid:image002.jpg@01CBC4AD.A3A7CAA0] <http://www.facebook.com/icimod> [cid:image003.jpg@01CBC4AD.A3A7CAA0] <http://www.twitter.com/icimod> [cid:image004.jpg@01CBC4AD.A3A7CAA0] <http://www.youtube.com/icimod> [cid:image005.jpg@01CBC4AD.A3A7CAA0] <http://www.icimod.org/enews>

2 1

GBIF registration options // Validate
by Aurelio Sanabria 03 Feb '11

03 Feb '11

Hello, I'm trying to change the data provider from DiGIR to IPT. Because I need to shut down the machine hosting the DiGIR data provider (it's almost dying). The problem comes when I try to validate the URL, in the "GBIF registration options" page. The current IPT instance can be accessed using this URL http://lucina.inbio.ac.cr/ipt/ Anyone knows if I'm missing something? PS: The error the IPT instance its giving me: "Validation failed! Please check your internet connection. If connected, check the Base URL and the Proxy URL (if needed on this server)." -- Regards, Aurelio Sanabria http://informatics.inbio.ac.cr/

2 2