<html><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Dear David,<div><br></div><div>Thank you very much for this thorough feedback, which will be logged as issues and addressed to improve the IPT. We are in a testing phase for a 2.0.2 release which has minor enhancements, so I would anticipate the majority of these being fixed for the 2.0.3 release.</div><div>It is reassuring to hear that you feel the IPT was easy to use and you don't consider these serious issues. </div><div><br></div><div>Thanks again,</div><div>Tim</div><div><br></div><div><br><div><div>On Apr 12, 2011, at 8:52 AM, David Remsen (GBIF) wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Guys - I received this message from one of the checklist award recipients. Fairly detailed set of IPT reviews. Some of these sure worthy of new issues.<div><br></div><div>DR<br><div><br><div>Begin forwarded message:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" color="#000000" style="font: 12.0px Helvetica; color: #000000"><b>From: </b></font><font face="Helvetica" size="3" style="font: 12.0px Helvetica">David Eades <<a href="mailto:dceades@illinois.edu">dceades@illinois.edu</a>></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" color="#000000" style="font: 12.0px Helvetica; color: #000000"><b>Date: </b></font><font face="Helvetica" size="3" style="font: 12.0px Helvetica">April 11, 2011 9:21:38 PM GMT+02:00</font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" color="#000000" style="font: 12.0px Helvetica; color: #000000"><b>To: </b></font><font face="Helvetica" size="3" style="font: 12.0px Helvetica">"'David Remsen \(GBIF\)'" <<a href="mailto:dremsen@gbif.org">dremsen@gbif.org</a>></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" color="#000000" style="font: 12.0px Helvetica; color: #000000"><b>Subject: </b></font><font face="Helvetica" size="3" style="font: 12.0px Helvetica"><b>IPT feedback notes</b></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font face="Helvetica" size="3" color="#000000" style="font: 12.0px Helvetica; color: #000000"><b>Reply-To: </b></font><font face="Helvetica" size="3" style="font: 12.0px Helvetica"><<a href="mailto:dceades@illinois.edu">dceades@illinois.edu</a>></font></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; min-height: 14px; "><br></div> </div><div>Dear David,<br><br>This message is sent in response to the request for feedback about our use<br>of IPT.<br><br>Overall, the IPT 2.0.1-r3048 is an easy to use and intuitive interface, it<br>does a good job at processing our tarball datasets and comes with a handy<br>interface to fill out the EML metadata. The installation was done<br>independently by two persons in two countries. The following is a list of<br>the issues mentioned. None of the comments indicate serious problems with a<br>good product. This information is about three weeks old. Some issues may<br>have already been fixed.<br><br>1. Updating resources from archives: We couldn't find a way in which an<br>updated archive dataset can be used to replace the existing resource<br>data. The only available method using the IPT web interfaces seems to<br>be by uploading updated versions of the TXT files one by one (using<br>exactly the same file names that were used when creating the resource<br>for the first time), and then press "publish". One alternative is<br>deleting the resource and re-create it from the updated archive, but<br>this option isn't convenient for registered resources. The other<br>alternative is to just replace the TXT files by writing to the file<br>system directly ([data_directory]/resources/[resource_name]/sources/*.txt)<br>and then<br>press "publish", but is this a supported method?<br><br>2. Unstable "main" administrator: When the IPT is set up for the first<br>time, there is an administrator user created and it is the one who<br>appears in messages like "If you don't have an account yet, please ask<br>your IPT administrator to create one for you.: admin_name<br><mail_address>". However, if later a new user is added with (or an<br>existing user promoted to) an administration role and its mail address<br>alphabetically precedes the current "main" administrator, then this<br>new administrator starts to appear in the aforementioned messages<br>instead of the original one.<br><br>3. Failed processing archives containing some empty TXT files: When an<br>archive is uploaded at the resource creation stage, the IPT immediately<br>begins processing it. However, as soon as an empty file is found, the<br>process stops without finish processing the remaining TXT files (but<br>creating the resource with the files it was able to process), and<br>suggesting to run the archive through the validator (which process<br>such files without complaint.) The workaround for this is to have<br>such files contain at least a newline character, but when the resource<br>is published, those files are generated completely empty once more,<br>and when the generated archive is submitted back to the IPT for<br>resource creation, it fails again (despite its generation by the IPT<br>itself.) We think such files should be accepted by the IPT, as them<br>should be considered empty (0 rows) tables. The reason we have some of<br>our datasets with empty extension files is because all of them are<br>generated from equally-capable databases all maintained by the same<br>software (SpeciesFile), but some of them don't have records for all<br>kinds of information yet (like common names for instance), however in<br>future updates such records may begin to appear.<br><br>4. Sometimes the file sizes are incorrectly reported: When managing a<br>resource ( /manage/resource.do?r=resource_name ), some file sizes are<br>incorrectly reported. Example: "vernacular [file] 0 bytes, 1 rows, 3<br>columns." The file contains this one line only, terminated by a<br>newline character: "1";"webspinners";"English". This doesn't seem to<br>be really a problem, but we make the comment just in case it's<br>actually more than a "cosmetic" issue. (The published archive the IPT<br>generates still contains the example line.)<br><br>5. Validator (<a href="http://tools.gbif.org/dwca-validator/">http://tools.gbif.org/dwca-validator/</a> ) and IPT out of<br>sync: This issue occurred when migrating from the release candidate 3<br>to the 2.0.1-r3048 version. Previously, the types and specimens<br>extension was of rowtype "<a href="http://rs.gbif.org/terms/1.0/Specimen">http://rs.gbif.org/terms/1.0/Specimen</a>",<br>however now it is no longer recognized by the IPT and<br>"<a href="http://rs.gbif.org/terms/1.0/TypesAndSpecimen">http://rs.gbif.org/terms/1.0/TypesAndSpecimen</a>" must be used instead<br>(which in turn is not recognized by the validator and this new<br>rowtype doesn't have the identificationRemarks term.) Also, with the<br>species profile extension we are using the livingPeriod term, but this<br>one is not recognized by the validator while it is accepted by the<br>IPT.<br><br>6. Explicitly set vocabularies in meta.xml are not preserved by the<br>IPT: For several of our columns we are using the vocabularies the IPT<br>comes with and we explicitly advertise that fact in our source<br>meta.xml (for example: <field index="6"<br>term="<a href="http://rs.tdwg.org/dwc/terms/taxonRank">http://rs.tdwg.org/dwc/terms/taxonRank</a>"<br>vocabulary="<a href="http://rs.gbif.org/vocabulary/gbif/rank.xml">http://rs.gbif.org/vocabulary/gbif/rank.xml</a>"/>.) However,<br>when the resource is published, the archive generated by the IPT<br>removes this information. We think that perhaps the IPT should not<br>remove the vocabularies from meta.xml and maybe for those vocabularies<br>the IPT is aware of it should either complain when a row violates the<br>vocabulary or else set the column value to NULL (like the automap option<br>of the value translation page does?)<br><br>7. When the id and a term in the core file are set to the same column<br>index the IPT generates a separate column with a duplicated value: Our<br>meta.xml defines the id and taxonID in the same way as in the example<br>at <a href="http://rs.tdwg.org/dwc/terms/guides/text/index.htm#implement">http://rs.tdwg.org/dwc/terms/guides/text/index.htm#implement</a> (id<br>and taxonID both mapped to column 0.) However, when we publish the<br>resource, the IPT keeps the id at column 0, but also creates a new<br>column for taxonID containing the same value column 0 has. This is not<br>much of a problem, but in doing so it takes more space than necessary.<br><br>8. The documentation is heavily biased toward Linux (IPTServerPreparation).<br><br>No support of Windows Server environments was apparent. We elected to<br>implement <br>a standalone Linux server, rather than try to use Microsoft interoperability<br>tools <br>under Windows Server.<br><br>It seems that the presumption is that the installer knows the <br>names and locations of the relevant files, which are often not explicitly<br>stated.<br><br>No step-by-step instruction is available. The section related <br>to Tomcat comprises about 9 sentences. The time it required to <br>research and implement the actual steps was several hours, shortened <br>by liberal use of a Linux-familiar collaborator.<br><br>9. No criteria for selection of the server infrastructure are <br>provided. This appears to be oriented toward making a given <br>pre-existing web server installation adapt to the IPT. Since in our <br>case, we created a server from the ground up, we selected an <br>Ubuntu/Tomcat approach for no better reason than we had some minimal <br>familiarity with them, and access to greater expertise.<br><br>10. Maintenance/version upgrade process (Starting Over) is <br>similarly sparsely detailed, and described in general terms. Since <br>in-house expertise is oriented toward WS2008 and IIS 7, the management <br>and maintenance of this web application in the selected environment is <br>by no means obvious. The initial setup eventually worked, but the <br>structure of the web server and relationships among the programs and <br>files was essentially mysterious, without reference material like: how <br>web applications are managed under Tomcat.<br><br>11. Pictorial information is limited to post-installation data <br>management user operations. Installation-related graphics would have <br>been useful. A disclaimer emphasizing the value of Linux expertise <br>would prevent the mistaken assumption that this is a cookbook <br>installation. This section of the documentation appears to be where <br>the effort was concentrated, but this was not the area of my activity<br>(setting up the server and site).<br><br>Hope this information is useful.<br><br><br></div></blockquote></div><br></div></div></blockquote></div><br></div></body></html>