[IPT] All space-characters in metadata are replaced to plus-character

Fri Mar 9 13:45:03 CET 2012

Hi

I added the URIEncoding="UTF-8" to the connector and sadly it does not helped.

In POST-data spaces are coded using '+' in IPT. I did send POST data using %20-coding and it works. Normally I've used percent-based url-coding. 

BR,
Tero

From: Kyle Braak [GBIF] [mailto:kbraak at gbif.org] 
Sent: Friday, March 09, 2012 1:49 PM
To: Tero Mononen
Cc: ipt at lists.gbif.org; Tim Robertson [GBIF]
Subject: Re: [IPT] All space-characters in metadata are replaced to plus-character

Hi Tero, 

It's nice that we might be on the verge of solving this now..

When you have this working, I recommend an upgrade from 2.0.2 to 2.0.3. The upgrade is relatively painless. 

To do so, 

a) back up your current data directory to somewhere safe
b) download the 2.0.3 war file, and rename it to ipt.war
c) stop tomcat, remove ipt/ and ipt.war from the tomcat webapps folder, and add the new ipt.war file
d) start tomcat, and open up the IPT in a browser
e) presented with the 1st setup page, enter the location of your existing data directory
f) press continue to proceed, then visit the Administration page and press the button "Publish all resources" (this may take a few moments as it will republish all resources. 

Caution: There is a known bug with the upgrade where a taxonomic resources switches to type occurrence. I know that you have purely occurrence resources, but if this switch happens, just try the upgrade again but de-select the subtype (and be sure to save) for each resource before going through the steps. 

Best of luck,

Kyle
On Mar 9, 2012, at 12:43 PM, Tim Robertson [GBIF] wrote:

This looks like you are hitting this Tomcat6 / OpenSUSE bug:
http://stackoverflow.com/questions/9290745/tomcat-6-doesnt-decode-to-space-in-parameter-values

Can you please try the suggested fix?

Thanks,
Tim

On Mar 9, 2012, at 12:25 PM, Tero Mononen wrote:

Hi

Only very minor updates from openSUSE to tomcat6 and kernel 2.6.34.

Tomcat from version 6.0.24-5.14.1 to version 6.0.24-5.16.1.
Kernel 2.6.34.10-0.4.1 -> 2.6.34.10-0.6.1

BR,
Tero

From: Tim Robertson [GBIF] [mailto:trobertson at gbif.org] 
Sent: Friday, March 09, 2012 12:48 PM
To: Tero Mononen
Cc: Kyle Braak [GBIF]; ipt at lists.gbif.org
Subject: Re: [IPT] All space-characters in metadata are replaced to plus-character

Hi

Thank you, I made new issue (http://code.google.com/p/gbif-providertoolkit/issues/detail?id=845).

We have tested many browsers on Windows and Linux. I think the reproduction is hard because it did work on our system well and then something weird happened causing this.

Hi Tero,

What changed between it working well, and then this happening please?  OS upgrades, OS changes, Tomcat upgrades, Java updates etc

The *good* news is the IPT still works - the EML is correct when accessed through Tomcat.  Of course we want to fix the direct file access route, but the IPT is operating correctly through the IPT interface (e.g. http://db.digitarium.fi:8080/ipt/eml.do?r=nylander is correct )

Thanks,
Tim

BR,
Tero

From: Kyle Braak [GBIF] [mailto:kbraak at gbif.org] 
Sent: Friday, March 09, 2012 11:54 AM
To: Tero Mononen
Cc: ipt at lists.gbif.org; Tim Robertson [GBIF]
Subject: Re: [IPT] All space-characters in metadata are replaced to plus-character

Dear Tero, 

Can I kindly ask you to fill in a bug report on the IPT Project Site: http://code.google.com/p/gbif-providertoolkit/issues/entry

In addition to the IPT version, Java version, and Tomcat version, can you also specify the Operating System you are using (with version), as well as the browser  (with version) that you are using to fill in the metadata? Out of curiosity, have you tried entering metadata through different browsers (Chrome, Firefox, IE)? 

This will help us try and reproduce the problem ourselves so that if there is something wrong with the application we can fix it ASAP.

Thanks a lot,

Kyle
On Mar 9, 2012, at 10:40 AM, Tim Robertson [GBIF] wrote:

Thanks Tero

Very curious indeed.  The single input fields are represented with + but the description field which is a Textarea on the form is not.

We'll look more into it.

Cheers,
Tim

On Mar 9, 2012, at 10:15 AM, Tero Mononen wrote:

Hi

There is the original eml.xml. Is that the only file you are interested in?

BR,
Tero

On 03/09/2012 11:05 AM, Tim Robertson [GBIF] wrote:
Hi Tero,

How very strange.
Given this is public anyway (http://db.digitarium.fi:8080/ipt/eml.do?r=nylander) would you mind attaching the file from the filesystem please?  I'd like to look at the file encoding etc.

Thanks,
Tim

On Mar 9, 2012, at 9:44 AM, Tero Mononen wrote:

Dear Tim,

There's only public example (Title: William+Nylander+lichen+collection):
http://db.digitarium.fi:8080/ipt/resource.do?r=nylander

It's interesting, eml.xml on file system contains line:
<title xml:lang="eng">William+Nylander+lichen+collection</title>

But eml via IPT/Tomcat offers line:
<title xml:lang="eng">William Nylander lichen collection</title>

It's pure Tomcat without proxies.

BR,
Tero Mononen

From: Tim Robertson [GBIF] [mailto:trobertson at gbif.org] 
Sent: Friday, March 09, 2012 10:37 AM
To: Tero Mononen
Cc: ipt at lists.gbif.org
Subject: Re: [IPT] All space-characters in metadata are replaced to plus-character

Dear Tero,

Could you please provide a link to an example file?  We'll investigate, but this is not a behavior I have ever seen with a Tomcat application.

Do you run any proxies or mod_jk or anything like that?

Thanks,
Tim

On Mar 9, 2012, at 9:35 AM, Tero Mononen wrote:

Hi

We have used this IPT-setup without any problems till now.

System:
Version 2.0.2-r3306
Apache Tomcat/6.0.24
JVM 1.6.0_29-b1

Problem:
All space-characters in metadata are replaced to plus-character. For example title "foo bar car" is saved as "foo+bar+car". It affects in every text field in metadata.

If I include at least one plus-char in a text value, it prevents the space replacement. For example "foo bar car+dar" is saved as it is.

I think this is related to Java/Tomcat/urlencode. Do you have suggestions how to fix this problem?

Best Regards,
Tero Mononen

--
Digitarium
Digitisation Centre of the Finnish Museum of Natural History and the University of Eastern Finland Faculty of Science and Forestry, Joensuu Science Park Länsikatu 15 (P.O. Box 111)
FI-80101 Joensuu
http://www.digitarium.fi/

Tero Mononen, Analyst Programmer
tero.mononen at uef.fi
+358-50-5631788

_______________________________________________
IPT mailing list
IPT at lists.gbif.org
http://lists.gbif.org/mailman/listinfo/ipt

-- 
Digitarium
Digitisation Centre of the Finnish Museum of Natural History and the University of Eastern Finland Faculty of Science and Forestry, Joensuu Science Park Länsikatu 15 (P.O. Box 111)
FI-80101 Joensuu
http://www.digitarium.fi/

Tero Mononen, Analyst Programmer
tero.mononen at uef.fi
+358-50-5631788
<eml.xml>

_______________________________________________
IPT mailing list
IPT at lists.gbif.org
http://lists.gbif.org/mailman/listinfo/ipt