Dear Tim,

There's only public example (Title: William+Nylander+lichen+collection): http://db.digitarium.fi:8080/ipt/resource.do?r=nylander

It's interesting, eml.xml on file system contains line: <title xml:lang="eng">William+Nylander+lichen+collection</title>

But eml via IPT/Tomcat offers line: <title xml:lang="eng">William Nylander lichen collection</title>

It's pure Tomcat without proxies.

BR, Tero Mononen

From: Tim Robertson [GBIF] [mailto:trobertson@gbif.org] Sent: Friday, March 09, 2012 10:37 AM To: Tero Mononen Cc: ipt@lists.gbif.org Subject: Re: [IPT] All space-characters in metadata are replaced to plus-character

Dear Tero,

Could you please provide a link to an example file? We'll investigate, but this is not a behavior I have ever seen with a Tomcat application.

Do you run any proxies or mod_jk or anything like that?

Thanks, Tim

On Mar 9, 2012, at 9:35 AM, Tero Mononen wrote:

Hi

We have used this IPT-setup without any problems till now.

System: Version 2.0.2-r3306 Apache Tomcat/6.0.24 JVM 1.6.0_29-b1

Problem: All space-characters in metadata are replaced to plus-character. For example title "foo bar car" is saved as "foo+bar+car". It affects in every text field in metadata.

If I include at least one plus-char in a text value, it prevents the space replacement. For example "foo bar car+dar" is saved as it is.

I think this is related to Java/Tomcat/urlencode. Do you have suggestions how to fix this problem?

Best Regards, Tero Mononen

-- Digitarium Digitisation Centre of the Finnish Museum of Natural History and the University of Eastern Finland Faculty of Science and Forestry, Joensuu Science Park Länsikatu 15 (P.O. Box 111) FI-80101 Joensuu http://www.digitarium.fi/

Tero Mononen, Analyst Programmer tero.mononen@uef.fimailto:juha.lehtonen@uef.fi +358-50-5631788

_______________________________________________ IPT mailing list IPT@lists.gbif.orgmailto:IPT@lists.gbif.org http://lists.gbif.org/mailman/listinfo/ipt

Hi

There is the original eml.xml. Is that the only file you are interested in?

BR, Tero

On 03/09/2012 11:05 AM, Tim Robertson [GBIF] wrote: Hi Tero,

How very strange. Given this is public anyway (http://db.digitarium.fi:8080/ipt/eml.do?r=nylander) would you mind attaching the file from the filesystem please? I'd like to look at the file encoding etc.

Thanks, Tim

On Mar 9, 2012, at 9:44 AM, Tero Mononen wrote:

Dear Tim,

There’s only public example (Title: William+Nylander+lichen+collection): http://db.digitarium.fi:8080/ipt/resource.do?r=nylander

It’s interesting, eml.xml on file system contains line: <title xml:lang="eng">William+Nylander+lichen+collection</title>

But eml via IPT/Tomcat offers line: <title xml:lang="eng">William Nylander lichen collection</title>

It’s pure Tomcat without proxies.

BR, Tero Mononen

From: Tim Robertson [GBIF] [mailto:trobertson@gbif.org] Sent: Friday, March 09, 2012 10:37 AM To: Tero Mononen Cc: ipt@lists.gbif.orgmailto:ipt@lists.gbif.org Subject: Re: [IPT] All space-characters in metadata are replaced to plus-character

Dear Tero,

Could you please provide a link to an example file? We'll investigate, but this is not a behavior I have ever seen with a Tomcat application.

Do you run any proxies or mod_jk or anything like that?

Thanks, Tim

On Mar 9, 2012, at 9:35 AM, Tero Mononen wrote:

Hi

We have used this IPT-setup without any problems till now.

System: Version 2.0.2-r3306 Apache Tomcat/6.0.24 JVM 1.6.0_29-b1

Problem: All space-characters in metadata are replaced to plus-character. For example title “foo bar car” is saved as “foo+bar+car”. It affects in every text field in metadata.

If I include at least one plus-char in a text value, it prevents the space replacement. For example “foo bar car+dar” is saved as it is.

I think this is related to Java/Tomcat/urlencode. Do you have suggestions how to fix this problem?

Best Regards, Tero Mononen

-- Digitarium Digitisation Centre of the Finnish Museum of Natural History and the University of Eastern Finland Faculty of Science and Forestry, Joensuu Science Park Länsikatu 15 (P.O. Box 111) FI-80101 Joensuu http://www.digitarium.fi/

Tero Mononen, Analyst Programmer tero.mononen@uef.fimailto:juha.lehtonen@uef.fi +358-50-5631788

_______________________________________________ IPT mailing list IPT@lists.gbif.orgmailto:IPT@lists.gbif.org http://lists.gbif.org/mailman/listinfo/ipt

-- Digitarium Digitisation Centre of the Finnish Museum of Natural History and the University of Eastern Finland Faculty of Science and Forestry, Joensuu Science Park Länsikatu 15 (P.O. Box 111) FI-80101 Joensuu http://www.digitarium.fi/

Tero Mononen, Analyst Programmer tero.mononen@uef.fimailto:tero.mononen@uef.fi +358-50-5631788

Dear Tero,

Can I kindly ask you to fill in a bug report on the IPT Project Site: http://code.google.com/p/gbif-providertoolkit/issues/entry

In addition to the IPT version, Java version, and Tomcat version, can you also specify the Operating System you are using (with version), as well as the browser (with version) that you are using to fill in the metadata? Out of curiosity, have you tried entering metadata through different browsers (Chrome, Firefox, IE)?

This will help us try and reproduce the problem ourselves so that if there is something wrong with the application we can fix it ASAP.

Thanks a lot,

Kyle On Mar 9, 2012, at 10:40 AM, Tim Robertson [GBIF] wrote:

Thanks Tero

Very curious indeed. The single input fields are represented with + but the description field which is a Textarea on the form is not.

We'll look more into it.

Cheers, Tim

On Mar 9, 2012, at 10:15 AM, Tero Mononen wrote:

...

Hi

There is the original eml.xml. Is that the only file you are interested in?

BR, Tero

On 03/09/2012 11:05 AM, Tim Robertson [GBIF] wrote:

...

Hi Tero,

How very strange. Given this is public anyway (http://db.digitarium.fi:8080/ipt/eml.do?r=nylander) would you mind attaching the file from the filesystem please? I'd like to look at the file encoding etc.

Thanks, Tim

On Mar 9, 2012, at 9:44 AM, Tero Mononen wrote:

...

Dear Tim,

There’s only public example (Title: William+Nylander+lichen+collection): http://db.digitarium.fi:8080/ipt/resource.do?r=nylander

It’s interesting, eml.xml on file system contains line:

<title xml:lang="eng">William+Nylander+lichen+collection</title>

But eml via IPT/Tomcat offers line:

<title xml:lang="eng">William Nylander lichen collection</title>

It’s pure Tomcat without proxies.

BR, Tero Mononen

From: Tim Robertson [GBIF] [mailto:trobertson@gbif.org] Sent: Friday, March 09, 2012 10:37 AM To: Tero Mononen Cc: ipt@lists.gbif.org Subject: Re: [IPT] All space-characters in metadata are replaced to plus-character

Dear Tero,

Could you please provide a link to an example file? We'll investigate, but this is not a behavior I have ever seen with a Tomcat application.

Do you run any proxies or mod_jk or anything like that?

Thanks, Tim

On Mar 9, 2012, at 9:35 AM, Tero Mononen wrote:

Hi

We have used this IPT-setup without any problems till now.

System: Version 2.0.2-r3306 Apache Tomcat/6.0.24 JVM 1.6.0_29-b1

Problem: All space-characters in metadata are replaced to plus-character. For example title “foo bar car” is saved as “foo+bar+car”. It affects in every text field in metadata.

If I include at least one plus-char in a text value, it prevents the space replacement. For example “foo bar car+dar” is saved as it is.

I think this is related to Java/Tomcat/urlencode. Do you have suggestions how to fix this problem?

Best Regards, Tero Mononen

-- Digitarium Digitisation Centre of the Finnish Museum of Natural History and the University of Eastern Finland Faculty of Science and Forestry, Joensuu Science Park Länsikatu 15 (P.O. Box 111) FI-80101 Joensuu http://www.digitarium.fi/

Tero Mononen, Analyst Programmer tero.mononen@uef.fi +358-50-5631788

---

IPT mailing list IPT@lists.gbif.org http://lists.gbif.org/mailman/listinfo/ipt

-- Digitarium Digitisation Centre of the Finnish Museum of Natural History and the University of Eastern Finland Faculty of Science and Forestry, Joensuu Science Park Länsikatu 15 (P.O. Box 111) FI-80101 Joensuu http://www.digitarium.fi/

Tero Mononen, Analyst Programmer tero.mononen@uef.fi +358-50-5631788 <eml.xml>

---

IPT mailing list IPT@lists.gbif.org http://lists.gbif.org/mailman/listinfo/ipt

Hi

Thank you, I made new issue (http://code.google.com/p/gbif-providertoolkit/issues/detail?id=845).

We have tested many browsers on Windows and Linux. I think the reproduction is hard because it did work on our system well and then something weird happened causing this…

Hi Tero,

What changed between it working well, and then this happening please? OS upgrades, OS changes, Tomcat upgrades, Java updates etc

The *good* news is the IPT still works - the EML is correct when accessed through Tomcat. Of course we want to fix the direct file access route, but the IPT is operating correctly through the IPT interface (e.g. http://db.digitarium.fi:8080/ipt/eml.do?r=nylander is correct )

Thanks, Tim

BR, Tero

From: Kyle Braak [GBIF] [mailto:kbraak@gbif.org] Sent: Friday, March 09, 2012 11:54 AM To: Tero Mononen Cc: ipt@lists.gbif.org; Tim Robertson [GBIF] Subject: Re: [IPT] All space-characters in metadata are replaced to plus-character

Dear Tero,

Can I kindly ask you to fill in a bug report on the IPT Project Site: http://code.google.com/p/gbif-providertoolkit/issues/entry

In addition to the IPT version, Java version, and Tomcat version, can you also specify the Operating System you are using (with version), as well as the browser (with version) that you are using to fill in the metadata? Out of curiosity, have you tried entering metadata through different browsers (Chrome, Firefox, IE)?

This will help us try and reproduce the problem ourselves so that if there is something wrong with the application we can fix it ASAP.

Thanks a lot,

Kyle On Mar 9, 2012, at 10:40 AM, Tim Robertson [GBIF] wrote:

Thanks Tero

Very curious indeed. The single input fields are represented with + but the description field which is a Textarea on the form is not.

We'll look more into it.

Cheers, Tim

On Mar 9, 2012, at 10:15 AM, Tero Mononen wrote:

Hi

There is the original eml.xml. Is that the only file you are interested in?

BR, Tero

On 03/09/2012 11:05 AM, Tim Robertson [GBIF] wrote: Hi Tero,

How very strange. Given this is public anyway (http://db.digitarium.fi:8080/ipt/eml.do?r=nylander) would you mind attaching the file from the filesystem please? I'd like to look at the file encoding etc.

Thanks, Tim

On Mar 9, 2012, at 9:44 AM, Tero Mononen wrote:

Dear Tim,

There’s only public example (Title: William+Nylander+lichen+collection): http://db.digitarium.fi:8080/ipt/resource.do?r=nylander

It’s interesting, eml.xml on file system contains line:

<title xml:lang="eng">William+Nylander+lichen+collection</title>

But eml via IPT/Tomcat offers line:

<title xml:lang="eng">William Nylander lichen collection</title>

It’s pure Tomcat without proxies.

BR, Tero Mononen

From: Tim Robertson [GBIF] [mailto:trobertson@gbif.org] Sent: Friday, March 09, 2012 10:37 AM To: Tero Mononen Cc: ipt@lists.gbif.org Subject: Re: [IPT] All space-characters in metadata are replaced to plus-character

Dear Tero,

Could you please provide a link to an example file? We'll investigate, but this is not a behavior I have ever seen with a Tomcat application.

Do you run any proxies or mod_jk or anything like that?

Thanks, Tim

On Mar 9, 2012, at 9:35 AM, Tero Mononen wrote:

Hi

We have used this IPT-setup without any problems till now.

System: Version 2.0.2-r3306 Apache Tomcat/6.0.24 JVM 1.6.0_29-b1

Problem: All space-characters in metadata are replaced to plus-character. For example title “foo bar car” is saved as “foo+bar+car”. It affects in every text field in metadata.

If I include at least one plus-char in a text value, it prevents the space replacement. For example “foo bar car+dar” is saved as it is.

I think this is related to Java/Tomcat/urlencode. Do you have suggestions how to fix this problem?

Best Regards, Tero Mononen

-- Digitarium Digitisation Centre of the Finnish Museum of Natural History and the University of Eastern Finland Faculty of Science and Forestry, Joensuu Science Park Länsikatu 15 (P.O. Box 111) FI-80101 Joensuu http://www.digitarium.fi/

Tero Mononen, Analyst Programmer tero.mononen@uef.fi +358-50-5631788

---

IPT mailing list IPT@lists.gbif.org http://lists.gbif.org/mailman/listinfo/ipt

-- Digitarium Digitisation Centre of the Finnish Museum of Natural History and the University of Eastern Finland Faculty of Science and Forestry, Joensuu Science Park Länsikatu 15 (P.O. Box 111) FI-80101 Joensuu http://www.digitarium.fi/

Tero Mononen, Analyst Programmer tero.mononen@uef.fi +358-50-5631788 <eml.xml>

---

IPT mailing list IPT@lists.gbif.org http://lists.gbif.org/mailman/listinfo/ipt

This looks like you are hitting this Tomcat6 / OpenSUSE bug: http://stackoverflow.com/questions/9290745/tomcat-6-doesnt-decode-to-space-i...

Can you please try the suggested fix?

Thanks, Tim

On Mar 9, 2012, at 12:25 PM, Tero Mononen wrote:

Hi

Only very minor updates from openSUSE to tomcat6 and kernel 2.6.34.

Tomcat from version 6.0.24-5.14.1 to version 6.0.24-5.16.1. Kernel 2.6.34.10-0.4.1 -> 2.6.34.10-0.6.1

BR, Tero

From: Tim Robertson [GBIF] [mailto:trobertson@gbif.org] Sent: Friday, March 09, 2012 12:48 PM To: Tero Mononen Cc: Kyle Braak [GBIF]; ipt@lists.gbif.org Subject: Re: [IPT] All space-characters in metadata are replaced to plus-character

Hi

Thank you, I made new issue (http://code.google.com/p/gbif-providertoolkit/issues/detail?id=845).

We have tested many browsers on Windows and Linux. I think the reproduction is hard because it did work on our system well and then something weird happened causing this…

Hi Tero,

What changed between it working well, and then this happening please? OS upgrades, OS changes, Tomcat upgrades, Java updates etc

The *good* news is the IPT still works - the EML is correct when accessed through Tomcat. Of course we want to fix the direct file access route, but the IPT is operating correctly through the IPT interface (e.g. http://db.digitarium.fi:8080/ipt/eml.do?r=nylander is correct )

Thanks, Tim

BR, Tero

From: Kyle Braak [GBIF] [mailto:kbraak@gbif.org] Sent: Friday, March 09, 2012 11:54 AM To: Tero Mononen Cc: ipt@lists.gbif.org; Tim Robertson [GBIF] Subject: Re: [IPT] All space-characters in metadata are replaced to plus-character

Dear Tero,

Can I kindly ask you to fill in a bug report on the IPT Project Site: http://code.google.com/p/gbif-providertoolkit/issues/entry

In addition to the IPT version, Java version, and Tomcat version, can you also specify the Operating System you are using (with version), as well as the browser (with version) that you are using to fill in the metadata? Out of curiosity, have you tried entering metadata through different browsers (Chrome, Firefox, IE)?

This will help us try and reproduce the problem ourselves so that if there is something wrong with the application we can fix it ASAP.

Thanks a lot,

Kyle On Mar 9, 2012, at 10:40 AM, Tim Robertson [GBIF] wrote:

Thanks Tero

Very curious indeed. The single input fields are represented with + but the description field which is a Textarea on the form is not.

We'll look more into it.

Cheers, Tim

On Mar 9, 2012, at 10:15 AM, Tero Mononen wrote:

Hi

There is the original eml.xml. Is that the only file you are interested in?

BR, Tero

On 03/09/2012 11:05 AM, Tim Robertson [GBIF] wrote: Hi Tero,

How very strange. Given this is public anyway (http://db.digitarium.fi:8080/ipt/eml.do?r=nylander) would you mind attaching the file from the filesystem please? I'd like to look at the file encoding etc.

Thanks, Tim

On Mar 9, 2012, at 9:44 AM, Tero Mononen wrote:

Dear Tim,

There’s only public example (Title: William+Nylander+lichen+collection): http://db.digitarium.fi:8080/ipt/resource.do?r=nylander

It’s interesting, eml.xml on file system contains line:

<title xml:lang="eng">William+Nylander+lichen+collection</title>

But eml via IPT/Tomcat offers line:

<title xml:lang="eng">William Nylander lichen collection</title>

It’s pure Tomcat without proxies.

BR, Tero Mononen

From: Tim Robertson [GBIF] [mailto:trobertson@gbif.org] Sent: Friday, March 09, 2012 10:37 AM To: Tero Mononen Cc: ipt@lists.gbif.org Subject: Re: [IPT] All space-characters in metadata are replaced to plus-character

Dear Tero,

Could you please provide a link to an example file? We'll investigate, but this is not a behavior I have ever seen with a Tomcat application.

Do you run any proxies or mod_jk or anything like that?

Thanks, Tim

On Mar 9, 2012, at 9:35 AM, Tero Mononen wrote:

Hi

We have used this IPT-setup without any problems till now.

System: Version 2.0.2-r3306 Apache Tomcat/6.0.24 JVM 1.6.0_29-b1

Problem: All space-characters in metadata are replaced to plus-character. For example title “foo bar car” is saved as “foo+bar+car”. It affects in every text field in metadata.

If I include at least one plus-char in a text value, it prevents the space replacement. For example “foo bar car+dar” is saved as it is.

I think this is related to Java/Tomcat/urlencode. Do you have suggestions how to fix this problem?

Best Regards, Tero Mononen

-- Digitarium Digitisation Centre of the Finnish Museum of Natural History and the University of Eastern Finland Faculty of Science and Forestry, Joensuu Science Park Länsikatu 15 (P.O. Box 111) FI-80101 Joensuu http://www.digitarium.fi/

Tero Mononen, Analyst Programmer tero.mononen@uef.fi +358-50-5631788

---

IPT mailing list IPT@lists.gbif.org http://lists.gbif.org/mailman/listinfo/ipt

-- Digitarium Digitisation Centre of the Finnish Museum of Natural History and the University of Eastern Finland Faculty of Science and Forestry, Joensuu Science Park Länsikatu 15 (P.O. Box 111) FI-80101 Joensuu http://www.digitarium.fi/

Tero Mononen, Analyst Programmer tero.mononen@uef.fi +358-50-5631788 <eml.xml>

---

IPT mailing list IPT@lists.gbif.org http://lists.gbif.org/mailman/listinfo/ipt

Hi

I added the URIEncoding="UTF-8" to the connector and sadly it does not helped.

In POST-data spaces are coded using '+' in IPT. I did send POST data using %20-coding and it works. Normally I've used percent-based url-coding.

BR, Tero

From: Kyle Braak [GBIF] [mailto:kbraak@gbif.org] Sent: Friday, March 09, 2012 1:49 PM To: Tero Mononen Cc: ipt@lists.gbif.org; Tim Robertson [GBIF] Subject: Re: [IPT] All space-characters in metadata are replaced to plus-character

Hi Tero,

It's nice that we might be on the verge of solving this now..

When you have this working, I recommend an upgrade from 2.0.2 to 2.0.3. The upgrade is relatively painless.

To do so,

a) back up your current data directory to somewhere safe b) download the 2.0.3 war file, and rename it to ipt.war c) stop tomcat, remove ipt/ and ipt.war from the tomcat webapps folder, and add the new ipt.war file d) start tomcat, and open up the IPT in a browser e) presented with the 1st setup page, enter the location of your existing data directory f) press continue to proceed, then visit the Administration page and press the button "Publish all resources" (this may take a few moments as it will republish all resources.

Caution: There is a known bug with the upgrade where a taxonomic resources switches to type occurrence. I know that you have purely occurrence resources, but if this switch happens, just try the upgrade again but de-select the subtype (and be sure to save) for each resource before going through the steps.

Best of luck,

Kyle On Mar 9, 2012, at 12:43 PM, Tim Robertson [GBIF] wrote:

This looks like you are hitting this Tomcat6 / OpenSUSE bug: http://stackoverflow.com/questions/9290745/tomcat-6-doesnt-decode-to-space-i...

Can you please try the suggested fix?

Thanks, Tim

On Mar 9, 2012, at 12:25 PM, Tero Mononen wrote:

Hi   Only very minor updates from openSUSE to tomcat6 and kernel 2.6.34.   Tomcat from version 6.0.24-5.14.1 to version 6.0.24-5.16.1. Kernel 2.6.34.10-0.4.1 -> 2.6.34.10-0.6.1     BR, Tero   From: Tim Robertson [GBIF] [mailto:trobertson@gbif.org]  Sent: Friday, March 09, 2012 12:48 PM To: Tero Mononen Cc: Kyle Braak [GBIF]; ipt@lists.gbif.org Subject: Re: [IPT] All space-characters in metadata are replaced to plus-character   Hi   Thank you, I made new issue (http://code.google.com/p/gbif-providertoolkit/issues/detail?id=845).   We have tested many browsers on Windows and Linux. I think the reproduction is hard because it did work on our system well and then something weird happened causing this.   Hi Tero,   What changed between it working well, and then this happening please?  OS upgrades, OS changes, Tomcat upgrades, Java updates etc   The *good* news is the IPT still works - the EML is correct when accessed through Tomcat.  Of course we want to fix the direct file access route, but the IPT is operating correctly through the IPT interface (e.g. http://db.digitarium.fi:8080/ipt/eml.do?r=nylander%C2%A0is correct )   Thanks, Tim

BR, Tero   From: Kyle Braak [GBIF] [mailto:kbraak@gbif.org]  Sent: Friday, March 09, 2012 11:54 AM To: Tero Mononen Cc: ipt@lists.gbif.org; Tim Robertson [GBIF] Subject: Re: [IPT] All space-characters in metadata are replaced to plus-character   Dear Tero,    Can I kindly ask you to fill in a bug report on the IPT Project Site: http://code.google.com/p/gbif-providertoolkit/issues/entry   In addition to the IPT version, Java version, and Tomcat version, can you also specify the Operating System you are using (with version), as well as the browser  (with version) that you are using to fill in the metadata? Out of curiosity, have you tried entering metadata through different browsers (Chrome, Firefox, IE)?    This will help us try and reproduce the problem ourselves so that if there is something wrong with the application we can fix it ASAP.   Thanks a lot,   Kyle On Mar 9, 2012, at 10:40 AM, Tim Robertson [GBIF] wrote:

Thanks Tero   Very curious indeed.  The single input fields are represented with + but the description field which is a Textarea on the form is not.   We'll look more into it.   Cheers, Tim     On Mar 9, 2012, at 10:15 AM, Tero Mononen wrote:

Hi

There is the original eml.xml. Is that the only file you are interested in?

BR, Tero

On 03/09/2012 11:05 AM, Tim Robertson [GBIF] wrote: Hi Tero,   How very strange. Given this is public anyway (http://db.digitarium.fi:8080/ipt/eml.do?r=nylander) would you mind attaching the file from the filesystem please?  I'd like to look at the file encoding etc.   Thanks, Tim     On Mar 9, 2012, at 9:44 AM, Tero Mononen wrote:

Dear Tim,   There's only public example (Title: William+Nylander+lichen+collection): http://db.digitarium.fi:8080/ipt/resource.do?r=nylander   It's interesting, eml.xml on file system contains line: <title xml:lang="eng">William+Nylander+lichen+collection</title>   But eml via IPT/Tomcat offers line: <title xml:lang="eng">William Nylander lichen collection</title>   It's pure Tomcat without proxies.     BR, Tero Mononen   From: Tim Robertson [GBIF] [mailto:trobertson@gbif.org]  Sent: Friday, March 09, 2012 10:37 AM To: Tero Mononen Cc: ipt@lists.gbif.org Subject: Re: [IPT] All space-characters in metadata are replaced to plus-character   Dear Tero,   Could you please provide a link to an example file?  We'll investigate, but this is not a behavior I have ever seen with a Tomcat application.   Do you run any proxies or mod_jk or anything like that?   Thanks, Tim     On Mar 9, 2012, at 9:35 AM, Tero Mononen wrote:

Hi   We have used this IPT-setup without any problems till now.   System: Version 2.0.2-r3306 Apache Tomcat/6.0.24 JVM 1.6.0_29-b1   Problem: All space-characters in metadata are replaced to plus-character. For example title "foo bar car" is saved as "foo+bar+car". It affects in every text field in metadata.   If I include at least one plus-char in a text value, it prevents the space replacement. For example "foo bar car+dar" is saved as it is.   I think this is related to Java/Tomcat/urlencode. Do you have suggestions how to fix this problem?     Best Regards, Tero Mononen   -- Digitarium Digitisation Centre of the Finnish Museum of Natural History and the University of Eastern Finland Faculty of Science and Forestry, Joensuu Science Park Länsikatu 15 (P.O. Box 111) FI-80101 Joensuu http://www.digitarium.fi/   Tero Mononen, Analyst Programmer tero.mononen@uef.fi +358-50-5631788   _______________________________________________ IPT mailing list IPT@lists.gbif.org http://lists.gbif.org/mailman/listinfo/ipt

Hi,

That's true, I have to make some new installations. I'll inform you if the problem was entirely from Tomcat. Thank you a lot for helping. :)

BR, Tero

-----Original Message----- From: Tim Robertson [GBIF] [mailto:trobertson@gbif.org] Sent: Friday, March 09, 2012 2:52 PM To: Tero Mononen Cc: Kyle Braak [GBIF]; ipt@lists.gbif.org Subject: Re: [IPT] All space-characters in metadata are replaced to plus-character

Thanks for testing that Tero,

I've been hunting around on OpenSuse and Tomcat forums and can't find much on this bug. Could you run a test on Tomcat7 at all? It might be worth posting this to any OpenSuse lists you are on as it is seemingly not an IPT specific issue - we are of course keen to see this sorted and help as much as possible.

Thanks, Tim

On Mar 9, 2012, at 1:45 PM, Tero Mononen wrote:

Hi

I added the URIEncoding="UTF-8" to the connector and sadly it does not helped.

In POST-data spaces are coded using '+' in IPT. I did send POST data using %20-coding and it works. Normally I've used percent-based url-coding.

BR, Tero

From: Kyle Braak [GBIF] [mailto:kbraak@gbif.org] Sent: Friday, March 09, 2012 1:49 PM To: Tero Mononen Cc: ipt@lists.gbif.org; Tim Robertson [GBIF] Subject: Re: [IPT] All space-characters in metadata are replaced to plus-character

Hi Tero,

It's nice that we might be on the verge of solving this now..

When you have this working, I recommend an upgrade from 2.0.2 to 2.0.3. The upgrade is relatively painless.

To do so,

a) back up your current data directory to somewhere safe b) download the 2.0.3 war file, and rename it to ipt.war c) stop tomcat, remove ipt/ and ipt.war from the tomcat webapps folder, and add the new ipt.war file d) start tomcat, and open up the IPT in a browser e) presented with the 1st setup page, enter the location of your existing data directory f) press continue to proceed, then visit the Administration page and press the button "Publish all resources" (this may take a few moments as it will republish all resources.

Caution: There is a known bug with the upgrade where a taxonomic resources switches to type occurrence. I know that you have purely occurrence resources, but if this switch happens, just try the upgrade again but de-select the subtype (and be sure to save) for each resource before going through the steps.

Best of luck,

Kyle On Mar 9, 2012, at 12:43 PM, Tim Robertson [GBIF] wrote:

This looks like you are hitting this Tomcat6 / OpenSUSE bug: http://stackoverflow.com/questions/9290745/tomcat-6-doesnt-decode-to-space-i...

Can you please try the suggested fix?

Thanks, Tim

On Mar 9, 2012, at 12:25 PM, Tero Mononen wrote:

Hi

Only very minor updates from openSUSE to tomcat6 and kernel 2.6.34.

Tomcat from version 6.0.24-5.14.1 to version 6.0.24-5.16.1. Kernel 2.6.34.10-0.4.1 -> 2.6.34.10-0.6.1

BR, Tero

From: Tim Robertson [GBIF] [mailto:trobertson@gbif.org] Sent: Friday, March 09, 2012 12:48 PM To: Tero Mononen Cc: Kyle Braak [GBIF]; ipt@lists.gbif.org Subject: Re: [IPT] All space-characters in metadata are replaced to plus-character

Hi

Thank you, I made new issue (http://code.google.com/p/gbif-providertoolkit/issues/detail?id=845).

We have tested many browsers on Windows and Linux. I think the reproduction is hard because it did work on our system well and then something weird happened causing this.

Hi Tero,

What changed between it working well, and then this happening please? OS upgrades, OS changes, Tomcat upgrades, Java updates etc

The *good* news is the IPT still works - the EML is correct when accessed through Tomcat. Of course we want to fix the direct file access route, but the IPT is operating correctly through the IPT interface (e.g. http://db.digitarium.fi:8080/ipt/eml.do?r=nylander is correct )

Thanks, Tim

BR, Tero

From: Kyle Braak [GBIF] [mailto:kbraak@gbif.org] Sent: Friday, March 09, 2012 11:54 AM To: Tero Mononen Cc: ipt@lists.gbif.org; Tim Robertson [GBIF] Subject: Re: [IPT] All space-characters in metadata are replaced to plus-character

Dear Tero,

Can I kindly ask you to fill in a bug report on the IPT Project Site: http://code.google.com/p/gbif-providertoolkit/issues/entry

In addition to the IPT version, Java version, and Tomcat version, can you also specify the Operating System you are using (with version), as well as the browser (with version) that you are using to fill in the metadata? Out of curiosity, have you tried entering metadata through different browsers (Chrome, Firefox, IE)?

This will help us try and reproduce the problem ourselves so that if there is something wrong with the application we can fix it ASAP.

Thanks a lot,

Kyle On Mar 9, 2012, at 10:40 AM, Tim Robertson [GBIF] wrote:

Thanks Tero

Very curious indeed. The single input fields are represented with + but the description field which is a Textarea on the form is not.

We'll look more into it.

Cheers, Tim

On Mar 9, 2012, at 10:15 AM, Tero Mononen wrote:

Hi

There is the original eml.xml. Is that the only file you are interested in?

BR, Tero

On 03/09/2012 11:05 AM, Tim Robertson [GBIF] wrote: Hi Tero,

How very strange. Given this is public anyway (http://db.digitarium.fi:8080/ipt/eml.do?r=nylander) would you mind attaching the file from the filesystem please? I'd like to look at the file encoding etc.

Thanks, Tim

On Mar 9, 2012, at 9:44 AM, Tero Mononen wrote:

Dear Tim,

There's only public example (Title: William+Nylander+lichen+collection): http://db.digitarium.fi:8080/ipt/resource.do?r=nylander

It's interesting, eml.xml on file system contains line:

<title xml:lang="eng">William+Nylander+lichen+collection</title>

But eml via IPT/Tomcat offers line:

<title xml:lang="eng">William Nylander lichen collection</title>

It's pure Tomcat without proxies.

BR, Tero Mononen

From: Tim Robertson [GBIF] [mailto:trobertson@gbif.org] Sent: Friday, March 09, 2012 10:37 AM To: Tero Mononen Cc: ipt@lists.gbif.org Subject: Re: [IPT] All space-characters in metadata are replaced to plus-character

Dear Tero,

Could you please provide a link to an example file? We'll investigate, but this is not a behavior I have ever seen with a Tomcat application.

Do you run any proxies or mod_jk or anything like that?

Thanks, Tim

On Mar 9, 2012, at 9:35 AM, Tero Mononen wrote:

Hi

We have used this IPT-setup without any problems till now.

System: Version 2.0.2-r3306 Apache Tomcat/6.0.24 JVM 1.6.0_29-b1

Problem: All space-characters in metadata are replaced to plus-character. For example title "foo bar car" is saved as "foo+bar+car". It affects in every text field in metadata.

If I include at least one plus-char in a text value, it prevents the space replacement. For example "foo bar car+dar" is saved as it is.

I think this is related to Java/Tomcat/urlencode. Do you have suggestions how to fix this problem?

Best Regards, Tero Mononen

-- Digitarium Digitisation Centre of the Finnish Museum of Natural History and the University of Eastern Finland Faculty of Science and Forestry, Joensuu Science Park Länsikatu 15 (P.O. Box 111) FI-80101 Joensuu http://www.digitarium.fi/

Tero Mononen, Analyst Programmer tero.mononen@uef.fi +358-50-5631788

---

IPT mailing list IPT@lists.gbif.org http://lists.gbif.org/mailman/listinfo/ipt

-- Digitarium Digitisation Centre of the Finnish Museum of Natural History and the University of Eastern Finland Faculty of Science and Forestry, Joensuu Science Park Länsikatu 15 (P.O. Box 111) FI-80101 Joensuu http://www.digitarium.fi/

Tero Mononen, Analyst Programmer tero.mononen@uef.fi +358-50-5631788 <eml.xml>

---

IPT mailing list IPT@lists.gbif.org http://lists.gbif.org/mailman/listinfo/ipt

Great! Glad it was so simple in the end.

Tim

On Mar 10, 2012, at 10:58 AM, Tero Mononen wrote:

Hi

Tomcat upgrade to version 7 really fixed this issue!

Damn openSUSE or Tomcat :)

Thanks, Tero Mononen

On 03/09/2012 02:54 PM, Tero Mononen wrote:

...

Hi,

That's true, I have to make some new installations. I'll inform you if the problem was entirely from Tomcat. Thank you a lot for helping. :)

BR, Tero

-----Original Message----- From: Tim Robertson [GBIF] [mailto:trobertson@gbif.org] Sent: Friday, March 09, 2012 2:52 PM To: Tero Mononen Cc: Kyle Braak [GBIF]; ipt@lists.gbif.org Subject: Re: [IPT] All space-characters in metadata are replaced to plus-character

Thanks for testing that Tero,

I've been hunting around on OpenSuse and Tomcat forums and can't find much on this bug. Could you run a test on Tomcat7 at all? It might be worth posting this to any OpenSuse lists you are on as it is seemingly not an IPT specific issue - we are of course keen to see this sorted and help as much as possible.

Thanks, Tim

On Mar 9, 2012, at 1:45 PM, Tero Mononen wrote:

...

Hi

I added the URIEncoding="UTF-8" to the connector and sadly it does not helped.

In POST-data spaces are coded using '+' in IPT. I did send POST data using %20-coding and it works. Normally I've used percent-based url-coding.

BR, Tero

From: Kyle Braak [GBIF] [mailto:kbraak@gbif.org] Sent: Friday, March 09, 2012 1:49 PM To: Tero Mononen Cc: ipt@lists.gbif.org; Tim Robertson [GBIF] Subject: Re: [IPT] All space-characters in metadata are replaced to plus-character

Hi Tero,

It's nice that we might be on the verge of solving this now..

When you have this working, I recommend an upgrade from 2.0.2 to 2.0.3. The upgrade is relatively painless.

To do so,

a) back up your current data directory to somewhere safe b) download the 2.0.3 war file, and rename it to ipt.war c) stop tomcat, remove ipt/ and ipt.war from the tomcat webapps folder, and add the new ipt.war file d) start tomcat, and open up the IPT in a browser e) presented with the 1st setup page, enter the location of your existing data directory f) press continue to proceed, then visit the Administration page and press the button "Publish all resources" (this may take a few moments as it will republish all resources.

Caution: There is a known bug with the upgrade where a taxonomic resources switches to type occurrence. I know that you have purely occurrence resources, but if this switch happens, just try the upgrade again but de-select the subtype (and be sure to save) for each resource before going through the steps.

Best of luck,

Kyle On Mar 9, 2012, at 12:43 PM, Tim Robertson [GBIF] wrote:

This looks like you are hitting this Tomcat6 / OpenSUSE bug: http://stackoverflow.com/questions/9290745/tomcat-6-doesnt-decode-to-space-i...

Can you please try the suggested fix?

Thanks, Tim

On Mar 9, 2012, at 12:25 PM, Tero Mononen wrote:

Hi

Only very minor updates from openSUSE to tomcat6 and kernel 2.6.34.

Tomcat from version 6.0.24-5.14.1 to version 6.0.24-5.16.1. Kernel 2.6.34.10-0.4.1 -> 2.6.34.10-0.6.1

BR, Tero

From: Tim Robertson [GBIF] [mailto:trobertson@gbif.org] Sent: Friday, March 09, 2012 12:48 PM To: Tero Mononen Cc: Kyle Braak [GBIF]; ipt@lists.gbif.org Subject: Re: [IPT] All space-characters in metadata are replaced to plus-character

Hi

Thank you, I made new issue (http://code.google.com/p/gbif-providertoolkit/issues/detail?id=845).

We have tested many browsers on Windows and Linux. I think the reproduction is hard because it did work on our system well and then something weird happened causing this.

Hi Tero,

What changed between it working well, and then this happening please? OS upgrades, OS changes, Tomcat upgrades, Java updates etc

The *good* news is the IPT still works - the EML is correct when accessed through Tomcat. Of course we want to fix the direct file access route, but the IPT is operating correctly through the IPT interface (e.g. http://db.digitarium.fi:8080/ipt/eml.do?r=nylander is correct )

Thanks, Tim

BR, Tero

From: Kyle Braak [GBIF] [mailto:kbraak@gbif.org] Sent: Friday, March 09, 2012 11:54 AM To: Tero Mononen Cc: ipt@lists.gbif.org; Tim Robertson [GBIF] Subject: Re: [IPT] All space-characters in metadata are replaced to plus-character

Dear Tero,

Can I kindly ask you to fill in a bug report on the IPT Project Site: http://code.google.com/p/gbif-providertoolkit/issues/entry

In addition to the IPT version, Java version, and Tomcat version, can you also specify the Operating System you are using (with version), as well as the browser (with version) that you are using to fill in the metadata? Out of curiosity, have you tried entering metadata through different browsers (Chrome, Firefox, IE)?

This will help us try and reproduce the problem ourselves so that if there is something wrong with the application we can fix it ASAP.

Thanks a lot,

Kyle On Mar 9, 2012, at 10:40 AM, Tim Robertson [GBIF] wrote:

Thanks Tero

Very curious indeed. The single input fields are represented with + but the description field which is a Textarea on the form is not.

We'll look more into it.

Cheers, Tim

On Mar 9, 2012, at 10:15 AM, Tero Mononen wrote:

Hi

There is the original eml.xml. Is that the only file you are interested in?

BR, Tero

On 03/09/2012 11:05 AM, Tim Robertson [GBIF] wrote: Hi Tero,

How very strange. Given this is public anyway (http://db.digitarium.fi:8080/ipt/eml.do?r=nylander) would you mind attaching the file from the filesystem please? I'd like to look at the file encoding etc.

Thanks, Tim

On Mar 9, 2012, at 9:44 AM, Tero Mononen wrote:

Dear Tim,

There's only public example (Title: William+Nylander+lichen+collection): http://db.digitarium.fi:8080/ipt/resource.do?r=nylander

It's interesting, eml.xml on file system contains line:

<title xml:lang="eng">William+Nylander+lichen+collection</title>

But eml via IPT/Tomcat offers line:

<title xml:lang="eng">William Nylander lichen collection</title>

It's pure Tomcat without proxies.

BR, Tero Mononen

From: Tim Robertson [GBIF] [mailto:trobertson@gbif.org] Sent: Friday, March 09, 2012 10:37 AM To: Tero Mononen Cc: ipt@lists.gbif.org Subject: Re: [IPT] All space-characters in metadata are replaced to plus-character

Dear Tero,

Could you please provide a link to an example file? We'll investigate, but this is not a behavior I have ever seen with a Tomcat application.

Do you run any proxies or mod_jk or anything like that?

Thanks, Tim

On Mar 9, 2012, at 9:35 AM, Tero Mononen wrote:

Hi

We have used this IPT-setup without any problems till now.

System: Version 2.0.2-r3306 Apache Tomcat/6.0.24 JVM 1.6.0_29-b1

Problem: All space-characters in metadata are replaced to plus-character. For example title "foo bar car" is saved as "foo+bar+car". It affects in every text field in metadata.

If I include at least one plus-char in a text value, it prevents the space replacement. For example "foo bar car+dar" is saved as it is.

I think this is related to Java/Tomcat/urlencode. Do you have suggestions how to fix this problem?

Best Regards, Tero Mononen

-- Digitarium Digitisation Centre of the Finnish Museum of Natural History and the University of Eastern Finland Faculty of Science and Forestry, Joensuu Science Park Länsikatu 15 (P.O. Box 111) FI-80101 Joensuu http://www.digitarium.fi/

Tero Mononen, Analyst Programmer tero.mononen@uef.fi +358-50-5631788

---

IPT mailing list IPT@lists.gbif.org http://lists.gbif.org/mailman/listinfo/ipt

-- Digitarium Digitisation Centre of the Finnish Museum of Natural History and the University of Eastern Finland Faculty of Science and Forestry, Joensuu Science Park Länsikatu 15 (P.O. Box 111) FI-80101 Joensuu http://www.digitarium.fi/

Tero Mononen, Analyst Programmer tero.mononen@uef.fi +358-50-5631788 <eml.xml>

---

IPT mailing list IPT@lists.gbif.org http://lists.gbif.org/mailman/listinfo/ipt

---

IPT mailing list IPT@lists.gbif.org http://lists.gbif.org/mailman/listinfo/ipt

-- Digitarium Digitisation Centre of the Finnish Museum of Natural History and the University of Eastern Finland Faculty of Science and Forestry, Joensuu Science Park Länsikatu 15 (P.O. Box 111) FI-80101 Joensuu http://www.digitarium.fi/

Tero Mononen, Analyst Programmer tero.mononen@uef.fi +358-50-5631788

This EML is older than the edited metadata. It was made before the problem appeared.

I just made a new EML in http://db.digitarium.fi:8080/ipt/resource.do?r=test The pluses do go in this.

Hannu

On 2012-03-09 10:44, Tero Mononen wrote:

Dear Tim,

There's only public example (Title: William+Nylander+lichen+collection):

http://db.digitarium.fi:8080/ipt/resource.do?r=nylander

It's interesting, eml.xml on file system contains line:

<title xml:lang="eng">William+Nylander+lichen+collection</title>

But eml via IPT/Tomcat offers line:

<title xml:lang="eng">William Nylander lichen collection</title>

It's pure Tomcat without proxies.

BR,

Tero Mononen

*From:*Tim Robertson [GBIF] [mailto:trobertson@gbif.org] *Sent:* Friday, March 09, 2012 10:37 AM *To:* Tero Mononen *Cc:* ipt@lists.gbif.org *Subject:* Re: [IPT] All space-characters in metadata are replaced to plus-character

Dear Tero,

Could you please provide a link to an example file? We'll investigate, but this is not a behavior I have ever seen with a Tomcat application.

Do you run any proxies or mod_jk or anything like that?

Thanks,

Tim

On Mar 9, 2012, at 9:35 AM, Tero Mononen wrote:

Hi

We have used this IPT-setup without any problems till now.

System:

Version 2.0.2-r3306

Apache Tomcat/6.0.24

JVM 1.6.0_29-b1

Problem:

All space-characters in metadata are replaced to plus-character. For example title "foo bar car" is saved as "foo+bar+car". It affects in every text field in metadata.

If I include at least one plus-char in a text value, it prevents the space replacement. For example "foo bar car+dar" is saved as it is.

I think this is related to Java/Tomcat/urlencode. Do you have suggestions how to fix this problem?

Best Regards,

Tero Mononen

--

Digitarium

Digitisation Centre of the Finnish Museum of Natural History and the University of Eastern Finland Faculty of Science and Forestry, Joensuu Science Park Länsikatu 15 (P.O. Box 111)

FI-80101 Joensuu

http://www.digitarium.fi/

Tero Mononen, Analyst Programmer

tero.mononen@uef.fi mailto:juha.lehtonen@uef.fi

+358-50-5631788

---

IPT mailing list IPT@lists.gbif.org mailto:IPT@lists.gbif.org http://lists.gbif.org/mailman/listinfo/ipt

---

IPT mailing list IPT@lists.gbif.org http://lists.gbif.org/mailman/listinfo/ipt