[Ala-portal] DwC-A loading problems

[Natasha.Quimby at csiro.au]

Hi Daniel,

When you specify a local DwcA Load the archive needs to be unzipped. Try unzipping 2f676abc-4503-489e-8f0c-fcb6e1bc554b.zip and then running the following:
sudo java -cp .:biocache.jar au.org.ala.util.DwCALoader dr7 -l /data/collectory/upload/1398658607824/2f676abc-4503-489e-8f0c-fcb6e1bc554b

If you configure the collectory to provide the dwca the biocache automatically unzips the archive for you.  You would need to configure dr7 with the following connection parameters:

"protocol":"DwCA"
"termsForUniqueKey":["occurrenceID"],
"url":"file:////data/collectory/upload/1398658607824/2f676abc-4503-489e-8f0c-fcb6e1bc554b.zip"

You could then load the resource by:
sudo java -cp .:biocache.jar au.org.ala.util.DwCALoader dr7

If you continue to have issues please let us know.

Hope that this helps.

Regards
Natasha

From: Daniel Lins <daniel.lins at gmail.com<mailto:daniel.lins at gmail.com>>
Date: Monday, 28 April 2014 3:54 PM
To: "ala-portal at lists.gbif.org<mailto:ala-portal at lists.gbif.org>" <ala-portal at lists.gbif.org<mailto:ala-portal at lists.gbif.org>>, "dos Remedios, Nick (CES, Black Mountain)" <Nick.Dosremedios at csiro.au<mailto:Nick.Dosremedios at csiro.au>>, "Martin, Dave (CES, Black Mountain)" <David.Martin at csiro.au<mailto:David.Martin at csiro.au>>
Subject: [Ala-portal] DwC-A loading problems

Hi Nick and Dave,

We are having some problems in Biocache during the upload of DwC-A files.

As shown below, after run the method "au.org.ala.util.DwCALoader", our system returns the error message "Exception in thread "main" org.gbif.dwc.text.UnkownDelimitersException: Unable to detect field delimiter"

I accomplished tests using DwC-A files with tab-delimited text files and comma-delimited text files. In both cases the error generated was the same.

What causes these problems? (** CSV Loader works great)

tab-delimited file test

poliusp at poliusp-VirtualBox:~/dev/biocache$ sudo java -cp .:biocache.jar au.org.ala.util.DwCALoader dr7 -l /data/collectory/upload/1398658607824/2f676abc-4503-489e-8f0c-fcb6e1bc554b.zip
2014-04-28 01:44:02,837 INFO : [ConfigModule] - Loading configuration from /data/biocache/config/biocache-config.properties
2014-04-28 01:44:03,090 INFO : [ConfigModule] - Initialise SOLR
2014-04-28 01:44:03,103 INFO : [ConfigModule] - Initialise name matching indexes
2014-04-28 01:44:03,605 INFO : [ConfigModule] - Initialise persistence manager
2014-04-28 01:44:03,606 INFO : [ConfigModule] - Configure complete
Loading archive /data/collectory/upload/1398658607824/2f676abc-4503-489e-8f0c-fcb6e1bc554b.zip for resource dr7 with unique terms List(dwc:occurrenceID) stripping spaces false incremental false testing false
Exception in thread "main" org.gbif.dwc.text.UnkownDelimitersException: Unable to detect field delimiter
        at org.gbif.file.CSVReaderFactory.buildArchiveFile(CSVReaderFactory.java:129)
        at org.gbif.file.CSVReaderFactory.build(CSVReaderFactory.java:46)
        at org.gbif.dwc.text.ArchiveFactory.readFileHeaders(ArchiveFactory.java:344)
        at org.gbif.dwc.text.ArchiveFactory.openArchive(ArchiveFactory.java:289)
        at au.org.ala.util.DwCALoader.loadArchive(DwCALoader.scala:129)
        at au.org.ala.util.DwCALoader.loadLocal(DwCALoader.scala:106)
        at au.org.ala.util.DwCALoader$.main(DwCALoader.scala:52)
        at au.org.ala.util.DwCALoader.main(DwCALoader.scala)


comma-delimited file test

poliusp at poliusp-VirtualBox:~/dev/biocache$ sudo java -cp .:biocache.jar au.org.ala.util.DwCALoader dr7 -l ./dwca-teste3.zip
2014-04-28 01:56:04,683 INFO : [ConfigModule] - Loading configuration from /data/biocache/config/biocache-config.properties
2014-04-28 01:56:04,940 INFO : [ConfigModule] - Initialise SOLR
2014-04-28 01:56:04,951 INFO : [ConfigModule] - Initialise name matching indexes
2014-04-28 01:56:05,437 INFO : [ConfigModule] - Initialise persistence manager
2014-04-28 01:56:05,438 INFO : [ConfigModule] - Configure complete
Loading archive ./dwca-teste3.zip for resource dr7 with unique terms List(dwc:occurrenceID) stripping spaces false incremental false testing false
Exception in thread "main" org.gbif.dwc.text.UnkownDelimitersException: Unable to detect field delimiter
        at org.gbif.file.CSVReaderFactory.buildArchiveFile(CSVReaderFactory.java:129)
        at org.gbif.file.CSVReaderFactory.build(CSVReaderFactory.java:46)
        at org.gbif.dwc.text.ArchiveFactory.readFileHeaders(ArchiveFactory.java:344)
        at org.gbif.dwc.text.ArchiveFactory.openArchive(ArchiveFactory.java:289)
        at au.org.ala.util.DwCALoader.loadArchive(DwCALoader.scala:129)
        at au.org.ala.util.DwCALoader.loadLocal(DwCALoader.scala:106)
        at au.org.ala.util.DwCALoader$.main(DwCALoader.scala:52)
        at au.org.ala.util.DwCALoader.main(DwCALoader.scala)


Thanks!

Regards.
--
Daniel Lins da Silva
(Mobile) 55 11 96144-4050<tel:55%2011%2096144-4050>
Research Center on Biodiversity and Computing (Biocomp)
University of Sao Paulo, Brazil
daniellins at usp.br<mailto:daniellins at usp.br>
daniel.lins at gmail.com<mailto:daniel.lins at gmail.com>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.gbif.org/pipermail/ala-portal/attachments/20140428/b42c345e/attachment.html