[API-users] understanding download status (specially FILE_ERASED)

[Daniel Noesgaard]

Hi David,

First of all, the status of a download request doesn't really say anything about actual usage. PREPARING and RUNNING  are initial statuses that are followed by SUCCEEDED, unless something goes wrong—in which case you may see FAILED or KILLED, if cancelled.

Due to storage restrictions we have had to delete some older, very large downloads, which now have status FILE_ERASED.

If a download is used and the usage properly cited, say in a journal publication, we will flag this download by linking the citing article to the download. This information, however, is not available in the regular API. You may access this information through

https://www.gbif.org/api/resource/search?contentType=literature&gbifDatasetKey={datasetKey}

or for a specific download

https://www.gbif.org/api/resource/search?contentType=literature&gbifDownloadKey={downloadKey}

If you'd like to discuss this further, feel free to reach out to me. I would be interested in learning more about what you're planning to do with this data?

Thanks,
Daniel
GBIF

________________________________
From: API-users <api-users-bounces at lists.gbif.org> on behalf of Herbario SANT <sant.herbarium at gmail.com>
Sent: Monday, January 20, 2020 21:25
To: api-users at lists.gbif.org <api-users at lists.gbif.org>
Subject: [API-users] understanding download status (specially FILE_ERASED)

I have a question about the results returned by the api when requesting the download activity of a dataset.
/occurrence/download/dataset/{datasetKey}<https://www.gbif.org/developer/occurrence#download>

Where can I find a description of the possible "status" of a given download? (I found at least 6 different values in our datasets activity).

I understand that KILLED, FAILED, RUNNING, PREPARING ... can all be interpreted as "nobody has (still) used that information".
Whereas a "SUCCEEDED" download means the opposite.

But what about "FILE_ERASED<http://api.gbif.org/v1/occurrence/download/0000316-140429114108248>"?
Does it mean the download was "SUCCEEDED" in the past, and some time later the info was deleted? (when looking at old downloads, the percentage of FILE_ERASED is much higher).

If I want to evaluate real usage of a dataset along years, should I better use the sum of "SUCCEEDED + FILE_ERASED". Right?

Also, I understand that the sum of SUCCEEDED + FILE_ERASED downloads for a given dataset and year, should be the same if somebody repeats the query in the future (whereas the sum of SUCCEEDED downloads will decrease).

Thanks a lot for your help
David

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.gbif.org/pipermail/api-users/attachments/20200121/81a202b0/attachment.html>