GFBio : DataCatalog

Changelog

What's changed

Notable changes to the catalog

v1.0.0-rc.3

Added
  • Safer editing when several people work on one record. If someone else saved it after you opened it, your save or delete stops and asks you to reload, instead of silently overwriting their changes.
  • Qualified relations can be entered inline on datasets, data services, catalogs and dataset series. Catalogs and dataset series also gain the identifiers, distributions, temporal coverage, provenance and attributions.
  • Catalog records can name the agent responsible for them, such as the curator or harvesting service that maintains the record (`wasAttributedTo`).
  • Every record has a stable identifier, `id`: its own URI where it has one, otherwise its address in this catalog followed by `#it`. RDF output uses the same identifier.
  • Pick a format in the URL with `?f=json`, `jsonld`, `ttl`, `rdf` or `nt` instead of the `Accept` header. Lists can now be fetched as RDF as well, with Hydra paging links that DCAT harvesters understand.
  • Conditional requests. Each record carries an `ETag` and a `Last-Modified` date, so a script can skip unchanged downloads (304) and send `If-Match` to avoid overwriting someone else's change (412).
  • Every list can be read to its last record: pass a page's `next_cursor` back as `after`. `nxs <record type> list --all` does this for you. Add `include_deleted=true` to see withdrawn records as well.
  • The catalog publishes its JSON-LD context and the SHACL shapes it validates against, under `/api/v1/context/…` and `/api/v1/shapes/…`. Each response links to the one that describes it.
  • A public log of harvest runs at `/api/v1/ingest`: which runs touched the catalog and what became of each dataset they delivered, for example created, unchanged, or refused with the reason.
  • Harvest mappings for ABCD 2.06 and BioCASe archives, EML 2.1.1 and 2.2, Darwin Core Archives and Darwin Core Data Packages. A dataset described in one of these formats can be harvested into the catalog as a complete record; for an archive, the catalog reads the dataset's metadata, not its records.
  • New helpers in the Python client for scripts: read a whole collection, find a record by its URI, look up a vocabulary value by its code or label, and `replace()`, `get_tagged()` and `if_match=` for safe updates.
Changed
  • Breaking for API users: JSON records now include `id` and `type`. A link to another record is read and written as an object under the field's own name, for example `"dcterms_publisher": {"ext_id": 12}`, with its names in `labels`.
  • Breaking for API users: values keep exactly the text they were sent in. Numbers come back as strings (`"1.10"` stays `"1.10"`), and dates as `{value, type}`, typed as a date or a date and time by what was entered.
  • Breaking for API users: `PATCH` takes a JSON merge patch (`application/merge-patch+json`); other content types are refused with 415. Use the new `PUT` to replace a record. What a `GET` returns can be sent back.
  • Breaking for API users: some error codes changed. Malformed JSON answers 400 (was 500), a link to a record that does not exist 422 (was 409), and an invalid query parameter 400 (was 422).
  • Breaking for API users: a request body over 10 MiB is refused with 413, an unsupported `Content-Type` with 415, and a format the resource cannot provide with 406. A new link to a withdrawn record answers 422.
  • Breaking for API users: the `ext_uri` filter matches the exact URI, no longer every URI containing the text. `skip` stops at 10,000; page further with `after`.
  • Breaking for API users: a catalog, dataset, dataset series or data service created through the API needs a title, and a catalog, dataset or dataset series a description. The forms already asked for both.
  • Writes are checked more strictly. Length limits on titles, descriptions, keywords and names are enforced for each translation, and RDF uploads meet the same rules as JSON, with a 422 that names the triple at fault.
  • Regional language tags such as `de-DE`, `en-GB` or `pt-BR` are accepted for the 24 EU languages the catalog supports. Forms let you type the tag and suggest the allowed ones; a malformed tag is refused.
  • Contact email addresses and phone numbers are stored without `mailto:` or `tel:` and shown as clickable links. A prefix sent with the value is removed, and RDF export still writes `mailto:` and `tel:` addresses.
  • Linked records are shown by their name in your chosen language, falling back to their URI. Repeated values, such as two identical keywords, are merged, and multiple values always appear in the same order.
  • RDF output writes values exactly as they were entered (`"30"^^xsd:decimal`, not `30.0`), and JSON-LD output is the JSON record with its context.
  • A nested entry sent with its URI, such as a distribution, is updated in place and keeps its id instead of being replaced by a copy, so repeating an unchanged harvest changes nothing.
  • Download the Python client and `nxs` again from `/api/v1/client/python`: the copies made for 1.0.0-rc.2 do not work with this release. `nxs edit` now saves only if nobody changed the record in the meantime.
Removed
  • The numeric reference fields in write requests, such as `dcterms_publisher_id`. Send the link under the field's own name instead, as `{"ext_id": 12}` or `{"id": "<URI>"}`.
  • `blueprint_hash`, `decision_profile` and `inheritance_strategy` are no longer part of the API contract at `/api/v1/contract`. They are listed under `lineage` at `/api/v1/identity`.
Fixed
  • Every value of a large vocabulary can be chosen. Common media types such as `text/csv` or `image/png`, and some languages, showed "No options available"; the selector now searches the whole vocabulary as you type.
  • Paging through a list no longer repeats or skips records: every list now has a fixed order.
  • Restoring a deleted record brings back exactly the nested entries deleted with it, not ones that were removed earlier.
  • Searches and writes containing characters XML does not allow, or numbers too large to store, now get a clear error instead of a server error.
  • Some properties whose value is a web address were exported in RDF as plain text; they are now exported as proper links (IRIs).
  • `/api/v1/policy` now states correctly that vocabulary concepts and concept schemes can be read without signing in.
  • The user administration page keeps the current page on screen while the next one loads, instead of going blank.
Security
  • Uploaded RDF is read without fetching anything from elsewhere. Remote JSON-LD contexts are not loaded, and RDF/XML that refers to an external DTD or entity is refused.
  • Responses about your own account and access tokens are marked private, so shared caches never store them.

v1.0.0-rc.2

Added
  • Interactive API documentation. Browse and try every catalog endpoint from your browser at `/api/docs`, or read it as a reference at `/api/redoc`. The machine-readable description lives at `/api/openapi.json`.
  • Every field and endpoint in that documentation now carries a description drawn from the catalog's own DCAT-AP vocabulary, so each one states what it means, what values it accepts and whether you may write to it.
  • A ready-made Python client, downloadable from `/api/v1/client/python`. Work with the catalog from scripts and notebooks without hand-writing HTTP requests.
  • `nxs`, a command-line client for the catalog. It is built from the catalog's own published API contract, so it always matches the server it talks to.
  • `nxs policy`, which shows which role is required for each operation.
Changed
  • Checksums are recorded directly on a distribution. Previously a checksum was a separate record you created and linked; it is now a pair of fields on the distribution itself.
  • The start and end of a time period are fields on the period itself. Previously each was a separate "time instant" record.
  • Temporal coverage and provenance statements now belong directly to the dataset they describe and are edited inline on the dataset, instead of being created separately and then linked.
  • Nested records can be edited several levels deep from within the parent form, rather than one level at a time.
  • Entering a date is simpler: the fine-grained OWL-Time precision fields are hidden, leaving the plain date and time.
Removed
  • "Checksum" and "Time instant" no longer exist as standalone record types. Their content now lives on distributions and time periods respectively. Existing entries are moved across automatically when this release is deployed; nothing needs to be re-entered.
Fixed
  • A conflict caused by a missing required value or a duplicate now names the field responsible, instead of reporting a generic error.