Skip to content
arocapi
These docs describe arocapi v5, which is in beta. Install it with pnpm add arocapi@beta.

Storage

GET and HEAD /entity/:id/metadata and /file/:id go through a storage backend. arocapi calls it only after the entity is Visible and the request has access, so it doesn’t need its own checks (see the request lifecycle). arocapi ships DirectoryStorage, the Metadata half EntityMetaMetadata, S3Storage and OcflStorage, or you can write your own. Storage compares ways to serve bytes.

Option Required Type Default Purpose
storage yes StorageBackend none Delivers an entity’s Metadata and a file’s Content.

Registration throws storage is required without it, and storage.metadata is required or storage.content is required without either half.

The halves

Section titled “The halves”

A storage backend is a plain object of two halves, { metadata, content }:

Half Serves Receives
metadata GET, HEAD /entity/:id/metadata the Entity row, including entity.meta, and the request context
content GET, HEAD /file/:id the File row and its file entity, both with meta, and the request context with contentDisposition

Each returns a delivery, or null for a 404, and may be async. HEAD and GET make the same call, so their headers always agree. The meta columns hold what a backend needs to find the bytes; see Entity meta.

Mixing sources

Section titled “Mixing sources”

The reference backends return both halves as a plain object, so either half can be replaced by another source: a reference backend’s half, a half-only source such as EntityMetaMetadata, or your own. Here Metadata comes from the database and Content from disk:

import type { StorageBackend } from 'arocapi';
import { DirectoryStorage } from 'arocapi/storage/directory';
import { EntityMetaMetadata } from 'arocapi/storage/entity-meta';


// Metadata kept in the database at Entity.meta.rocrate, Content on disk
export const storage: StorageBackend = {
  metadata: EntityMetaMetadata(),
  content: DirectoryStorage({ root: '/srv/archive' }).content,
};

Each half reads its own key from meta, so one row can carry both, for example rocrate for Metadata beside storagePath for Content.

DirectoryStorage

Section titled “DirectoryStorage”

Serves both halves from files below a directory. Import it from arocapi/storage/directory:

import { DirectoryStorage } from 'arocapi/storage/directory';


// Metadata from Entity.meta.storagePath and Content from File.meta.storagePath, both below root
export const storage = DirectoryStorage({ root: '/srv/archive' });


// Or derive each location yourself, ignoring meta
export const located = DirectoryStorage({
  root: '/srv/archive',
  locate: {
    metadata: (entity) => `${new URL(entity.id).pathname.slice(1)}/ro-crate-metadata.json`,
    content: (file, entity) =>
      entity.memberOf ? `${new URL(entity.memberOf).pathname.slice(1)}/${file.filename}` : null,
  },
});
Option Required Type Default Purpose
root yes string none The directory every Storage key is relative to.
locate no { metadata?, content? } read meta.storagePath Where each item sits below root: metadata(entity) and content(file, entity) return, or resolve to, a relative path or null.

Without locate, Metadata is read from Entity.meta.storagePath and Content from File.meta.storagePath. A Metadata key names the RO-Crate metadata file itself, so several entities can share one.

  • A missing key, a locate returning null, or no file at the path is a 404.
  • A storagePath that isn’t a string, null included, is a 500, logged with the entity id and key.
  • A key that is absolute or has a .. segment is a 500, before anything is read. An absolute storagePath written for arocapi 4 fails this way rather than being read as relative.
  • Symlinks below root are followed, even to files outside it. The key check guards what meta and locate can name; anyone who can write to root is trusted.

Content-Length and Last-Modified come from the file, with a weak ETag from its size and modification time. Content-Type comes from File.mediaType; Metadata is always application/ld+json.

EntityMetaMetadata

Section titled “EntityMetaMetadata”

A Metadata half only, for RO-Crate metadata kept in the database: it serves the JSON object at a key in Entity.meta. Pair it with another source’s content, as in Mixing sources. Import it from arocapi/storage/entity-meta:

import { EntityMetaMetadata } from 'arocapi/storage/entity-meta';


// Metadata from the JSON object at Entity.meta.rocrate
export const metadata = EntityMetaMetadata();


// Or from another key, such as Entity.meta.crate
export const fromCrate = EntityMetaMetadata({ key: 'crate' });
Option Required Type Default Purpose
key no string rocrate The key in Entity.meta holding the RO-Crate metadata.
  • A missing key is a 404.
  • A value that isn’t a JSON object (a string, array, number, boolean or null) is a 500, logged with the entity id and key, so an indexing bug isn’t served as Metadata.

arocapi serialises the object once, with its Content-Length and a strong ETag from those bytes, and no Last-Modified. PostgreSQL stores meta as jsonb, so key order and whitespace aren’t kept from what you wrote.

S3Storage

Section titled “S3Storage”

Serves both halves from a bucket on any S3-compatible store, such as AWS S3, Cloudflare R2, Garage or VersityGW. Import it from arocapi/storage/s3, and install its optional peer dependencies @aws-sdk/client-s3 and @aws-sdk/s3-request-presigner:

import { S3Client } from '@aws-sdk/client-s3';
import { S3Storage } from 'arocapi/storage/s3';


const client = new S3Client({ region: 'ap-southeast-2' });


// Metadata from Entity.meta.storageKey and Content from File.meta.storageKey; Content redirects to a presigned URL
export const storage = S3Storage({ client, bucket: 'archive' });


// Or proxy Content through arocapi, keeping the bucket private
export const proxied = S3Storage({ client, bucket: 'archive', delivery: 'stream' });
Option Required Type Default Purpose
client yes S3Client none Your client. You configure its credentials and endpoint, and destroy it at shutdown.
bucket yes string none The bucket every Storage key is relative to.
locate no { metadata?, content? } read meta.storageKey Where each item sits in the bucket: metadata(entity) and content(file, entity) return, or resolve to, a key or null.
delivery no 'redirect' or 'stream' 'redirect' How Content reaches the client. Metadata always streams.
expiresIn no number 300 Seconds a presigned Content URL stays valid.

Without locate, Metadata is read from Entity.meta.storageKey and Content from File.meta.storageKey.

  • redirect answers /file/:id with a 302 to a presigned GetObject URL, for HEAD as well as GET. The URL makes the bucket answer with Content-Type from File.mediaType and the request’s Content-Disposition, so ?disposition and ?filename still apply. Presigning makes no request, so a missing object is only found when the client follows the URL. The URL bypasses access until it expires, so keep expiresIn short; a download only has to start within it. A browser on another origin needs CORS on the bucket, or ?noRedirect=true. arocapi can’t add its protective headers to the bucket’s response; see protecting the target.
  • stream proxies the bytes through arocapi, so clients never reach the bucket. Metadata is always served this way. arocapi reads Content-Length, ETag and Last-Modified with HeadObject, then fetches that same version with GetObject and If-Match; an object replaced in between is a 500 rather than a mix of two versions.

A missing key, a locate returning null, or no object at the key (on stream, and for Metadata) is a 404. A storageKey that isn’t a string (null included), or a key that is absolute or has a .. segment, is a 500 before any request or presign. Any other error from the store is a logged 500.

S3 answers a missing key with 403, not 404, unless the credentials allow s3:ListBucket on the bucket, so grant it for missing objects to be 404s.

OcflStorage

Section titled “OcflStorage”

Serves both halves from the head version of objects in an OCFL storage root on local disk, through @ocfl/ocfl-fs. Install it alongside arocapi (pnpm add @ocfl/ocfl-fs); it is an optional peer dependency, loaded only by arocapi/storage/ocfl. @ocfl/ocfl-fs is licensed GPL-3.0-or-later: the licence applies to your app if you use this backend and distribute it.

OcflStorage is async, because it reads the storage root at startup:

import { OcflStorage } from 'arocapi/storage/ocfl';


// Reads ocfl_layout.json at startup; Metadata from Entity.meta.ocflObject (+ ocflPath, default
// ro-crate-metadata.json) and Content from File.meta.ocflObject + File.meta.ocflPath
export const storage = await OcflStorage({ root: '/srv/ocfl' });


// A storage root with a layout OcflStorage doesn't implement, and items found without meta
export const custom = await OcflStorage({
  root: '/srv/ocfl',
  objectRoot: (id) => `objects/${encodeURIComponent(id)}`,
  locate: {
    metadata: (entity) => ({ object: entity.id, path: 'ro-crate-metadata.json' }),
    content: (file, entity) => (entity.memberOf ? { object: entity.memberOf, path: file.filename } : null),
  },
});
Option Required Type Default Purpose
root yes string none The OCFL storage root.
objectRoot no (id) => string the layout in ocfl_layout.json Where an object sits, relative to root; may be async.
locate no { metadata?, content? } read meta.ocflObject and meta.ocflPath Where each item is: metadata(entity) and content(file, entity) return, or resolve to, an OcflLocation { object, path } or null.

At startup OcflStorage reads ocfl_layout.json and its extension config (extensions/<layout>/config.json, such as the digest algorithm and tuple size and number), and implements these storage layouts:

For any other layout, or a root without ocfl_layout.json, pass objectRoot, or it throws. It also throws if root isn’t an OCFL storage root.

Without locate, Metadata is read from Entity.meta.ocflObject and Entity.meta.ocflPath, which defaults to ro-crate-metadata.json, and Content from File.meta.ocflObject and File.meta.ocflPath. object is the OCFL object id and path a logical path in the object’s head version.

  • Each request loads the object’s inventory, with no cache, and serves the head version. Older versions and a mutable HEAD (extension 0005) are never served.
  • A missing key, a locate returning null, an object that doesn’t exist or a logical path not in head is a 404.
  • An ocflObject or ocflPath that isn’t a string, null included, is a 500, logged with the entity id and key, as is a logical path that is absolute or has a .. segment. An object id is any string, mapped by the storage layout.
  • An inventory that fails its sidecar digest check or won’t parse is a 500, as is an object root outside the storage root, or a manifest path outside its object or not naming a file. A request made while a writer swaps an inventory may also get a 500.
  • These checks compare paths, not symlink targets. OCFL forbids symlinks in a storage root; any present are followed, even outside the root, so keep the root writable only by trusted tools.

Content-Length and Last-Modified come from the file on disk, with a strong ETag from its inventory digest. Content-Type comes from File.mediaType; Metadata is always application/ld+json.

Protective headers

Section titled “Protective headers”

Content and Metadata come from depositors, and arocapi serves them on its own origin. So that a deposited HTML or SVG file can’t run scripts there, every stream, path and json delivery is sent, on GET and HEAD, with:

Header Value Sent on
X-Content-Type-Options nosniff every delivery
Content-Security-Policy default-src 'none'; style-src 'unsafe-inline'; sandbox all but audio/* and video/*

sandbox gives the page an opaque origin and blocks its scripts, and default-src 'none' stops it loading outside images, fonts or stylesheets, and data: images and fonts embedded in it too; inline styles still apply. PDFs, images and text still display inline, though Safari before 18.4 won’t show a PDF under sandbox. nosniff stops a browser treating a file as a more dangerous type than its Content-Type.

Audio and video get no CSP, because Chrome won’t play media opened directly under it. A media document can’t run script, and nosniff stops a file labelled as media being sniffed into HTML. The exemption needs a single audio/* or video/* type, in upper or lower case and with any parameters such as ; codecs=opus, including quoted values with commas such as ; codecs="avc1.42E01E, mp4a.40.2". A list such as audio/mpeg, text/html keeps the CSP, as a browser reads the last type in it. <audio>, <video> and <img> elements on other pages ignore the CSP and keep working.

Redirect deliveries, and the noRedirect answer, carry neither header, since they can’t protect the target.

The headers are fixed. A deployment that needs deposited HTML to run can replace or remove them in its own onSend hook. Add the hook and register arocapi together in one plugin, so the hook sees only arocapi’s routes and leaves your own pages’ headers, such as a CSP from @fastify/helmet, alone:

import arocapi, { type Options } from 'arocapi';
import Fastify from 'fastify';


// Your arocapi options
declare const options: Options;


const fastify = Fastify();


// In its own scope with arocapi, so the hook sees only arocapi's routes
await fastify.register(async (scope) => {
  // Let deposited HTML run scripts, still in an opaque origin
  scope.addHook('onSend', async (_request, reply, payload) => {
    if (reply.hasHeader('content-security-policy')) {
      reply.header('Content-Security-Policy', 'sandbox allow-scripts');
    }


    return payload;
  });


  await scope.register(arocapi, options);
});

ADR 0005 records why.

Query parameters

Section titled “Query parameters”

/file/:id accepts, on GET and HEAD:

Parameter Values Default Effect
disposition inline or attachment inline Content-Disposition type.
filename string file.filename Filename in Content-Disposition.
noRedirect true or false false Answer a redirect delivery with 200 { "location": url }.

arocapi sends Content-Disposition on stream and path deliveries, and passes the same value to content as contentDisposition, so a redirecting backend can bake it into its URL. Any other disposition or noRedirect value is a 400. /entity/:id/metadata takes no query parameters.

Range requests

Section titled “Range requests”

arocapi doesn’t handle Range headers yet: a stream or path delivery is always sent whole, with the full Content-Length. A redirect to storage that seeks, such as a presigned object storage URL, supports ranges today, and path deliveries will once Range support lands.