Storage

arocapi doesn’t know where your files live. A storage backend of two halves fetches them for it: metadata serves /entity/:id/metadata and content serves /file/:id. Both run only after the entity is visible and the request may read it (see Visibility vs Access), so a storage backend never checks access itself.

This page helps you choose: which backend holds your bytes, where Metadata comes from, and how bytes reach the client. Each backend’s options are under Configuration → Storage, and the meta keys they read under Entity meta.

Choosing a backend

Backend	Content delivery	Ranges today	Peer dependencies	Pick it when
DirectoryStorage	path	No	None	Files sit on a disk the API server can read.
S3Storage	presigned redirect (default), or stream	Yes, on redirect	@aws-sdk/client-s3, @aws-sdk/s3-request-presigner	Files are in a bucket on AWS S3 or another S3-compatible store.
OcflStorage	path	No	@ocfl/ocfl-fs (GPL-3.0-or-later)	Files are preserved in an OCFL storage root on local disk, which stays the source of truth.

The peer dependencies are optional: install only the ones for the backend you import. @ocfl/ocfl-fs is licensed GPL-3.0-or-later, which matters only if you use OcflStorage and distribute your app.

If none of these fits, write your own.

Where Metadata comes from

An entity’s RO-Crate metadata can come from one of three places:

• Stored beside the Content. Each backend’s metadata half reads ro-crate-metadata.json from the same disk, bucket or OCFL object as the files. This keeps one source of truth.
• Kept in the database. EntityMetaMetadata serves the object your indexer wrote to Entity.meta.rocrate. Nothing is read from storage, but the copy is only as fresh as your last index run.
• Generated per request. A metadata half of your own builds the document, for example to redact it, and returns it as a json delivery.

Mix sources by taking one half from each. Here Metadata comes from the database and Content from OCFL:

import type { StorageBackend } from 'arocapi';
import { EntityMetaMetadata } from 'arocapi/storage/entity-meta';
import { OcflStorage } from 'arocapi/storage/ocfl';

const ocfl = await OcflStorage({ root: '/srv/ocfl' });

// Metadata from Entity.meta.rocrate, Content from the head version of each OCFL object
export const storage: StorageBackend = {
  metadata: EntityMetaMetadata(),
  content: ocfl.content,
};

A generated half that redacts contact details from the copy in the database:

import type { MetadataSource } from 'arocapi';

type Crate = { '@graph': Array<Record<string, unknown>> };

type EntityMeta = { rocrate: Crate };

// Serve Entity.meta.rocrate without contact details; arocapi serialises the value and sets its length and ETag
export const metadata: MetadataSource = (entity) => {
  const crate = (entity.meta as EntityMeta | null)?.rocrate;
  if (!crate) {
    return null;
  }

  const graph = crate['@graph'].map(({ email: _email, ...node }) => node);

  return { type: 'json', value: { ...crate, '@graph': graph } };
};

arocapi serialises the value once and sets its Content-Length and a strong ETag from those bytes.

Stream, path or redirect

A content half answers with one of three deliveries, which decide who moves the bytes:

Delivery	Who moves the bytes	Trade-off
path	arocapi, from a file on local disk	Simple and private, but sent whole for now.
stream	arocapi, from a stream the backend opens	Every byte stays behind your API, and your Node process carries the whole transfer.
redirect	Whatever serves the URL, such as object storage or a CDN	Offloads the transfer and supports ranges, at the cost of a second request, a second origin and a URL that works for anyone who holds it.

sequenceDiagram
  participant Client
  participant arocapi
  participant Bucket as Object storage
  Client->>arocapi: GET /file/{id}
  arocapi->>arocapi: visibility and access
  alt redirect
    arocapi-->>Client: 302 Location presigned URL
    Client->>Bucket: GET presigned URL, with Range
    Bucket-->>Client: 206 Partial Content
  else stream
    arocapi->>Bucket: GetObject
    Bucket-->>arocapi: bytes
    arocapi-->>Client: 200 with the whole file
  end

Ranges: an S3 redirect seeks today, and path deliveries will once Support Range requests on GET /file lands. Until then, a media player seeking in a path or stream file downloads it from the start. For seekable Content from disk or OCFL today, redirect to a content route of your own, as in the signed redirect example.

S3Storage redirects by default. Shorten the expiry, or stream instead:

import { S3Client } from '@aws-sdk/client-s3';
import { S3Storage } from 'arocapi/storage/s3';

const client = new S3Client({ region: 'ap-southeast-2' });

// Redirect to presigned URLs that only need to outlive the start of a download
export const storage = S3Storage({ client, bucket: 'archive', expiresIn: 60 });

// A bucket clients can't reach: arocapi proxies every byte
export const proxied = S3Storage({ client, bucket: 'archive', delivery: 'stream' });

When you redirect, plan for:

• A short presign expiry. arocapi checks access before it presigns, so the URL is the permission: anyone holding it can download the file until it expires. The default is 300 seconds, and a download only has to start within it.
• Bucket CORS. A browser client on another origin can only read a redirected response if the bucket’s CORS rules allow that origin.
• ?disposition and ?filename baked into the URL. arocapi passes the Content-Disposition it computed to the content half as contentDisposition, and S3Storage signs it into the URL so the bucket sends it. A backend of your own that redirects has to do the same, or the download name is lost.
• ?noRedirect=true. A client that can’t follow a cross-origin redirect can call /file/:id?noRedirect=true, get 200 { "location": url }, and fetch the URL itself.
• HEAD on a redirect. HEAD /file/:id answers 302 with the presigned URL, but that URL is signed for GET, so a client that repeats the HEAD against Location gets 403 from the bucket. To learn the size without the body, follow with a GET carrying Range: bytes=0-0 and read the total from Content-Range. See #131.
• Protecting the target. arocapi sandboxes the bytes it sends itself (ADR 0005), but it can’t add headers to a redirect target. Point redirects at an origin that holds no sign-ins, or have the target send Content-Security-Policy: default-src 'none'; style-src 'unsafe-inline'; sandbox (except for audio and video) and X-Content-Type-Options: nosniff itself. That includes a content route of your own, as in the signed redirect example.
• S3Storage and a CDN. A presigned URL is already on the bucket’s own origin, separate from the API’s. If the bucket sits behind a CDN on the API’s site, have the CDN add the headers, for example with a CloudFront response headers policy.

Choose delivery: 'stream' when clients can’t reach the bucket, such as a store on a private network, or when you don’t want URLs that outlive the access check. Metadata always streams from S3, whichever you choose.

Write your own

A storage backend is a plain object of two halves, and either half can be your own beside a reference backend’s. Reference → Storage backend has the contract, the deliveries and what each sends, with a signed redirect as a worked example.