Datasets

Datasets let a plugin publish structured data that other extensions can read – stat blocks, spell lists, reference tables, remote API endpoints. Extensions consume datasets through the SDK’s Data API. See Data Providers for the full model.

A dataset is either static (bundled JSON records with optional user edits) or dynamic (named URL templates to a remote API). A single dataset manifest chooses one or the other – never both.

Declaring datasets in the plugin manifest#

Point the plugin manifest’s datasets array at each dataset manifest file:

{
  "id": "@me/dnd5e-srd",
  "name": "D&D 5e SRD",
  "version": "1.0.0",
  "datasets": [
    { "$ref": "datasets/monsters.json" },
    { "$ref": "datasets/monsters-live.json" }
  ]
}

Each $ref path resolves relative to the plugin manifest. Paths that escape the plugin directory are rejected at load time.

Dataset IDs#

Overseer constructs each dataset’s ID by combining the plugin ID with the dataset manifest’s id:

@pluginId:localId

For a plugin with id: "@me/dnd5e-srd" and a dataset with id: "monsters", the resulting dataset ID is @me/dnd5e-srd:monsters. Plugins cannot claim foreign namespaces – the loader owns ID construction.

Static datasets#

Static datasets ship a JSON array of records. Extensions can read, paginate, and optionally create, update, or delete records.

Directory structure#

@me/dnd5e-srd/
  manifest.json
  datasets/
    monsters.json           -- the dataset manifest
    data/
      monsters.json         -- the records array

Dataset manifest#

{
  "id": "monsters",
  "label": "D&D 5e Monsters",
  "description": "Stat blocks for the SRD monsters.",
  "version": "1.0.0",
  "idField": "slug",
  "source": {
    "local": { "$ref": "./data/monsters.json" }
  }
}

Records file#

The file referenced by source.local.$ref must be a JSON array. Every record must carry the idField property:

[
  { "slug": "goblin", "name": "Goblin", "cr": 0.25, "hp": 7 },
  { "slug": "orc", "name": "Orc", "cr": 0.5, "hp": 15 }
]

User edits#

When an extension calls create, update, or delete through the SDK, the change is persisted to a user-owned JSON file, not the records file you shipped:

${dataDirectory}/Data/@me/dnd5e-srd/monsters.json

On read, user records are merged over bundled records – on ID collision, the user record wins. Deleting a user record that overrides a bundled one reverts the bundled version on the next read. Writes are atomic (written to .tmp then renamed), so a crash cannot corrupt the file.

Limits#

Datasets above 50,000 records are truncated to that count with a warning in the logs. Datasets whose records file is larger than 50 MB are skipped entirely (also with a warning) to protect the main process from out-of-memory crashes. If your catalog is larger, split it into multiple datasets or expose it as a dynamic dataset.

Dynamic datasets#

Dynamic datasets expose named URL templates that resolve to remote API calls. Extensions invoke an action by name with parameters; Overseer fetches on their behalf (bypassing CORS) and returns the raw JSON response.

Dataset manifest#

{
  "id": "monsters-live",
  "label": "D&D 5e Monsters (Live)",
  "description": "Queries the Open5e API.",
  "version": "1.0.0",
  "source": {
    "remote": {
      "search": "https://api.open5e.com/v2/monsters/?search={q}&cr_min={cr_min}&cr_max={cr_max}",
      "byType": "https://api.open5e.com/v2/monsters/?type={type}"
    }
  }
}

URL templates#

• {paramName} placeholders are substituted with values from the caller’s params argument.
• Values are passed through encodeURIComponent, so callers do not need to pre-encode.
• Only https:// URLs are accepted. http:// is rejected at validation time.
• GET requests only in v1. Custom headers, POST bodies, and auth are deferred.

Calling an action#

Extensions invoke actions via the SDK:

import { callDataDynamic } from '@overseer-studio/sdk';

const results = await callDataDynamic(
  '@me/dnd5e-srd:monsters-live',
  'search',
  { q: 'goblin', cr_min: 0, cr_max: 1 },
);

Overseer does not normalize the response – the consumer receives whatever the API returned.

Signaling changes#

Dynamic datasets don’t know when a remote API’s state changes. If an extension performs a write through an action, it can call invalidateData(datasetId) from @overseer-studio/sdk to signal other consumers to refetch. See Data Providers – Reactive updates.

Complete examples#

Static: D&D 5e monsters#

Directory layout:

@me/dnd5e-srd/
  manifest.json
  datasets/
    monsters.json
    data/
      monsters.json

manifest.json:

{
  "id": "@me/dnd5e-srd",
  "name": "D&D 5e SRD",
  "version": "1.0.0",
  "datasets": [
    { "$ref": "datasets/monsters.json" }
  ]
}

datasets/monsters.json:

{
  "id": "monsters",
  "label": "D&D 5e Monsters",
  "version": "1.0.0",
  "idField": "slug",
  "source": {
    "local": { "$ref": "./data/monsters.json" }
  }
}

datasets/data/monsters.json:

[
  { "slug": "goblin", "name": "Goblin", "cr": 0.25, "hp": 7 },
  { "slug": "orc", "name": "Orc", "cr": 0.5, "hp": 15 }
]

Extensions consume it as @me/dnd5e-srd:monsters.

Dynamic: Open5e live monster search#

datasets/monsters-live.json:

{
  "id": "monsters-live",
  "label": "D&D 5e Monsters (Live)",
  "version": "1.0.0",
  "source": {
    "remote": {
      "search": "https://api.open5e.com/v2/monsters/?search={q}&cr_min={cr_min}&cr_max={cr_max}",
      "byType": "https://api.open5e.com/v2/monsters/?type={type}"
    }
  }
}

Extensions consume it as @me/dnd5e-srd:monsters-live and invoke actions by name:

import { callDataDynamic } from '@overseer-studio/sdk';

await callDataDynamic('@me/dnd5e-srd:monsters-live', 'search', { q: 'goblin' });

Mixing datasets with other plugin content#

A plugin can ship datasets alongside extensions, themes, presets, and locales:

{
  "id": "@me/dnd5e-srd",
  "name": "D&D 5e SRD",
  "version": "1.0.0",
  "extensions": [
    { "$ref": "extensions/monster-viewer/manifest.json" }
  ],
  "datasets": [
    { "$ref": "datasets/monsters.json" },
    { "$ref": "datasets/spells.json" }
  ]
}

Next steps#

• Data Providers – the conceptual model behind datasets.
• Extension Manifest – Consuming datasets – declare dataset dependencies in an extension.
• SDK Reference – Data API – read datasets at runtime.