Skip to content

Working with JSON

The @dpkit/json package provides comprehensive support for loading and saving data in JSON and JSONL (JSON Lines) formats. It leverages Polars DataFrames for efficient data processing and supports flexible data transformations through dialect configurations.

The JSON package is part of dpkit’s modular architecture:

Terminal window
npm install @dpkit/json
import { loadJsonTable } from "@dpkit/json"
// Load from local file
const table = await loadJsonTable({ path: "data.json" })
// Load from remote URL
const table = await loadJsonTable({
path: "https://example.com/data.json"
})
// Load multiple files (concatenated)
const table = await loadJsonTable({
path: ["file1.json", "file2.json"]
})
import { loadJsonlTable } from "@dpkit/json"
// Load JSONL (JSON Lines) format
const table = await loadJsonlTable({ path: "data.jsonl" })
import { saveJsonTable, saveJsonlTable } from "@dpkit/json"
// Save as JSON
await saveJsonTable(table, { path: "output.json" })
// Save as JSONL
await saveJsonlTable(table, { path: "output.jsonl" })

The package supports two main JSON formats:

Standard JSON arrays of objects:

[
{"id": 1, "name": "Alice"},
{"id": 2, "name": "Bob"}
]

Newline-delimited JSON objects:

{"id": 1, "name": "Alice"}
{"id": 2, "name": "Bob"}

Dialects provide flexible data transformation capabilities:

Extract data from nested objects using the property option:

// Input: {"users": [{"id": 1, "name": "Alice"}]}
const table = await loadJsonTable({
path: "data.json",
dialect: { property: "users" }
})

Select specific fields using itemKeys:

// Only load 'name' field from each record
const table = await loadJsonTable({
path: "data.json",
dialect: { itemKeys: ["name"] }
})

Handle CSV-style array data with itemType: "array":

// Input: [["id", "name"], [1, "Alice"], [2, "Bob"]]
const table = await loadJsonTable({
path: "data.json",
dialect: { itemType: "array" }
})