Quick Start

WDB SDK

The easiest way to get started is to connect with a remote WeaveDB rollup node with WDB SDK.

Installation

yarn add wdb-sdk

NodeJS

import { DB } from "wdb-sdk"
 
// assume you have JWK for an Arweave wallet
const db = new DB({ 
  jwk, 
  url: "http://34.18.53.73:6364", 
  hb: "http://34.18.53.73:10001"
})
 
// create a new DB instance
const id = await db.spawn()
console.log(`new DB ID: ${id}`)
 
// create users dir
await db.mkdir({
  name: "users",
  schema: { type: "object", required: ["name", "age"] },
  auth: [["set:user,del:user", [["allow()"]]]],
})
 
// add users
await db.set("set:user", { name: "Bob", age: 20 }, "users", "Bob")
await db.set("set:user", { name: "Alice", age: 30 }, "users", "Alice")
await db.set("set:user", { name: "Mike", age: 25 }, "users", "Mike")
 
// get Alice
const Alice = await db.get("users", "Alice")
 
// get users
const users = await db.get("users")
 
// sort by age
const users2 = await db.get("users", ["age", "desc"]) // [Alice, Bob, Mike]
 
// only get 2
const users3 = await db.get("users", ["age", "asc"], 2) // [Mike, Bob]
 
// delete Mike
await db.del("users", "Mike")
 
// get 2 again
const users4 = await db.get("users", ["age", "asc"], 2) // [Bob, Alice]
 
// get where age equals 30
const users5 = await db.get("users", ["age", "==", 30]) // [Alice]

Running Rollup Node

A WeaveDB rollup node can automatically start with HyperBEAM.

Clone the weavedb branch from our HyperBEAM repo.

git clone -b weavedb https://github.com/weavedb/HyperBEAM.git
cd HyperBEAM

Start HyperBEAM rebar3 shell with as weavedb.

rebar3 as weavedb shell --eval 'hb:start_mainnet(#{ port => 10001, priv_key_location => <<".wallet.json">> })'

You can explicitlyt start the WeaveDB rollup node by visiting http://localhost:10001/~weavedb@1.0/start.

Then check the rollup node status at http://localhost:6364/status.

Now you can interact with the nodes with wdb-sdk.

HyperBEAM : http://localhost:10001
WeaveDB Rollup : http://localhost:6364

Writing Tests

WAO makes testing WeaveDB absolutely easy.

test/wdb.test.js

import assert from "assert"
import { describe, it, before, after } from "node:test"
import { HyperBEAM, wait } from "wao/test"
import { DB } from "wdb-sdk"
import server from "../src/server.js"
import { genDir } from "./test-utils.js"
 
const users = {
  name: "users",
  schema: { type: "object", required: ["name", "age"] },
  auth: [["set:user,del:user", [["allow()"]]]],
}
 
const bob = { name: "Bob", age: 23 }
describe("WeaveDB SDK", () => {
  let db, hbeam
  before(async () => {
    hbeam = await new HyperBEAM({ as: ["weavedb"] }).ready()
    db = new DB({ hb: hbeam.url, jwk: hbeam.jwk })
	const dbpath = genDir()
    await wait(5000)
  })
  after(() => hbeam.kill())
  
  it("should deploy a database", async () => {
    const pid = await db.spawn()
    assert((await db.mkdir(users)).success)
    assert((await db.set("set:user", bob, "users", "bob")).success)
    assert.deepEqual(await db.get("users", "bob"), bob)
  })
})

Then run the tests.

yarn test test/wdb.test.js

WeaveDB Scan

When running local servers, you can also run a local explorer to view transactions.

git clone https://github.com/weavedb/weavedb.git
cd weavedb/scan && yarn
yarn dev

Now the explorer is runnint at localhost:3000.

We have a simple public explorer for the demo at scan.weavedb.dev.

Building Social App

Define Database

To keep it simple, we will only make one dir called posts, and allow add:post.

import { DB } from "wdb-sdk"
 
const schema = {
  type: "object",
  required: ["uid", "body", "date"],
  properties: {
    uid: { type: "string", pattern: "^[a-zA-Z0-9_-]{43}quot; },
    body: { type: "string", minLength: 1, maxLength: 280 },
    date: { type: "integer", minimum: 0, maximum: 9999999999999 },
  },
}
const auth_add_post = [
  ["mod()", { uid: "$signer", date: "$ts" }], // add uid and date
  ["allow()"] // allow anyone
]
const posts = {
  name: "posts", // dirname
  schema, // JSON Schema
  auth: [
    ["add:post", auth_add_post], // define a custom query type
  ],
}
 
const db = new DB({ jwk })
const pid = await db.spawn() // spawn a new DB instance
await db.mkdir(posts) // make posts dir
 
console.log(pid) // note the database pid

Frontend Dapp

We are going to build the simplest social app ever using NextJS!

For simplicity, use the old pages structure insted of apps.

npx create-next-app myapp && cd myapp

/pages/index.js

import { useRef, useEffect, useState } from "react"
import { DB } from "wdb-sdk"
 
export default function Home() {
  const [posts, setPosts] = useState([])
  const [body, setBody] = useState("")
  const db = useRef()
  const getPosts = async () => {
    setPosts(await db.current.get("posts", ["date", "desc"], 10))
  }
  useEffect(() => {
    void (async () => {
      db.current = new DB({ id: YOUR_DB_ID })
      await getPosts()
    })()
  }, [])
  return (
    <>
      <textarea value={body} onChange={e => setBody(e.target.value)} />
      <button
        onClick={async () => {
          await db.current.set("add:post", { body }, "posts")
          setBody("")
          await getPosts()
        }}
      >
        Post
      </button>
      {posts.map(v => (
        <article>
          <p>{v.body}</p>
          <footer>
            <time>{new Date(v.date).toLocaleString()}</time> by{" "}
            <address>{v.uid}</address>
          </footer>
        </article>
      ))}
    </>
  )
}

You might think this is too simple, but add some styles in global.css, and witness the magic!

Running Validator Node

A validator node is a separate process that handles the following steps.

Download WAL from HyperBEAM
Verify all messages and hashpaths
Compact updates with ARJSON
Calculate zkJSON sparse merkle trees
Commit to the database process
Receive $DB reward for the work

Thanks to ARJSON, only the absolute minimum bits required for full database recovery will be stored on the Arweave permanent storage, which drastically reduces the database cost.

git clone https://github.com/weavedb/weavedb.git
cd weavedb && yarn && cd hb && yarn && cd ..
yarn validator --pid DB_PID --vid VALIDATION_PID

A new validator process will be spawned if vid is not specified.

Running ZK Proof Generator Node

A zk proof generator node is a separate process that handles the following steps.

Download validated ARJSON bits from HyperBEAM or Arweave
Decode ARJSON into database structures
Calculate zkJSON sparse merkle trees
Commit the root merkle hash to EVM blockchains
Generate zkJSON proofs on demand

git clone https://github.com/weavedb/weavedb.git
cd weavedb && yarn && cd hb && yarn && mkdir -p src/circom/db/index_js
curl -L -o src/circom/db/index_0001.zkey "https://firebasestorage.googleapis.com/v0/b/weavedb-8c88c.appspot.com/o/zkp%2Fdb%2Findex_0001.zkey?alt=media&token=96c8ea6c-ea93-4b21-a345-34d28d8dda0a"
curl -L -o src/circom/db/index_js/index.wasm "https://firebasestorage.googleapis.com/v0/b/weavedb-8c88c.appspot.com/o/zkp%2Fdb%2Findex.wasm?alt=media&token=19d0c4ca-946a-482e-b1bb-61a01c48ba3d"
cd ..
yarn zkp --vid VALIDATION_PID

You can get zk proofs at http://localhost:6365/zkp.

const { zkp, zkhash } = await fetch("http://localhost:6365/zkp", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({ dir: "posts", doc: "A", path: "body" }),
}).then(r => r.json())

Query from Ethereum with ZK Proof

You can query WeaveDB from Ethereum Solidity contarcts.

Since we are working on the local environment, let's create a test with Hardhat.

mkdir zkdb && cd zkdb && npx hardhat init
npm install zkjson wdb-sdk

We will create ZKDB contract by extending the simple optimistic zk rollup contract from the zkjson package, which comes with the zkQuery interface.

/contracts/ZKDB.sol

// SPDX-License-Identifier: MIT
 
pragma solidity >=0.7.0 <0.9.0;
 
import "zkjson/contracts/OPRollup.sol";
 
interface VerifierDB {
  function verifyProof(uint[2] calldata _pA, uint[2][2] calldata _pB, uint[2] calldata _pC, uint[16] calldata _pubSignals) view external returns (bool);
}
 
contract ZKDB is OPRollup {
  uint constant SIZE_PATH = 4;
  uint constant SIZE_VAL = 8;
  address public verifierDB;
 
  constructor (address _verifierDB, address _committer){
    verifierDB = _verifierDB;
    committer = _committer;
  }
  
  function validateQuery(uint[] memory path, uint[] memory zkp) private view returns(uint[] memory){
    verify(zkp, VerifierDB.verifyProof.selector, verifierDB);
    return _validateQueryRU(path, zkp, SIZE_PATH, SIZE_VAL);    
  }
 
  function qInt (uint[] memory path, uint[] memory zkp) public view returns (int) {
    uint[] memory value = validateQuery(path, zkp);
    return _qInt(value);
  }
 
  function qFloat (uint[] memory path, uint[] memory zkp) public view returns (uint[3] memory) {
    uint[] memory value = validateQuery(path, zkp);
    return _qFloat(value);
  }
 
  function qRaw (uint[] memory path, uint[] memory zkp) public view returns (uint[] memory) {
    uint[] memory value = validateQuery(path, zkp);
    return _qRaw(value);
  }
  
  function qString (uint[] memory path, uint[] memory zkp) public view returns (string memory) {
    uint[] memory value = validateQuery(path, zkp);
    return _qString(value);
  }
 
  function qBool (uint[] memory path, uint[] memory zkp) public view returns (bool) {
    uint[] memory value = validateQuery(path, zkp);
    return _qBool(value);
  }
  
  function qNull (uint[] memory path, uint[] memory zkp) public view returns (bool) {
    uint[] memory value = validateQuery(path, zkp);
    return _qNull(value);
  }
 
  function qCond (uint[] memory path, uint[] memory cond, uint[] memory zkp) public view returns (bool) {
    uint[] memory value = validateQuery(path, zkp);
    return _qCond(value, cond);
  }
 
  function qCustom (uint[] memory path, uint[] memory path2, uint[] memory zkp) public view returns (int) {
    uint[] memory value = validateQuery(path, zkp);
    return getInt(path2, value);
  }
}

Now, you can commit zkhash, generate zk proofs from a zk prover node, then query WeaveDB from Solidity with the zkp.

const { loadFixture } = require("@nomicfoundation/hardhat-network-helpers")
const { expect } = require("chai")
const { toIndex, path } = require("zkjson")
const { DB } = require("wdb-sdk")
 
const wait = ms => new Promise(res => setTimeout(() => res(), ms))
async function deploy() {
  const [committer] = await ethers.getSigners()
  const VerifierDB = await ethers.getContractFactory(
    "zkjson/contracts/verifiers/verifier_db.sol:Groth16VerifierDB",
  )
  const verifierDB = await VerifierDB.deploy()
  const ZKDB = await ethers.getContractFactory("ZKDB")
  return (zkdb = await ZKDB.deploy(verifierDB.target, committer.address))
}
 
describe("ZKDB", function () {
  this.timeout(0)
  it("should query WeaveDB from Solidity", async function () {
    const zkdb = await loadFixture(deploy)
    const db = new DB({ jwk, id: DB_ID })
    await db.set("add:post", { body: "my first post!" }, "posts")
    const post = (await db.cget("posts", ["date", "desc"]))[0]
    await wait(20000)
    const { zkp, zkhash, dirid } = await fetch("http://localhost:6365/zkp", {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({ dir: "posts", doc: post.id, path: "body" }),
    }).then(r => r.json())
    await zkdb.commitRoot(zkhash)
    expect(
      await zkdb.qString([dirid, toIndex(post.id), ...path("body")], zkp),
    ).to.eql("my first post!")
  })
})

This ZKDB demo demonstrates a simplified version of the zk proof generating process. It uses the NORU (No Rollup) contract to omit the root hash commitments to bypass the need of keeping 2 chains in sync.

Query from AOS Processes

You can query WeaveDB from any AO processes including AOS Lua scripts. We will use WAO SDK for simplicity.

AOS processes can Send a message with Query action to receive() from the WeaveDB validation process.

import { AO } from "wao"
 
const lua_script = `
Handlers.add("Query", "Query", function (msg)
  local data = Send({ 
    Target = msg.DB, 
	Action = "Query", 
	Query = msg.Query
  }).receive().Data
  msg.reply({ Data = data })
end)`
 
const ao = await new AO({ module_type: "mainnet", hb: hb_url }).init(jwk)
const { p } = await ao.deploy({ src_data: lua_script })
const data = await p.m("Query", {
  DB: validation_pid,
  Query: JSON.stringify(["posts"]),
})
console.log(JSON.parse(data))