Advanced Start with ZK

In this advanced tutorial, you will build a database for a social dapp with zk circuits, and query it from Ethereum, as well as AO.

create a db project using the web-cli create command.

npx wdb-cli create mydb && cd mydb
yarn add wdb-sdk

Define Database

To keep it simple, we will only make one dir called posts, and allow add:post.

/db/schema.js

export default {
  posts: {
    type: "object",
    required: ["uid", "body", "date"],
    properties: {
      uid: { type: "string", pattern: "^[a-zA-Z0-9_-]{43}quot; },
      body: { type: "string", minLength: 1, maxLength: 240 },
      date: { type: "integer", minimum: 0, maximum: 9999999999999 },
    },
  },   
}

/db/auth.js

export default {
  posts: [
    [
      "add:post",
      [
        ["mod()", { uid: "$signer", date: "$ts" }], // add uid and date
        ["allow()"], // allow anyone
      ],
    ],
  ],
}

/db/indexes.js

export default {}

/db/triggers.js

export default {}

Make sure you are running a local rollup node and a HyperBEAM node, then have .wallet.json in the app root directory.

Let's deploy the DB.

yarn deploy --wallet .wallet.json

Frontend Dapp

We are going to build the simplest social app ever using NextJS!

For simplicity, use the old pages structure insted of apps.

npx create-next-app myapp && cd myapp

/pages/index.js

import { useRef, useEffect, useState } from "react"
import { DB } from "wdb-sdk"
 
export default function Home() {
  const [posts, setPosts] = useState([])
  const [body, setBody] = useState("")
  const db = useRef()
  const getPosts = async () => {
    setPosts(await db.current.get("posts", ["date", "desc"], 10))
  }
  useEffect(() => {
    void (async () => {
      db.current = new DB({ id: YOUR_DB_ID })
      await getPosts()
    })()
  }, [])
  return (
    <>
      <textarea value={body} onChange={e => setBody(e.target.value)} />
      <button
        onClick={async () => {
          await db.current.set("add:post", { body }, "posts")
          setBody("")
          await getPosts()
        }}
      >
        Post
      </button>
      {posts.map(v => (
        <article>
          <p>{v.body}</p>
          <footer>
            <time>{new Date(v.date).toLocaleString()}</time> by{" "}
            <address>{v.uid}</address>
          </footer>
        </article>
      ))}
    </>
  )
}

You might think this is too simple, but add some styles in global.css, and witness the magic!

Add .env.local.

/.env.local

NEXT_PUBLIC_DB_ID="bee90xprxmqrrqxqcrxfsjex59wtww2w0isa-svwvco"
NEXT_PUBLIC_RU_URL="https://db-demo.wdb.ae:10003"
NEXT_PUBLIC_ZKP_URL="https://zkp-demo.wdb.ae:10004"
NEXT_PUBLIC_HB_URL="https://hb-demo.wdb.ae:10002"
NEXT_PUBLIC_SCAN_URL="https://scan.weavedb.dev"
NEXT_PUBLIC_ETH_CONTRACT="0x22f327A810aEEBdfe07379096efd66b698D5b7C5"

Run the app.

yarn dev --port 4000

Now the app is runnint at http://localhost:4000.

Running Validator Node

A validator node is a separate process that handles the following steps.

Download WAL from HyperBEAM
Verify all messages and hashpaths
Compact updates with ARJSON
Calculate zkJSON sparse merkle trees
Commit to the database process
Receive $DB reward for the work

Thanks to ARJSON, only the absolute minimum bits required for full database recovery will be stored on the Arweave permanent storage, which drastically reduces the database cost.

git clone https://github.com/weavedb/weavedb.git
cd weavedb && yarn && cd hb && yarn && cd ..
yarn validator --pid DB_PID --vid VALIDATION_PID

A new validator process will be spawned if vid is not specified.

Running ZK Proof Generator Node

A zk proof generator node is a separate process that handles the following steps.

Download validated ARJSON bits from HyperBEAM or Arweave
Decode ARJSON into database structures
Calculate zkJSON sparse merkle trees
Commit the root merkle hash to EVM blockchains
Generate zkJSON proofs on demand

git clone https://github.com/weavedb/weavedb.git
cd weavedb && yarn && cd hb && yarn && mkdir -p src/circom/db/index_js
curl -L -o src/circom/db/index_0001.zkey "https://firebasestorage.googleapis.com/v0/b/weavedb-8c88c.appspot.com/o/zkp%2Fdb%2Findex_0001.zkey?alt=media&token=96c8ea6c-ea93-4b21-a345-34d28d8dda0a"
curl -L -o src/circom/db/index_js/index.wasm "https://firebasestorage.googleapis.com/v0/b/weavedb-8c88c.appspot.com/o/zkp%2Fdb%2Findex.wasm?alt=media&token=19d0c4ca-946a-482e-b1bb-61a01c48ba3d"
cd ..
yarn zkp --vid VALIDATION_PID

You can get zk proofs at http://localhost:6365/zkp.

const { zkp, zkhash } = await fetch("http://localhost:6365/zkp", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({ dir: "posts", doc: "A", path: "body" }),
}).then(r => r.json())

Query from Ethereum with ZK Proof

You can query WeaveDB from Ethereum Solidity contarcts.

Since we are working on the local environment, let's create a test with Hardhat.

mkdir zkdb && cd zkdb && npx hardhat init
npm install zkjson wdb-sdk

We will create ZKDB contract by extending the simple optimistic zk rollup contract from the zkjson package, which comes with the zkQuery interface.

/contracts/ZKDB.sol

// SPDX-License-Identifier: MIT
 
pragma solidity >=0.7.0 <0.9.0;
 
import "zkjson/contracts/OPRollup.sol";
 
interface VerifierDB {
  function verifyProof(uint[2] calldata _pA, uint[2][2] calldata _pB, uint[2] calldata _pC, uint[16] calldata _pubSignals) view external returns (bool);
}
 
contract ZKDB is OPRollup {
  uint constant SIZE_PATH = 4;
  uint constant SIZE_VAL = 8;
  address public verifierDB;
 
  constructor (address _verifierDB, address _committer){
    verifierDB = _verifierDB;
    committer = _committer;
  }
  
  function validateQuery(uint[] memory path, uint[] memory zkp) private view returns(uint[] memory){
    verify(zkp, VerifierDB.verifyProof.selector, verifierDB);
    return _validateQueryRU(path, zkp, SIZE_PATH, SIZE_VAL);    
  }
 
  function qInt (uint[] memory path, uint[] memory zkp) public view returns (int) {
    uint[] memory value = validateQuery(path, zkp);
    return _qInt(value);
  }
 
  function qFloat (uint[] memory path, uint[] memory zkp) public view returns (uint[3] memory) {
    uint[] memory value = validateQuery(path, zkp);
    return _qFloat(value);
  }
 
  function qRaw (uint[] memory path, uint[] memory zkp) public view returns (uint[] memory) {
    uint[] memory value = validateQuery(path, zkp);
    return _qRaw(value);
  }
  
  function qString (uint[] memory path, uint[] memory zkp) public view returns (string memory) {
    uint[] memory value = validateQuery(path, zkp);
    return _qString(value);
  }
 
  function qBool (uint[] memory path, uint[] memory zkp) public view returns (bool) {
    uint[] memory value = validateQuery(path, zkp);
    return _qBool(value);
  }
  
  function qNull (uint[] memory path, uint[] memory zkp) public view returns (bool) {
    uint[] memory value = validateQuery(path, zkp);
    return _qNull(value);
  }
 
  function qCond (uint[] memory path, uint[] memory cond, uint[] memory zkp) public view returns (bool) {
    uint[] memory value = validateQuery(path, zkp);
    return _qCond(value, cond);
  }
 
  function qCustom (uint[] memory path, uint[] memory path2, uint[] memory zkp) public view returns (int) {
    uint[] memory value = validateQuery(path, zkp);
    return getInt(path2, value);
  }
}

Now, you can commit zkhash, generate zk proofs from a zk prover node, then query WeaveDB from Solidity with the zkp.

const { loadFixture } = require("@nomicfoundation/hardhat-network-helpers")
const { expect } = require("chai")
const { toIndex, path } = require("zkjson")
const { DB } = require("wdb-sdk")
 
const wait = ms => new Promise(res => setTimeout(() => res(), ms))
async function deploy() {
  const [committer] = await ethers.getSigners()
  const VerifierDB = await ethers.getContractFactory(
    "zkjson/contracts/verifiers/verifier_db.sol:Groth16VerifierDB",
  )
  const verifierDB = await VerifierDB.deploy()
  const ZKDB = await ethers.getContractFactory("ZKDB")
  return (zkdb = await ZKDB.deploy(verifierDB.target, committer.address))
}
 
describe("ZKDB", function () {
  this.timeout(0)
  it("should query WeaveDB from Solidity", async function () {
    const zkdb = await loadFixture(deploy)
    const db = new DB({ jwk, id: DB_ID })
    await db.set("add:post", { body: "my first post!" }, "posts")
    const post = (await db.cget("posts", ["date", "desc"]))[0]
    await wait(20000)
    const { zkp, zkhash, dirid } = await fetch("http://localhost:6365/zkp", {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({ dir: "posts", doc: post.id, path: "body" }),
    }).then(r => r.json())
    await zkdb.commitRoot(zkhash)
    expect(
      await zkdb.qString([dirid, toIndex(post.id), ...path("body")], zkp),
    ).to.eql("my first post!")
  })
})

This ZKDB demo demonstrates a simplified version of the zk proof generating process. It uses the NORU (No Rollup) contract to omit the root hash commitments to bypass the need of keeping 2 chains in sync.

Query from AOS Processes

You can query WeaveDB from any AO processes including AOS Lua scripts. We will use WAO SDK for simplicity.

AOS processes can Send a message with Query action to receive() from the WeaveDB validation process.

import { AO } from "wao"
 
const lua_script = `
Handlers.add("Query", "Query", function (msg)
  local data = Send({ 
    Target = msg.DB, 
	Action = "Query", 
	Query = msg.Query
  }).receive().Data
  msg.reply({ Data = data })
end)`
 
const ao = await new AO({ module_type: "mainnet", hb: hb_url }).init(jwk)
const { p } = await ao.deploy({ src_data: lua_script })
const data = await p.m("Query", {
  DB: validation_pid,
  Query: JSON.stringify(["posts"]),
})
console.log(JSON.parse(data))