A flexible AST, Codegen and Virtual Machine library
for building your own toy language, scripting engines and DSLs.

nimble install vancode

API reference

About

VanCode is a tiny library for building bytecode interpreters and virtual machines in Nim. It provides a simple and efficient way to define and execute bytecode instructions, making it easier to create custom programming languages, scripting engines and DSLs (domain-specific languages).

This is a bring-your-own-parsing library, so it doesn't come with a built-in Parser or Lexer. When implementing your own Parser you must use the VanCode AST provided by the library. No worries, most of the AST is pretty self-explanatory and easy to use.

It's also pretty flexible so you can easily extend it with your own custom nodes via Nim's macro system at compile-time without having to modify the library's source code.

😍 Key Features

• Bring-your-own Lexer and Parser for maximum flexibility
• Built-in AST (Abstract Syntax Tree) representation
• Simple and efficient bytecode instruction definition and execution
• Support for multiple data types and operations
• FFI for calling Nim code or external libraries
• Generate self-contained executables
• JIT (Just-In-Time) compilation for improved performance
• (Ahead-Of-Time) compilation for static binaries
• Built-in package manager for easy distribution and installation
• Written in Nim language

Note

VanCode is far from being a complete solution, I'm planning to add more features and improvements as I go, while still learning about how to design a good flexible interpreter and VM.

Examples

Let's showcase some cool examples!

Calculator example

Here is a simple calculator example that demonstrates how to use VanCode to construct the AST for a simple expression, pass it trough the code generator to produce bytecode, and then execute it in the VM.

import std/options
import vancode/interpreter/[ast, codegen, chunk, value, vm, sym]
import vancode/interpreter/stdlib/syslib

# 1. Build the AST for: 1 + 2 * 3
let astExpr =
  ast.newCall(
    ast.newIdent("echo"),
    ast.newTree(nkInfix,
      ast.newIdent("+"),
      ast.newIntLit(1),
      ast.newTree(nkInfix,
        ast.newIdent("*"),
        ast.newIntLit(2),
        ast.newIntLit(3)
      )
    )
  )

# 2. Wrap in a script AST node
let astScript = Ast(
  sourcePath: "calculator",
  nodes: @[astExpr]
)

# 3. Prepare codegen context
let mainChunk = newChunk("calculator")
let script = newScript(mainChunk)

let module = newModule("calculator", some("calculator"))
block init_system_module:
  module.initSystemTypes()
  script.initSystemOps(module)

  # Adding a FFI proc for `echo` so we can see the output of the calculation
  script.addProc(module, "echo", @[paramDef("x", ttyInt)], ttyVoid,
    proc (args: StackView, argc: int): Value =
      echo args[0].intVal)

  script.addProc(module, "echo", @[paramDef("x", ttyFloat)], ttyVoid,
    proc (args: StackView, argc: int): Value =
      echo args[0].floatVal)

# 4. Generate bytecode from AST
let gen = initCompiler(script, module, mainChunk, nil, nil)
gen.genScript(astScript, none(string))

# 5. Run in the VM
let vmInstance = newVm()
discard vmInstance.interpret(script, mainChunk)

👉 Check the examples/calculator.nim for a more complete REPL implementation of the calculator example.

What's included

Codegen prebuilt operations

VanCode comes with a set of prebuilt operations for common tasks like arithmetic, logic, control flow, function calls and more. Check Codegen API reference for the full list of prebuilt operations and how to use them.

Find the system operation functions in the vancode/interpreter/stdlib/syslib.nim module. You can use these prebuilt functions to quickly add common functionality to your language without having to implement them from scratch.

FFI support

VanCode provides a simple FFI (Foreign Function Interface) mechanism that allows you to call Nim code or C libraries directly from your Script modules. From there you can easily expose any functionality you want to your scripts.

AST prebuilt nodes

VanCode also comes with a set of prebuilt AST nodes for common constructs like literals, variables, function calls, control flow and more. Check AST API reference for the full list.

Operation Codes

VanCode's VM comes with a set of prebuilt instructions for executing the bytecode generated by the Codegen. These instructions cover a wide range of operations, from basic arithmetic and logic to control flow and function calls. Check Chunk API reference for more details.

Extend AST, Codegen and VM with Voodoo

VanCode provides a powerful and flexible way to extend the AST, Codegen and VM with your own custom nodes, instructions and operations using Nim's macro system.

todo example here

Note

For a full example of how to use Voodoo to extend the AST, Codegen and VM, check the Tim Engine transformers module.

Projects using VanCode

• Tim Engine - A beautiful template engine and DSL for generating HTML templates

Roadmap

• AOT/JIT compilation using gccjit bindings
• Self-contained executable generation (Similar to how Node/Bun generate self-contained executables)
• VM Hot code optimization
• Add more Voodoo flexibility and extensibility features to the AST, Codegen and VM

Notes:

• https://vivekn.dev/blog/bytecode-vm-scratch/
• https://github.com/liquidev/hayago - VanCode contains work from hayago, a very interesting project that is no longer maintained but provides a good reference for Nim-based bytecode VMs. Cheers to the author! 😻

❤ Contributions & Support

• 🐛 Found a bug? Create a new Issue
• 👋 Wanna help? Fork it!
• 😎 Get €20 in cloud credits from Hetzner

🎩 License

LGPLv3 license. Made by Humans from OpenPeeps.
Copyright OpenPeeps & Contributors — All rights reserved.