1. The High-Level Pipeline: Text to Execution
Computers do not understand human-readable source code like let x = 10 + 20. They only execute low-level machine instructions. An interpreter acts as a bridge that reads human code, validates its structure, translates it into intermediate steps, and executes it.
RuLang processes source code in four structured phases:
Fig 1. RuLang Engine Pipeline Flow
Scans individual source characters and groups them into meaningful words called Tokens (e.g., keywords, numbers, math operators).
Verifies grammar rules and organizes raw tokens into a nested structure called an Abstract Syntax Tree (AST).
Flattens the hierarchical tree into a linear sequence of compact 8-bit operations (Opcodes) that run quickly in memory.
A virtual processor loop that fetches each byte, uses an evaluation stack to calculate values, and triggers actions.
2. Parsing: Turning Text into Trees (AST)
Consider an arithmetic expression: 3 + 4 * 5. Because multiplication has higher precedence than addition, standard left-to-right evaluation would give an incorrect result ((3 + 4) * 5 = 35 instead of 3 + (4 * 5) = 23).
The parser solves this by constructing an Abstract Syntax Tree (AST) where operation priority dictates tree depth. Deeper nodes execute first.
Fig 2. Abstract Syntax Tree (AST) for "3 + 4 * 5"
In RuLang, every AST node is represented as a tagged C structure holding specific node metadata:
// Simple C AST Representation
typedef enum {
AST_LITERAL, // Represents numbers, strings, booleans
AST_BINARY, // Binary expression (+, -, *, /)
AST_IDENTIFIER // Variable names
} ASTNodeType;
typedef struct Node {
ASTNodeType type;
union {
double number_value;
struct {
struct Node* left;
int operator_type;
struct Node* right;
} binary;
} as;
} Node;3. Compiling AST Nodes to Linear Bytecode
Tree traversal at runtime can be inefficient due to constant pointer chasing across fragmented memory addresses. RuLang addresses this by flattening the AST into linear 8-bit operational instructions called Bytecode.
The compiler visits nodes using post-order depth-first traversal (left leaf, right leaf, operator). This structure places values onto the evaluation stack before applying operators.
// RuLang Compiler Recursion (Excerpts from compiler.c)
bool compile_expression(RuCompiler *compiler, Node *node) {
if (node->type == AST_LITERAL) {
// Emit code to load a constant value
emit_constant(compiler, node->as.number_value);
return true;
}
if (node->type == AST_BINARY) {
// 1. Visit left operand (e.g., 4)
compile_expression(compiler, node->as.binary.left);
// 2. Visit right operand (e.g., 5)
compile_expression(compiler, node->as.binary.right);
// 3. Emit instruction for operator
switch (node->as.binary.operator_type) {
case TOK_PLUS: write_byte(compiler, OP_ADD); break;
case TOK_STAR: write_byte(compiler, OP_MUL); break;
}
return true;
}
}For 3 + 4 * 5, the compiler produces this flat stream of opcode bytes:
0000 OP_CONSTANTIndex 0 (Value 3)Push 3 onto VM Stack0002 OP_CONSTANTIndex 1 (Value 4)Push 4 onto VM Stack0004 OP_CONSTANTIndex 2 (Value 5)Push 5 onto VM Stack0006 OP_MULNonePops 5 & 4, Pushes 200007 OP_ADDNonePops 20 & 3, Pushes 230008 OP_RETURNNoneReturn stack top (23)4. The Virtual Machine Evaluation Loop
The Virtual Machine is an execution loop that processes bytecode arrays sequentially using an instruction pointer (ip) and an evaluation stack array (stack) managed by a stack pointer (sp).
Fig 3. Step-by-Step Execution of "3 + 4 * 5" on VM Stack
Here is a look at the inner C dispatch loop in RuLang's VM engine:
// Inner Interpreter Loop (Excerpt from vm.c)
void vm_run(RuVM *vm) {
RuFrame *frame = &vm->frames[vm->frame_count - 1];
for (;;) {
uint8_t instruction = *frame->ip++; // Fetch next byte opcode
switch (instruction) {
case OP_CONSTANT: {
RuObject *constant = read_constant(frame);
vm_push(vm, constant);
break;
}
case OP_MUL: {
double b = vm_pop(vm)->number;
double a = vm_pop(vm)->number;
vm_push(vm, create_number(a * b));
break;
}
case OP_ADD: {
double b = vm_pop(vm)->number;
double a = vm_pop(vm)->number;
vm_push(vm, create_number(a + b));
break;
}
case OP_RETURN: {
return; // Execution completes
}
}
}
}5. Lexical Closures & Variable Scoping
When a inner function references a variable declared in an outer scope, that variable must remain available even after the parent function returns.
// RuLang Closure Example
fn make_adder(x) {
fn add(y) {
return x + y; // 'x' comes from make_adder's scope!
}
return add;
}
let add5 = make_adder(5);
print(add5(10)); // Prints 15RuLang implements closures using a specialized heap structure called an Upvalue.
- Open Upvalue: While the parent function is executing,
xsits directly on the active stack slot. The upvalue points to that stack memory address. - Closed Upvalue: When
make_adderfinishes and its stack frame is recycled, the VM automatically movesxoff the stack and copies it into heap memory insideRuUpvalueObject.
// Excerpt from vm.c: Transitioning stack variables to heap memory
static void close_upvalues(RuVM *vm, RuObject **last_slot) {
while (vm->open_upvalues != NULL && vm->open_upvalues->location >= last_slot) {
RuUpvalueObject *upvalue = vm->open_upvalues;
// Copy value from stack directly into the heap object
upvalue->closed = *upvalue->location;
// Point location reference to internal heap variable
upvalue->location = &upvalue->closed;
// Unlink from open upvalue chain
vm->open_upvalues = upvalue->next;
}
}Summary
By combining a Recursive Descent frontend with an AST-to-bytecode compiler pass and a stack-based VM, RuLang converts raw text strings into fast, structured execution loops in standard C.