vgi-rpc-go

Go implementation of the vgi_rpc framework — Apache Arrow IPC-based RPC for high-performance data services.

Built by Query.Farm

Define RPC methods with typed Go structs annotated with vgirpc struct tags. The framework derives Apache Arrow schemas from struct fields and provides server dispatch with automatic serialization/deserialization.

Key Features

• Unary RPCs with typed parameters and results via struct tags
• Producer streams for server-initiated data flows
• Exchange streams for bidirectional batch processing
• Stream headers for metadata before the first data batch
• Client-directed logging at configurable levels
• context.Context support for cancellation and deadlines
• HTTP transport with signed state tokens for stateless exchange
• ArrowSerializable interface for complex nested types

Three Method Types

Unary

A single request produces a single response. The client sends parameters, the server returns a result.

Client  ──  add(a=2, b=3)  ──▸  Server
Client  ◂──     5.0         ──  Server

Producer

The server pushes batches to the client until calling out.Finish():

Client  ──  countdown(n=3)  ──▸  Server
Client  ◂──  {value: [3]}   ──  Server
Client  ◂──  {value: [2]}   ──  Server
Client  ◂──  {value: [1]}   ──  Server
Client  ◂──    [finish]     ──  Server

Exchange

Lockstep bidirectional streaming — one request, one response, repeat:

Client  ──  transform(factor=2)  ──▸  Server
Client  ──    {value: [10]}      ──▸  Server
Client  ◂──   {result: [20]}     ──  Server
Client  ──    {value: [5]}       ──▸  Server
Client  ◂──   {result: [10]}     ──  Server
Client  ──      [close]          ──▸  Server

Installation

go get github.com/Query-farm/vgi-rpc-go/vgirpc

Quick Start

package main

import (
    "context"
    "github.com/Query-farm/vgi-rpc-go/vgirpc"
)

type GreetParams struct {
    Name string `vgirpc:"name"`
}

func main() {
    server := vgirpc.NewServer()

    vgirpc.Unary(server, "greet", func(_ context.Context, ctx *vgirpc.CallContext, p GreetParams) (string, error) {
        return "Hello, " + p.Name + "!", nil
    })

    server.RunStdio()
}

Next Steps

• Read the Guide for struct tags, streaming, HTTP transport, and more
• Browse the API Reference for all exported types and functions
• Check out the Examples for runnable programs

---

Query.Farm