One RPC, every layer.

This is the marquee page of Layer 3. We trace one RPC — bdev_lvol_create — from the Go call site in diskengine, through the JSON-RPC socket, into the bdev / lvol layer in C, all the way to the actual blob create and back. Every step shows the real code with the line numbers you'll need to follow along. By the end you'll be able to read any other RPC the same way.

sequenceDiagram
participant Go as Go: Client.Call
participant Enc as json.Encoder
participant Sock as Unix socket
participant Poll as spdk_jsonrpc_server_poll
participant Parse as jsonrpc_parse_request
participant Handler as jsonrpc_handler
participant Reg as g_rpc_methods
participant RPCH as rpc_bdev_lvol_create
participant Lvol as vbdev_lvol_create
participant Blob as spdk_blob_create
participant Dec as json.Decoder
Go->>Enc: createRequest
Enc->>Sock: encode (bytes)
Sock->>Poll: recv
Poll->>Parse: jsonrpc_parse_request
Parse->>Handler: handle_request(method, params)
Handler->>Reg: _get_rpc_method
Reg-->>Handler: rpc_bdev_lvol_create
Handler->>RPCH: invoke(request, params)
RPCH->>RPCH: spdk_json_decode_object (params)
RPCH->>RPCH: vbdev_get_lvol_store_by_uuid_xor_name
RPCH->>Lvol: vbdev_lvol_create (async)
Lvol->>Blob: spdk_bs_create_blob
Blob-->>Lvol: completion (cb on reactor)
Lvol-->>RPCH: rpc_bdev_lvol_create_cb
RPCH-->>Poll: spdk_jsonrpc_end_result
Poll->>Sock: send response bytes
Sock->>Dec: read
Dec-->>Go: Response(Result: uuid)

fig. 1   The full life of one RPC. The handler returns control to the reactor after queueing the async bdev work; the response is written from the lvol completion callback, which itself runs on the reactor thread.

1. The Go call site

The trace starts in diskengine's Go client. A caller — say, the lvol create handler in internal/handlers/... — has a spdkclient.Client instance and a spdkclient.BdevLvolCreateParams struct. It calls:

uuid, err := spdkClient.BdevLvolCreate(spdkclient.BdevLvolCreateParams{
    LvolName:      "vol-0001",
    SizeInMib:     1024,
    ThinProvision: &thinTrue,
    LvstoreName:   &lvsName,
})

The wrapper is at BdevLvolCreate:42 . It is one of the simplest wrappers in the file — single call, string result, no nested unmarshaling. Three jobs:

• c.Call("bdev_lvol_create", params) — generic dispatch. Note the method name is the exact string SPDK registered via SPDK_RPC_REGISTER. A typo here is a runtime error, not a compile error.

• resp.Result.(string) — the result field is any (interface). The C handler returned a string (the lvol UUID), so we type-assert. If SPDK returned a different shape, this would panic.

• The wrapped error includes the method name — useful when a single handler does multiple RPCs.

The struct itself is at BdevLvolCreateParams:14 :

Important details:

• LvolName and SizeInMib are non-pointer types — they're required fields. If the JSON omits either, SPDK's decoder will reject the request with SPDK_JSONRPC_ERROR_INVALID_PARAMS.

• The other four are pointer types: pointer means "optional" — if the field is nil in Go, the omitempty tag drops it from the JSON entirely. The C decoder treats missing fields as not-present, which is what the true in the decoder table at

  module/bdev/lvol/vbdev_lvol_rpc.c:302

  means (the 4th arg to the decoder registration).

• The field names in the JSON tags are what the C handler expects. "lvol_name", not "lvolName": SPDK uses snake_case throughout.

2. Encoding: Go struct → JSON bytes

Client.Call at

is the actual workhorse. It does seven things, and three of them are relevant here:

• id := c.requestId.Add(1) — atomically bump the per-Client counter. This is what allows concurrent Call invocations to issue distinct IDs, but see the warning below.

• c.codec.encoder.Encode(request) — JSON-encode the Request struct, write to the socket. The encoder wraps net.Conn, so this is a direct kernel write(2).

• c.codec.decoder.Decode(response) — read one JSON value from the socket, decode into the Response struct. Blocks.

createRequest (at client.go:172) builds the Request struct with four fields: jsonrpc, method, params, id. Note ID is uint64 in Go, int in the response. The mismatch check at client.go:85 does uint64(response.ID) to cast. JSON numbers are not type-tagged — the wire format doesn't carry a distinction.

3. Bytes on the wire: the request envelope

For the BdevLvolCreate call above, the actual bytes that go out on the Unix socket look like this:

{"jsonrpc":"2.0","id":17,"method":"bdev_lvol_create","params":{"lvol_name":"vol-0001","size_in_mib":1024,"thin_provision":true,"lvs_name":"lvs0"}}

Two things to note that the encoder does and that matter:

• The encoder appends a newline. json.Encoder.Encode always ends with a \n. JSON itself doesn't require it, but line-delimited JSON is a convention, and SPDK doesn't care — the JSON parser doesn't look at the trailing newline because it's outside the top-level value. The newline does land in the recv buffer, and if the request doesn't end with the newline, the parser still works.

• Field order is Go's struct order. json tags preserve struct order, so the bytes are jsonrpc, id, method, params — the order SPDK's decoder happens to expect but does not require. Reordering the Go struct would not break anything.

4. SPDK recv: the bytes land in C

The bytes flow into the SPDK process via the Unix socket. Inside the SPDK app, the reactor thread is sitting in spdk_jsonrpc_server_poll. At lib/jsonrpc/jsonrpc_server_tcp.c:389 it calls jsonrpc_server_conn_recv:

5. JSON parse: from bytes to spdk_json_val tree

The two-pass parse happens at lib/jsonrpc/jsonrpc_server.c:171 :

6. Method lookup: name → function pointer

parse_single_request finishes by calling jsonrpc_server_handle_request at lib/jsonrpc/jsonrpc_server_tcp.c:226 . That's a one-liner that calls the handle_request function pointer set when the server was created. For SPDK's high-level server, that pointer is jsonrpc_handler at lib/rpc/rpc.c:103 :

7. Handler dispatch: rpc_bdev_lvol_create

The handler is the function pointer that SPDK_RPC_REGISTER("bdev_lvol_create", rpc_bdev_lvol_create, SPDK_RPC_RUNTIME) registered at ELF constructor time. Its body is at module/bdev/lvol/vbdev_lvol_rpc.c:331 .

8. Inside the handler: parameter decoding, validation, bdev call

The handler body, in order, does six things:

Decode params (line 342)

spdk_json_decode_object walks the spdk_json_val tree for params and matches each key against the decoder table. For each match, it calls the appropriate spdk_json_decode_* function (e.g. spdk_json_decode_string) which writes the decoded value into the struct field at the given offset. If any required field is missing or the type is wrong, the function returns non-zero and the handler sends an SPDK_JSONRPC_ERROR_INTERNAL_ERROR with the message "spdk_json_decode_object failed".

Resolve lvstore (line 351)

The Go client supplied either uuid or lvs_name (XOR — exactly one). The helper at

enforces this and looks up the lvstore. Failure sends the -errno as the error code and strerror(-rc) as the message.

Translate clear_method (line 357)

clear_method is a free-form string in the JSON ("none", "unmap", "write_zeroes"). The handler maps it to an enum lvol_clear_method. Unknown values produce an -EINVAL error with the message "Invalid clean_method option" (note: "clean" not "clear" — typo in the message).

Call the bdev layer (line 372)

This is the actual work:

rc = vbdev_lvol_create(lvs, req.lvol_name, req.size_in_mib * 1024 * 1024,
                       req.thin_provision, clear_method,
                       rpc_bdev_lvol_create_cb, request);

The MiB → bytes conversion happens here. The handler multiplies size_in_mib by 1024 * 1024 before passing it down. The callback rpc_bdev_lvol_create_cb is what gets invoked when the async work completes.

The interesting bit is the return value: rc < 0 means "synchronous failure" (couldn't even queue the work) and the handler sends an error response. rc == 0 means "queued successfully" — the actual response will come from rpc_bdev_lvol_create_cb later, on the reactor.

9. The bdev layer: vbdev_lvol_create

vbdev_lvol_create is the bridge from the RPC layer to the lvol subsystem. It is in vbdev_lvol.c, and it ultimately calls into the blobstore. The shape of the function:

1. Validate inputs (size alignment, name uniqueness).
2. Look up the underlying bdev for the lvstore.
3. Call spdk_bs_create_blob (asynchronous — submits a metadata write and returns).
4. The blobstore eventually posts completion. The completion runs spdk_lvol_create_on_bs_cb.
5. That callback creates the spdk_lvol wrapper struct, registers it as a virtual bdev with spdk_bdev_register, and calls our rpc_bdev_lvol_create_cb (passed as the cb_arg).

The full body of vbdev_lvol_create is out of scope for this page — it's covered in 5.1. What matters here is that the lvol subsystem is asynchronous: the RPC handler returns control to the reactor long before the bdev has actually been created.

10. Completion: the response flows back

The completion callback at module/bdev/lvol/vbdev_lvol_rpc.c:311 is where the response is built:

The conn's send queue is drained on the next poll tick. The server does:

rc = send(conn->sockfd, request->send_buf + request->send_offset,
          request->send_len, 0);

The bytes go back to the Go client's decoder.

11. Go receives the result

Back in the Go side, decoder.Decode(response) at

parses the response. The Response struct is at client.go:298: Version, Error (pointer), Result (any), ID (int).

The decoder matches keys by tag name. For our response, {"jsonrpc":"2.0","id":17,"result":"bd56a4e6-..."}, it sets Result to "bd56a4e6-..." as a string, ID to 17 as int, Error to nil.

Back in the wrapper, the type assertion resp.Result.(string) succeeds, and BdevLvolCreate returns the UUID to the caller.

12. Failure modes: what can go wrong at each step

Mapping every step to the things that can break, and what the Go caller sees:

What to take away

One RPC, twelve steps, two languages, one connection. Three patterns to remember:

• The handler is the contract. The C decoder table is the actual schema. The Go struct is a Go-side mirror that you must keep in sync. If they drift, the SPDK_JSONRPC_ERROR_INTERNAL_ERROR you'll get back tells you almost nothing about which field was wrong.

• Async means "split in time." The handler returns before the work is done. The response is written from a callback. Any logging that timestamps the RPC will show a long gap between handler entry and response send.

• The response shape is the API. For bdev_lvol_create the result is a bare string (the UUID). For bdev_lvol_create_lvstore it's a bare string (the lvstore UUID). For bdev_lvol_get_lvstores it's an array of objects. The shape lives entirely in the handler's spdk_json_write_* calls.

You now know enough to read any RPC handler in SPDK. The next layer is the bdev framework itself — how those lvols become bdevs, how bdevs are stacked, and what spdk_bdev_io actually is.