Electron Adventures: Episode 58: Notebook Ruby Engine

Communicating with external program over HTTP is just one approach. Let's try to run it directly instead.

To talk to it we'll be sending JSON messages, one JSON per line. It will reply with JSON messages, one JSON per line as well.

This "JSON streaming" approach is a fairly popular solution for communication between programs, mostly because how easy it is to code.

src/App.js

All our code will live in the preload, so all we need to change in the frontend is how we run code:

  let runCode = async (code) => {
    let {error, output} = await window.api.runCode(sessionId, code)
    if (error) {
      return output + "\n" + error
    } else {
      return output
    }
  }

We can also remove axios and proxy settings from package.json.

Node IO

We already figured out how to create a new process, so now it's just await stdin.writeLine(...) and await stdout.readLine(), right?

Well, wrong! Node somehow never added any functionality for something as sophisticated like "reading a line". It's the first programming langugae I've ever see that just doesn't. Even C had gets (which didn't check for memory overflows, but that's just C being C).

This shouldn't really be that surprising as JavaScript is a browser-first language, and it lacks a lot of the same functionality that we take for granted coding in Unix-first language like Ruby or Python.

Fortunately [promise-readline](https://www.npmjs.com/package/promise-readline) can rescue us here, so let's install that. Not to be confused with readline-promise and a lot of other similarly named packages which don't do anything like that. Searching that was pretty difficult.

src/preload.js

We have just one choice to make:

• create new process for each sessions
• create one process, and send it session id with every request

This time we'll implement the "one process, multiple sessions" solution, but separating them would lead to better isolation and wouldn't be that much more complex.

Other than lineReader(process.stdout) and await stdin.write from promise-readline, everything else should be already familiar.

let child_process = require("child_process")
let lineReader = require("promise-readline")
let { contextBridge } = require("electron")

let languageServer = null

async function startLanguageServer() {
  let process = child_process.spawn(
    "./ruby_language_server",
    [],
    {
      stdio: ["pipe", "pipe", "inherit"],
    },
  )
  languageServer = {
    process,
    stdin: process.stdin,
    stdout: lineReader(process.stdout),
  }
}

async function runCode(session_id, code) {
  if (!languageServer) {
    await startLanguageServer()
  }
  let { stdin, stdout } = languageServer
  await stdin.write(JSON.stringify({ code, session_id }) + "\n")
  let line = await stdout.readLine()
  return JSON.parse(line)
}

contextBridge.exposeInMainWorld(
  "api", { runCode }
)

ruby_language_server

And now we can simply take the Sinatra server we had before, and replace Sinatra logic by a simple STDIN.each_line loop. In Ruby there's zero trouble doing line-oriented I/O.

#!/usr/bin/env ruby

require "stringio"
require "json"

def capture_output(new_stdout, new_stderr)
  begin
    $stdout, save_stdout = new_stdout, $stdout
    $stderr, save_stderr = new_stderr, $stderr
    yield
  ensure
    $stdout = save_stdout
    $stderr = save_stderr
  end
end

def eval_and_capture_output(code, context)
  response = {}
  output = StringIO.new
  capture_output(output, output) do
    begin
      eval(code, context)
    rescue Exception => e
      response["error"] = "#{e.class}: #{e.message}"
    end
  end
  response["output"] = output.string
  response
end

def new_binding
  Object.new.instance_eval{ binding }
end

bindings = Hash.new{|h,k| h[k] = new_binding}

STDOUT.sync = true

STDIN.each_line do |line|
  data = JSON.parse(line)
  code = data["code"]
  session_id = data["session_id"]
  response = eval_and_capture_output(code, bindings[session_id])
  puts response.to_json
end

What did we gain?

The biggest gain compared to the HTTP solution is that we also no longer have extremely insecure server literally executing any code we send to it.

We also have full control over starting and shutting down the language server, and it no longer needs to be started and stopped separately. We could even isolate different contexts in different processes, and we'll take advantage of that in the future.

Result

Here's the result if we press "Run All" button:

In the next episode we'll try to do the same thing for Python as we did for Ruby.

As usual, all the code for the episode is here.