100 Languages Speedrun: Episode 48: Elvish

Elvish is a recent attempt at creating a better shell.

I briefly tried it as a shell, and I instantly hated it, but that's possibly because default settings need adjusting. Anyway, complaints from the first few minutes of interactive use:

• Ctrl-A and Ctrl-E don't work
• typing name of a directory doesn't go to that directory (autocd mode)
• tab completion wouldn't work from any part of the name like cd elvish<TAB> wouldn't expand to cd episode-48-elvish, it only works with start of the name
• there's no smart autocompletion at all, like git <TAB> has no idea what git even is
• documentation for customizing the shell is nonexistent
• it was also quite unclear where the hell it even keeps the data on OSX (apparently ~/.local/state/elvish/db.bolt for history file, ~/.config/elvish/rc.elv for its RC file)
• history file is some kind of binary in nonstandard format (not even SQLite), which is really inconvenient
• it doesn't support alias etc., so I cannot easily setup RC file by just copying by .zshrc and adjusting things a bit
• how do I even customize prompt? default is bad, and there's no documentation again
• it doesn't seem to support redirection from commands like diff <(fizzbuzz.elv) <(fizzbuzz.py)

Anyway, I'm not here to evaluate it for interactive use. These issues are likely solvable with better documentation and maybe better presets or migration script.

Mainly I want to see how well it's doing as a programming language for shell scripts.

You can install it from brew install elvish and starting elvish from whichever shell you're using. This way it will inherit most environmental settings like $PATH, $EDITOR etc. from your normal shell.

Overall documentation is very poor. Many things have some definition but no examples, many things don't have any explanation at all. I needed to figure things out by experimenting, so I might have missed something in this episode due to that. Of course for its excuse, Elvish is not 1.0 yet, and documenting things is hard.

Hello, World!

#!/usr/bin/env elvish

echo "Hello, World!"

We can run it from a file:

$ ./hello.elv
Hello, World!

And of course we can run it interactively from Elvish shell:

$ echo "Hello, World!"
Hello, World!

JSON

Elvish wants to live in a world of proper data structures, but Unix commands work with either lines of text, or just one big unstructured blob of text.

To get some interoperability going, Elvish uses to-json and from-json commands. These don't convert to and from JSON document, they convert to and from JSON streams.

$ echo '{"name": "Alice", "surname": "Smith"}' | from-json | each {|x| echo "Hello, "$x[name]"!" }
Hello, Alice!

Weirdly there's no string interpolation in double quotes like in other shells - you need to unquote, put the expression, and resume quoting.

$ var catfacts = (curl -s 'https://cat-fact.herokuapp.com/facts' | from-json)
$ for fact $catfacts { echo $fact[text] }
Wikipedia has a recording of a cat meowing, because why not?
When cats grimace, they are usually "taste-scenting." They have an extra organ that, with some breathing control, allows the cats to taste-sense the air.
Cats make more than 100 different sounds whereas dogs make around 10.
Most cats are lactose intolerant, and milk can cause painful stomach cramps and diarrhea. It's best to forego the milk and just give your cat the standard: clean, cool drinking water.
Owning a cat can reduce the risk of stroke and heart attack by a third.

FizzBuzz

The FizzBuzz isn't too difficult:

#!/usr/bin/env elvish

# FizzBuzz in Elvish
fn fizzbuzz {|n|
  if (== (% $n 15) 0) {
    echo "FizzBuzz"
  } elif (== (% $n 5) 0) {
    echo "Buzz"
  } elif (== (% $n 3) 0) {
    echo "Fizz"
  } else {
    echo $n
  }
}

seq 1 100 | each {|n| fizzbuzz $n}

You probably know what FizzBuzz does by now, so let's go through the code:

• # for comments as usual
• seq 1 100 is a standard Unix builtin command, printing numbers 1 to 100 on separate lines
• you can pipe it into | each, that processes things line by line normally
• there's a Ruby-style block syntax, like {|variables| code}
• fn name {|arguments| code} is a function definition
• if/elif/else syntax looks totally reasonable
• surprisingly for a shell, math uses Lisp style prefix notation. This actually makes a lot of sense, as in shell first thing is usually command and rest are arguments anyway, so Elvish just interpreted parentheses as a subcommand

Fibonacci

Let's try to write a Fibonacci function in Elvish:

#!/usr/bin/env elvish

fn fib {|a b|
  echo $a
  fib $b (+ $a $b)
}

fib 1 1

It keeps printing forever, but that's totally fine, as it's shell, so we can just | head -n 5, right? Well...

$ ./fib.elv | head -n 5
1
1
2
3
5
Exception: reader gone
Traceback:
  fib.elv, line 4:
      echo $a
  fib.elv, line 5:
      fib $b (+ $a $b)
  fib.elv, line 5:
      fib $b (+ $a $b)
  fib.elv, line 5:
      fib $b (+ $a $b)
  fib.elv, line 5:
      fib $b (+ $a $b)
  fib.elv, line 5:
      fib $b (+ $a $b)
  fib.elv, line 5:
      fib $b (+ $a $b)
  fib.elv, line 8:
    fib 1 1
Exception: ./fib.elv exited with 2
[tty 120], line 1: ./fib.elv | head -n 5

Well, one nice thing - full stacktrace. But what about that exception? Time for a lesson on Unix.

SIGPIPE

Did you even think how the hell | head -n 5 works anyway? If you have a command that generates a lot of output, and you pipe it into head, at some point head will go "OK, I had enough" and exit. Then the command will try to send data to a program that's no longer running, and crash. So why you can do seq 1 1000000 | head -n 5 or cat /usr/share/dict/words | head -n 5 or such?

To support such cases - and pipelines are very important for Unix systems to work - operating system sends SIGPIPE signal to the program trying to sending data. By default, and that's why all programs written in C and such work so well as Unix commands, this command just tells the program to exit on the spot.

If you want your language so be friendly with Unix utilities, this is also behavior you need. There are really only three modern languages that attempt that - Perl, Ruby, and Raku. Perl (as well as various older languages like Sed, Awk, all other shells, and so on), just don't do anything about SIGPIPE, which makes them work perfectly out of the box in this situation.

$ perl -le 'print for 1..1_000_000' | head -n 5
1
2
3
4
5

Ruby by default catches SIGPIPE, and converts sending data to program that exit to an exception, so you cannot do that in Ruby:

$ ruby -e '(1..1_000_000).each{|n| puts n}' | head -n 5
1
2
3
4
5
Traceback (most recent call last):
    5: from -e:1:in `<main>'
    4: from -e:1:in `each'
    3: from -e:1:in `block in <main>'
    2: from -e:1:in `puts'
    1: from -e:1:in `puts'
-e:1:in `write': Broken pipe @ io_writev - <STDOUT> (Errno::EPIPE)

On the other hand, it is recognized as a common enough use case, so you can tell Ruby to do the traditional thing with trap("PIPE", "EXIT"):

$ ruby -e 'trap("PIPE", "EXIT"); (1..1_000_000).each{|n| puts n}' | head -n 5
1
2
3
4
5

And because very specific exception is thrown in this case Errno::EPIPE, you can also catch just that exception while letting all other exceptions happen. So what Ruby is doing is perfectly fine, the default is not very pipeline friendly but there are two very easy ways (trap("PIPE", "EXIT") or Error::EPIPE exception) to make your program pipeline friendly.

Raku also converts this to an exception, which is fine default:

$ raku -e 'say $_ for 1..1_000_000' | head -n 5
1
2
3
4
5
Failed to write bytes to filehandle: Broken pipe
  in block <unit> at -e line 1

Unhandled exception: Failed to write bytes to filehandle: Broken pipe
   at SETTING::src/core.c/Exception.pm6:568  (/usr/local/Cellar/rakudo-star/2021.04/bin/../share/perl6/runtime/CORE.c.setting.moarvm:<anon>)
 from gen/moar/stage2/NQPHLL.nqp:2117  (/usr/local/Cellar/rakudo-star/2021.04/bin/../share/nqp/lib/NQPHLL.moarvm:command_eval)
 from gen/moar/Compiler.nqp:109  (/usr/local/Cellar/rakudo-star/2021.04/bin/../share/perl6/lib/Perl6/Compiler.moarvm:command_eval)
 from gen/moar/stage2/NQPHLL.nqp:2036  (/usr/local/Cellar/rakudo-star/2021.04/bin/../share/nqp/lib/NQPHLL.moarvm:command_line)
 from gen/moar/rakudo.nqp:127  (/usr/local/Cellar/rakudo-star/2021.04/bin/../share/perl6/runtime/perl6.moarvm:MAIN)
 from gen/moar/rakudo.nqp:1  (/usr/local/Cellar/rakudo-star/2021.04/bin/../share/perl6/runtime/perl6.moarvm:<mainline>)
 from <unknown>:1  (/usr/local/Cellar/rakudo-star/2021.04/bin/../share/perl6/runtime/perl6.moarvm:<main>)
 from <unknown>:1  (/usr/local/Cellar/rakudo-star/2021.04/bin/../share/perl6/runtime/perl6.moarvm:<entry>)

Unfortunately it doesn't seem to have any workarounds for this. Trapping the SIGTRAP will make it try to write the data second time after the trap, so you get a double exception instead:

$ raku -e 'signal(SIGPIPE).tap:{exit}; say $_ for 1..1_000_000' | head -n 5
1
2
3
4
5
Unhandled exception in code scheduled on thread 4
Failed to write bytes to filehandle: Broken pipe
  in block  at -e line 1

Unhandled exception: Failed to write bytes to filehandle: Broken pipe
   at <unknown>:1  (/usr/local/Cellar/rakudo-star/2021.04/bin/../share/perl6/runtime/CORE.c.setting.moarvm:)
 from SETTING::src/core.c/Rakudo/Internals.pm6:1788  (/usr/local/Cellar/rakudo-star/2021.04/bin/../share/perl6/runtime/CORE.c.setting.moarvm:exit)
 from SETTING::src/core.c/Rakudo/Internals.pm6:1796  (/usr/local/Cellar/rakudo-star/2021.04/bin/../share/perl6/runtime/CORE.c.setting.moarvm:exit)
 from SETTING::src/core.c/Rakudo/Internals.pm6:1794  (/usr/local/Cellar/rakudo-star/2021.04/bin/../share/perl6/runtime/CORE.c.setting.moarvm:exit)
 from SETTING::src/core.c/Scheduler.pm6:29  (/usr/local/Cellar/rakudo-star/2021.04/bin/../share/perl6/runtime/CORE.c.setting.moarvm:handle_uncaught)
 from SETTING::src/core.c/ThreadPoolScheduler.pm6:261  (/usr/local/Cellar/rakudo-star/2021.04/bin/../share/perl6/runtime/CORE.c.setting.moarvm:)
Failed to write bytes to filehandle: Broken pipe
  in block <unit> at -e line 1
  in block <unit> at -e line 1

 from SETTING::src/core.c/ThreadPoolScheduler.pm6:260  (/usr/local/Cellar/rakudo-star/2021.04/bin/../share/perl6/runtime/CORE.c.setting.moarvm:)
 from SETTING::src/core.c/ThreadPoolScheduler.pm6:259  (/usr/local/Cellar/rakudo-star/2021.04/bin/../share/perl6/runtime/CORE.c.setting.moarvm:)
 from SETTING::src/core.c/Rakudo/Internals.pm6:1788  (/usr/local/Cellar/rakudo-star/2021.04/bin/../share/perl6/runtime/CORE.c.setting.moarvm:exit)
 from SETTING::src/core.c/Rakudo/Internals.pm6:1795  (/usr/local/Cellar/rakudo-star/2021.04/bin/../share/perl6/runtime/CORE.c.setting.moarvm:exit)
 from SETTING::src/core.c/Rakudo/Internals.pm6:1794  (/usr/local/Cellar/rakudo-star/2021.04/bin/../share/perl6/runtime/CORE.c.setting.moarvm:exit)
 from -e:1  (<ephemeral file>:)
 from SETTING::src/core.c/Supply-factories.pm6:153  (/usr/local/Cellar/rakudo-star/2021.04/bin/../share/perl6/runtime/CORE.c.setting.moarvm:)
 from SETTING::src/core.c/Supply-factories.pm6:117  (/usr/local/Cellar/rakudo-star/2021.04/bin/../share/perl6/runtime/CORE.c.setting.moarvm:)
 from SETTING::src/core.c/Lock/Async.pm6:204  (/usr/local/Cellar/rakudo-star/2021.04/bin/../share/perl6/runtime/CORE.c.setting.moarvm:run-under-recursion-list)
Unhandled exception: Failed to write bytes to filehandle: Broken pipe
 from SETTING::src/core.c/Lock/Async.pm6:183  (/usr/local/Cellar/rakudo-star/2021.04/bin/../share/perl6/runtime/CORE.c.setting.moarvm:run-with-updated-recursion-list)
 from SETTING::src/core.c/Lock/Async.pm6:146  (/usr/local/Cellar/rakudo-star/2021.04/bin/../share/perl6/runtime/CORE.c.setting.moarvm:protect-or-queue-on-recursion)
 from SETTING::src/core.c/Supply-factories.pm6:117  (/usr/local/Cellar/rakudo-star/2021.04/bin/../share/perl6/runtime/CORE.c.setting.moarvm:)
 from SETTING::src/core.c/signals.pm6:55  (/usr/local/Cellar/rakudo-star/2021.04/bin/../share/perl6/runtime/CORE.c.setting.moarvm:)
   at SETTING::src/core.c/Exception.pm6:568  (/usr/local/Cellar/rakudo-star/2021.04/bin/../share/perl6/runtime/CORE.c.setting.moarvm:<anon>)
 from SETTING::src/core.c/ThreadPoolScheduler.pm6:248  (/usr/local/Cellar/rakudo-star/2021.04/bin/../share/perl6/runtime/CORE.c.setting.moarvm:)
 from SETTING::src/core.c/ThreadPoolScheduler.pm6:245  (/usr/local/Cellar/rakudo-star/2021.04/bin/../share/perl6/runtime/CORE.c.setting.moarvm:)
 from SETTING::src/core.c/ThreadPoolScheduler.pm6:242  (/usr/local/Cellar/rakudo-star/2021.04/bin/../share/perl6/runtime/CORE.c.setting.moarvm:run-one)
 from gen/moar/stage2/NQPHLL.nqp:2117  (/usr/local/Cellar/rakudo-star/2021.04/bin/../share/nqp/lib/NQPHLL.moarvm:command_eval)
 from SETTING::src/core.c/ThreadPoolScheduler.pm6:297  (/usr/local/Cellar/rakudo-star/2021.04/bin/../share/perl6/runtime/CORE.c.setting.moarvm:)
 from gen/moar/Compiler.nqp:109  (/usr/local/Cellar/rakudo-star/2021.04/bin/../share/perl6/lib/Perl6/Compiler.moarvm:command_eval)
 from SETTING::src/core.c/Thread.pm6:54  (/usr/local/Cellar/rakudo-star/2021.04/bin/../share/perl6/runtime/CORE.c.setting.moarvm:THREAD-ENTRY)
 from gen/moar/stage2/NQPHLL.nqp:2036  (/usr/local/Cellar/rakudo-star/2021.04/bin/../share/nqp/lib/NQPHLL.moarvm:command_line)
 from gen/moar/rakudo.nqp:127  (/usr/local/Cellar/rakudo-star/2021.04/bin/../share/perl6/runtime/perl6.moarvm:MAIN)
 from gen/moar/rakudo.nqp:1  (/usr/local/Cellar/rakudo-star/2021.04/bin/../share/perl6/runtime/perl6.moarvm:<mainline>)
 from <unknown>:1  (/usr/local/Cellar/rakudo-star/2021.04/bin/../share/perl6/runtime/perl6.moarvm:<main>)
 from <unknown>:1  (/usr/local/Cellar/rakudo-star/2021.04/bin/../share/perl6/runtime/perl6.moarvm:<entry>)

If there's a way to do this, it's not in documentation, or anywhere else I checked. This looks like a use case Raku should address, probably doing something similar to what Ruby is doing.

Anyway, back to other languages. If your language really cares about Unix pipelines (that's why I singled out C, Awk, Sed, shells, Perl, Ruby, and Raku, as these do - and all except Raku work or just need a simple switch), instantly hard quitting in this case is fine. For everything else, if the language supports exceptions, it would much rather throw a specific exception, which both lets application deal with the error, and lets the langugae do all the cleanup tasks it might have scheduled.

Just instantly hard quitting when you receive a signal is not something most languages want to do. So in languages that support exceptions, which is most of them, they'd much rather throw an exception, and then do all the proper cleanup and exit properly. So it's totally reasonable for Python to do this:

$ echo -e 'for n in range(1,10000):\n  print(n)' | python3 | head -n 5
1
2
3
4
5
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
BrokenPipeError: [Errno 32] Broken pipe
Exception ignored in: <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>
BrokenPipeError: [Errno 32] Broken pipe

It's not even that bad, as it's specific exception BrokenPipeError, and you can deal with it. Weirdly it's not even that bad to do that in Python:

$ echo -e 'import signal\nsignal.signal(signal.SIGPIPE, signal.SIG_DFL)\nfor n in range(1,10000):\n  print(n)' | python3 | head -n 5
1
2
3
4
5

Anyway, somehow Go - which in many ways is a terribly designed language that obviously only got popular due to a certain evil Big Tech company pushing it - also catches SIGPIPE. Even thought it was written by computer equivalent of the Amish, and doesn't even have exceptions in this day and age! And Elvish is written in Go, never reverted this, so it also catches SIGPIPE, and so Elvish programs crash if you | head -n 5 them.

That was a long detour, but that's the story why Elvish is doing the wrong thing.

This is absolutely baffling in a shell, as of all the languages in existence, shell scripts simply have to support being piped to | head -n 5 and such! What's even the point of having Unix shell if you can't do something as simple. It looks like it was reported before, and closed without any fix. As far as I can tell, there's no way to even change that in options in Elvish the way you can in Ruby - there's no interface to operating system signals, and there are no specific exception types in Elvish, so if you wrap it in a try block, it will catch all exceptions.

And Raku should also provide some way to get exit-on-SIGPIPE like Ruby and Python do. It looks like it almost got there, it just needs one more keyword argument to signal function.

Exception handling

We can't make it autoquit, but we can catch the exception, so let's try just that:

#!/usr/bin/env elvish

fn fib {|a b|
  echo $a
  fib $b (+ $a $b)
}

try {
  fib 1 1
} except e {
  echo $e[reason] >&2
}

$ ./fib2.elv | head -n 400 | tail -n 10
2315700212878644019141884587055058551781936851295739770295042651311250305217930509
3746881652193013001948452031301618270087167924331813432662649918207032018665867029
6062581865071657021090336618356676821869104775627553202957692569518282323883797538
9809463517264670023038788649658295091956272699959366635620342487725314342549664567
15872045382336327044129125268014971913825377475586919838578035057243596666433462105
25681508899600997067167913917673267005781650175546286474198377544968911008983126672
41553554281937324111297039185688238919607027651133206312776412602212507675416588777
67235063181538321178464953103361505925388677826679492786974790147181418684399715449
108788617463475645289761992289049744844995705477812699099751202749393926359816304226
176023680645013966468226945392411250770384383304492191886725992896575345044216019675
<unknown reader gone>

The good things - it now works, and Elvish supports big integers by default. Bad thing - there's no way to specify what kind of exception we're catching, it's either all or nothing. If you wrap your program in a try except block, it will catch all the exceptions, not just EPIPE / BrokenPipeError / or whatever it would like to be called.

I wanted to write a few more examples, but the SIGPIPE detour took way too much time.

Should you use Elvish?

Other than completely failing at handling SIGPIPE, it's a nicer scripting language than shell. But then you know what you could do instead of looking for a nicer shell? Use a real programming language like Ruby or Python or one of so many others.

Well, people just refuse to listen to good advice to stop writing programs in shell. And given that, it makes a lot of sense for people to keep trying to create a "better shell". For that I think Elvish is doing a lot of things right, but it just isn't ready to be treated seriously. Maybe in a few years it will be a serious contender.

Code

All code examples for the series will be in this repository.

Code for the Elvish episode is available here.