Use Go as a Shell Scripting Language

A very long time ago, I stumbled upon this article "Use Java for Everything". While I disagree that you should use Java for everything (or any programming language, for that matter), the author mentions that he wrote a wrapper script that lets him use Java for shell scripts (ones where you execute the Java source file directly, without the "write, compile, run" steps).

I wanted to do something similar for Go, because I had a very simple Go program I wanted to be able to throw into my .dotfiles repo and run without needing to do too many things first: a simple static HTTP server.

In Python, there's a standard module named SimpleHTTPServer, where if all you want to do is quickly share some files from your computer, you could open a terminal and run:

$ python -m SimpleHTTPServer

...and it would make your current working directory available over HTTP at http://localhost:8000/.

The problem though is that the Python SimpleHTTPServer is single-threaded so it's very limited in its use case. For example if a user is downloading a large file, all other HTTP requests are blocked until that file completes. Somebody wrote a ThreadedSimpleHTTPServer, but that requires an extra pip install to get, and Python isn't very well suited at being an HTTP server in the first place.

Anyway, I wrote my own wrapper script for Go programs that I named gosh (for "go shell"). It lets me write my own version of SimpleHTTPServer using Go, without needing to compile it first. And since Go was designed to run HTTP servers, it does the job very well and with high concurrency. Here is my SimpleHTTPServer implementation. The only difference between this and a normal Go program is the shebang header at the top of the file, which tells it to use my gosh wrapper:

///bin/true && exec /usr/bin/env gosh "$0" "$@"
// vim:set ft=go:
package main

// SimpleHTTPServer is a simple Go static file server, similar to the Python
// module of the same name, but which supports high concurrency and all the
// other niceties that you get from Go out of the box.
//
// It runs via my `gosh` wrapper for treating simple Go programs as shell
// scripts. See my `gosh` script, or just remove the shebang line at the top
// of this file to `go build` your own version.

import (
	"flag"
	"fmt"
	"log"
	"net/http"
)

// LogMiddleware logs all HTTP requests.
func LogMiddleware(next http.Handler) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		res := &ResponseWriter{w, 200}
		next.ServeHTTP(res, r)
		log.Printf("%s %d %s %s\n",
			r.RemoteAddr,
			res.Status,
			r.Method,
			r.RequestURI,
		)
	})
}

// ResponseWriter is my own wrapper around http.ResponseWriter that lets me
// capture its status code, for logging purposes.
type ResponseWriter struct {
	http.ResponseWriter
	Status int
}

// WriteHeader wraps http.WriteHeader to also capture the status code.
func (w *ResponseWriter) WriteHeader(code int) {
	w.ResponseWriter.WriteHeader(code)
	w.Status = code
}

func main() {
	// Command line flag: the port number to listen on.
	port := flag.Int("port", 8000, "The port number to listen on.")
	flag.Parse()

	fmt.Printf("Serving at http://0.0.0.0:%d/\n", *port)
	err := http.ListenAndServe(
		fmt.Sprintf(":%d", *port),
		LogMiddleware(http.FileServer(http.Dir("."))),
	)
	panic(err)
}

So with this I could put the SimpleHTTPServer script on my $PATH and just run it.

Here is the actual source of the gosh wrapper. I wrote it in Python, because I wanted to put it in my dotfiles and I don't want to check in compiled binary artifacts into my git repo. It's a very simple wrapper script that copies your Go source code into a temporary file (so it can remove the shebang header, and rename it to a .go file, to make it go runable without syntax errors), and runs it passing along any remaining command line arguments.

#!/usr/bin/env python

"""gosh: use Golang as a shell scripting language.

This is written in Python so that I don't have to commit any binaries to my
dotfiles repo, and my Go shell scripts can also remain in source form.

Usage: write a Go program with this shebang comment on top immediately before
the `package main` statement:

    #!/usr/bin/env gosh
    package main

And make it executable and run it like any shell script.
"""

import codecs
import os
import subprocess
import sys
import tempfile

def main():
    if len(sys.argv) == 1:
        die("Usage: gosh <file.go>")

    # Get the Go source file from the command line.
    source = sys.argv[1]
    argv   = sys.argv[2:]
    if not os.path.isfile(source):
        die("{}: not a file".format(source))

    # Make a temp file that lacks the shebang line of the input file.
    with codecs.open(source, "r", "utf-8") as fh:
        # Get the shebang off and sanity check it.
        shebang = fh.readline()
        if not "gosh" in shebang:
            die("{}: doesn't appear to be a Go script".format(source))

        # Write it to a temp file, sans shebang.
        temp = tempfile.NamedTemporaryFile(delete=False, suffix=".go")
        temp.write(fh.read().strip().decode())
        temp.close()

        # Call it.
        subprocess.call(["go", "run", temp.name] + argv)

        # Clean up.
        os.unlink(temp.name)

def die(message):
    print(message)
    sys.exit(1)

if __name__ == "__main__":
    main()

The source codes to both of these things are available on my dotfiles repo:

• gosh
• SimpleHTTPServer

Update (Dec 19 2017)

I looked into what other people are doing for this and have improved the shebang line syntax from the original one in this blog post.

Originally I used #!/usr/bin/env gosh as the shebang line, but that had a few problems:

• The # comment is a syntax error in Go (gosh worked around it by stripping the shebang line from the source)
• Without the .go file extension, text editors couldn't determine that Go syntax was used, and if you did set it to Go syntax, the shebang line would be highlighted as an error. This breaks the text editor and you also end up losing nice Go plugin features in Atom, for example.

The new shebang is ///bin/true && exec /usr/bin/env gosh "$0" "$@", with a trouble /// as per a comment on the StackOverflow page. It passes the script to gosh still, so that the script doesn't need a .go file extension and it continues to work just fine.

Since // is the valid comment characters in Go, the syntax highlighter works and text editor plugins can work fine. I installed the atom-vimline plugin, so I can include a Vim modeline and set the syntax to Go automatically. Another advantage of not using the *.go file extension is that I can have multiple Go scripts in the same folder, without my editor complaining about having a duplicate main() function.