I was on the Cup O’ Go podcast this last week! I talked about my SQLite package and the history behind it. Go check it out!

I’m proud to announce that my CGo-less SQLite Go package, zombiezen.com/go/sqlite, has finally reached version 1.0. This has been the culmination of almost three years of work: I started on this project in March of 2021 as a fork of David Crawshaw’s crawshaw.io/sqlite to build on top of Jan Mercl’s amazing modernc.org/sqlite. I’ve built a number of features on top of this package such as:

• A simple schema migration framework
• A basic REPL
• User-defined functions, including windows and aggregates
• Custom virtual tables
• Utilities for running embedded SQL scripts
• A go fix-like tool for migrating existing code using crawshaw.io/sqlite
• Support for running restricted SQL

Over this time, the project has been used in about a dozen open source projects and has over 350 stars on GitHub. I’ve successfully used it in a number of personal projects, including a personal accounting program and my Nix caching layer.

With the 1.0 release, I’m proud to call the API stable so that Go applications can continue to build on it for their storage needs. If you’re using zombiezen.com/go/sqlite for something interesting, let me know about it, and consider supporting me on GitHub.

Today, I released a small library called postgrestest. It spins up an ephemeral PostgreSQL database in Go. I’ve found it quite useful for writing tests that use PostgreSQL while keeping the test hermetic and reasonably fast. In my benchmarks, starting a server takes roughly 650 milliseconds and creating a database takes roughly 20 milliseconds — a 70% improvement and 90% improvement, respectively, over a postgres Docker container in the default configuration.

You can install it with:

go get zombiezen.com/go/postgrestest

I’m excited to share my hobby project from the last few months: a GraphQL server library for Go. While it doesn’t implement the whole spec and might not be ready for production, I think it’s in a useful enough state to share more broadly. I’ve tested this library out with some toy projects and the results have been promising: I’m able to quickly publish a Go struct as a GraphQL endpoint.

Check it out now by reading the docs and adding it to your project with:

go get zombiezen.com/go/graphql-server/graphql

Frequently when people discuss what is a “good” Go library, they usually use terms like “idiomatic” or “the Go way”. These terms are imprecise and make it difficult to discuss the merits of different API designs. I recently re-read Don Norman’s The Design of Everyday Things and realized that the same principles of design discussed in the book can be used to evaluate the design of APIs.

I originally gave this as a talk at the Seattle Go Meetup on 2017-02-16 (video). The following is a refined version of the talk, not just a verbatim transcript, based on my speaker notes.

This report details an experience I had while writing an RPC system in Go. While Go’s standard I/O libraries make a great many things simple, I found cancellation to be more complex than I would have liked. Parts of this situation have improved in the last couple of Go releases (as I have noted below). I hope this positive trend continues in a way that allows the Go ecosystem to easily propagate cancellation, deadlines, and request values. My intent in this report — as well as the proposal I created back in May 2017 — is to give background and feedback to inform future design decisions. Suggestions for solutions welcome!

(Thanks to Ian Lance Taylor, Damien Neil, Cassandra Salisbury, and Andrew Bonventre for reviewing this report for accuracy and clarity.)

In April, I relaunched moviegolf.com, a website I’ve operated since 2009. Since this is one of the flashier programs I’ve written and certainly one of the longest-lasting, I wanted to recount its history. My style of programming has definitely shifted in the intervening years.

I’ve posted a project I’ve been hacking on to my Bitbucket account: cardcpx. This has nothing to do with my day job, it’s just something I did to help out on a film shoot (which also had nothing to do with my day job).

From the README:

cardcpx is a specialized UI for copying files from a camera card to 1+ replicas. The replica copies happen concurrently, so if you are copying N bytes to R replicas, the time is O(N) instead of O(N * R).

The interface also has includes simple scene/take ingestion, which is stored in a CSV transaction log. Selects will be copied first, so you can do a proofing check on a fast disk while your import finishes.

cardcpx supports a flat directory structure as well as the RED camera directory structure. It assumes that your clip names do not overlap. Attempting to copy the same file name will not overwrite data.

It’s still a bit rough around the edges, but I’m happy with how Go allowed me to compose a fairly complex a tool out of easy-to-understand pieces. This was my first excursion into writing a frontend with AngularJS, but I’m also similarly pleased about the design.

Everything’s open source. Feel free to dig in and use. I’d love to know if other people find this program useful.

(This developed from a thread with some other Cal Poly students discussing Go.)

One of the frequently asked questions about Go is why are there no generics in the language. It’s a fair point from the perspective of other OOP languages, but idiomatic Go code un-asks the question.

First, it is important to understand how interfaces work in Go. The gist is that interfaces use virtual method lookups, but retain the underlying type, so a conversion back to the original type is still type-safe. An empty interface (interface{}) works as a type-safe equivalent of C’s void*: you can put any type inside of an empty-interface variable, and then you can un-box it later.

The simplest way to define a linked list is through the empty interface: interface{}.

type Node struct {
    Value interface{}
    Next *Node
}

And indeed, this is how the container/list package package works, give or take some complexity. However, my argument is that this is the wrong way to do this. It requires an explicit boxing/un-boxing hit for every access, and this is where (I think) most of the call for generics comes from. There’s a much more idiomatic way to do this. My two case studies here are the sort package and the container/heap package from the standard library.

We start with sort, which defines an interface, sort.Interface:

package sort

func Sort(data Interface)

type Interface interface {
    // Len is the number of elements in the collection.
    Len() int
    // Less returns whether the element with index i should sort
    // before the element with index j.
    Less(i, j int) bool
    // Swap swaps the elements with indexes i and j.
    Swap(i, j int)
}

This may not strike you as a generic container, but that is precisely the point. You can – with a bit of boilerplate – create a type that wraps a typed slice that implements sort.Interface. Thus, the algorithm can work on any type and only works uses indices. This is much cheaper than boxing/unboxing on access.

The second example is container/heap:

package heap

func Init(h Interface)
func Pop(h Interface) interface{}
func Push(h Interface, x interface{})
func Remove(h Interface, i int) interface{}

type Interface interface {
    sort.Interface
    Push(x interface{}) // add x as element Len()
    Pop() interface{}   // remove and return element Len() - 1.
}

Admittedly, there is a bit more boxing/un-boxing going on here, but the idea is still similar. We now have a data structure that requires very few box/unbox operations (two per push/pop). It is important to note that this is still type-safe.

You could implement a binary tree structure similarly:

package tree

func Search(root *Node, val interface{}) *Node

type Node struct {
    Index       int
    Left, Right *Node
}

type Interface interface {
    Len() int
    Less(i, j int) bool
    Push(x interface{})
    Pop() interface{}
}

The Search function would push the val interface and use it for comparisons against the values stored in the tree, then pop it once finished. Admittedly, this isn’t the prettiest interface, but it works in a pinch. It works out to be only one box/un-box round-trip.

As we’ve seen, by “flipping” around the algorithms and separating them from the data structures using interfaces, we can still reap the benefits of generic containers from other languages.

Yesterday, the Go programming language announced their second stable release, introducing language changes, better packages, and general speed-ups. I’m proud to say that I helped out in a small part by contributing a patch to the zlib package.

For those of you who haven’t heard me talk about Go, it’s a programming language that brings all of the pleasant features from scripting languages and mixes it with the simplicity of C. Its syntax and feel is unlike any other language, but after a week, it begins to feel like second nature, and it’s now one of the first languages I grab to solve a problem. I can’t say enough good things about it, and if you haven’t already, download it and try the tutorial. (And yes, it will feel weird at first. Give it a week if you’re coming in fresh.)

I’m also excited that the next weekly will include my implementation of the SPDY protocol — Google’s improvement to the HTTP protocol — as one of the standard packages! The Go development team is exceptionally friendly and open to contributions, and I must say, this is the most fun open source project I’ve worked on. If you want a fun open source project to help out with, start hanging out on the go-nuts mailing list.

Congratulations to Russ and everyone involved on a successful release!

It’s been a while since I’ve posted because of some events that required my attention, but I’m back with yet another goray rendered image! A lot has changed in the rendering infrastructure (refactoring and such), and now it supports spot lights and texture mapping! Texture mapping and Blender export have not been merged to the trunk yet, but they’re coming soon.

The other exciting news is, if you’re bold enough, you can play with goray, too! Check out the Sourceforge project!

angelsfancrc:

zombiezen:

After initially posting about Google’s programming language, and working with it for about three weeks, I think I’m ready to give it a review.

Just a quick overview of Go: type names on the right side, language-level support for concurrent processing, and objects are interface-based. If you’re interested, go check out http://golang.org/.

First of all, I applaud Google for coming up with a language that actually brings something new to the table, instead of the usual “You use this syntax to do the exact same thing you could have done more cleanly in Python!” Go has actually helped me make programs that are more multiprocessing-aware, and that’s something that’s pretty amazing.

I think my main problem with it right now is that the language is fluctuating so much that it’s hard to keep up with. I’ve met some unstable code bases before, but this breaks my code every three hours if I stay on the bleeding edge. Heaven forbid if I don’t update for three days, I have to recompile everything. It’s just ridiculous.

Yes, I know that Go is weeks old. I know that there isn’t an infrastructure for formalizing language changes yet. I further know that the poor developers are getting beaned over the head by trolls on the Google group who just want to whine about how different the language is, without even coding in it in some cases. Darn you, Internet trolls. I like Go’s potential, but I think I’m going to wait a month before trying to do any serious work in it yet.

Definitely wait. I havent been keeping up with it, but from your description I imagine they prematurely released any compiler for the language, if this is not some sort of Alpha or Beta at all.

You’re right, it is my own fault. I broke Rule #42 of Code-land: Any code that doesn’t have a version number is trouble. :)

After initially posting about Google’s programming language, and working with it for about three weeks, I think I’m ready to give it a review.

Dude. Google built a programming language.