07 Jul, 2022

Go plugins

Exploring Go's plugin package.

cobra is a library for creating powerful modern CLI applications, and is currently used by popular projects like Kubernetes, Hugo, etc.

This post focuses on adding plugins to a Cobra application, as an example for introducing Go’s plugin package.

Go and Plugins

Plugins are useful for extending an application’s feature list. Go generally supports two kinds of plugins:

Compile-time plugins

Compile-time plugins consist of code packages that get compiled into the application’s main binary. Compile-time plugins are easy to discover and register as they’re baked in the binary itself.

Compile-time plugins
Compile-time plugins

Run-time plugins

Run-time plugins hook up to an application at run-time.

A special build mode enables compiling packages into shared object (.so) libraries. The plugin package provides simple functions for loading shared libraries and getting symbols from them.

Run-time plugins
Run-time plugins

An example

Let’s try using run-time plugins for our Cobra application. As a first step, we will need to build a CLI using Cobra:

Code can be found here.

// main.go

package main

import (
	"errors"
	"fmt"
	"log"
	"plugin"

	"github.com/spf13/cobra"
)

func main() {
	mainCmd := &cobra.Command{
		Use:   "main",
		Short: "Simple cobra app",
		Run: func(cmd *cobra.Command, args []string) {
			fmt.Println("Main app called!")
		},
	}

	// Load the plugin.
	pluginCmd, err := LoadPlugin("./plugin/plugin.so", "PluginCmd")
	if err != nil {
		log.Fatalf("failed to load plugin: %v\n", err)
	}

	// Register the plugin at runtime.
	mainCmd.AddCommand(pluginCmd)

	if err := mainCmd.Execute(); err != nil {
		log.Fatalf("error: %v\n", err)
	}
}

The above code snippet does the following:

Here is the code for loading a plugin:

// main.go

// LoadPlugin loads a cobra command from a shared object file.
func LoadPlugin(pluginPath, cmdName string) (*cobra.Command, error) {
	p, err := plugin.Open(pluginPath)
	if err != nil {
		return nil, err
	}

	c, err := p.Lookup("New" + cmdName)
	if err != nil {
		return nil, err
	}

	// Get access to the plugin function and get the command.
	pluginFunc, ok := c.(func() *cobra.Command)
    if !ok {
        return nil, errors.New("failed to perform a lookup.")
    }

	pluginCmd := pluginFunc()

	return pluginCmd, nil
}

In the above code snippet, we use Open() for loading the plugin object. We assume that the function exposing the command has this naming convention: New<CommandName>, for example, NewPluginCmd, etc.

Since there is no way for looking up all symbols (exported functions), having a naming convention helps, as it ensures that the lookups are easy to maintain.

Since we have the command name, we perform a lookup using our naming convention. Once we have the symbol, we assert the type of the symbol. In our case, we expect the signature of func() *cobra.Command. This function is used for retrieving the cobra command, which is eventually returned to the caller.

Let’s implement the plugin:

// plugin/plugin.go

package main

import (
	"fmt"

	"github.com/spf13/cobra"
)

func NewPluginCmd() *cobra.Command {
	pluginCmd := &cobra.Command{
		Use:   "plugin",
		Short: "Plugin command",
		Run: func(cmd *cobra.Command, args []string) {
			fmt.Println("Plugin called!")
		},
	}

	return pluginCmd
}

We can now build the plugin using -buildmode=plugin. The plugin buildmode expects you to have at least one main package.

go build -buildmode=plugin .

Tada! We can see a plugin.so in our directory:

$ tree
.
├── go.mod
├── go.sum
├── main.go
└── plugin
    ├── plugin.go
    └── plugin.so

1 directory, 5 files

Here is a visualization on how the run-time plugin is registered into our Cobra app:

Visualization of loading plugins into the main app
Registering a plugin

Let’s run the main app:

$ go run main.go --help
Simple cobra app

Usage:
  main [flags]
  main [command]

Available Commands:
  completion  Generate the autocompletion script for the specified shell
  help        Help about any command
  plugin      Plugin command

Flags:
  -h, --help   help for main

Use "main [command] --help" for more information about a command.

Nice! The plugin got registered to the main Cobra app at runtime!

Let’s run the plugin:

$ go run main.go plugin
Plugin called!

We now have a plugin loaded at run-time, and is accessible through the main CLI.

Challenges

Run-time plugins are a nice addition; however, they have some challenges:

Tips

Here are some tips while building and using plugins:

Distributing plugins

Ok, we have plugins. But how do we distribute them?

The easiest way to distribute plugins would be to use a plugin registry/index. The main app can request for a plugin from the registry and load the plugin at runtime.

Here is a neat visualization on how the distribution flow works:

Distributing plugins
Distributing plugins

The main app asks for plugin2 from the “Plugin Handler” component. The Plugin Handler component has two components:

The plugin fetcher downloads plugin2 from the registry, which is eventually returned by the plugin loader. The main app now loads plugin2 at runtime.

Conclusion

Plugins allow an application to extend its functionality. Plugin support was requested by the Cobra community a couple of times (#691, #1026, #1361), but at the time of posting this blog, plugins are not natively supported by Cobra.

Plugins can be a great way to reduce your binary size. Your CLI can choose to have a subset of your available commands, and the commands with lesser priority can be set up at runtime. This however comes at a cost of maintenance, but allows builds to be faster due to less bloat in your CLI.

I would also recommend taking a look at RPC-based plugin systems, for example, hashicorp/go-plugin and natefinch/pie.

Further reading