Byte for your thoughts

I like Go programing language, and I like writing command line tools with it. I feel it strikes the perfect balance of convenience without magic, access to OS and portability. I also like using Cobra library for writing my CLIs. It helps me break my app into sub-commands and build consistent and feature rich command line experiences.

However, I've found that Cobra's documentation and its generation tool for scaffolding CLI projects encourage writing code with coupled responsibilities that's hard to test. In this post I'll explore an alternative approach to structuring Cobra based CLI apps. My goal is to decouple business logic from the command line interface, and to cover it with unit tests. I've put together a simple project to showcase the idea: Passgen.

Exploring the problem

Let's start by creating a new project using Cobra Generator:

$ cobra init passgen --pkg-name github.com/antolis/passgen
Your Cobra applicaton is ready at ~/passgen
$ tree passgen
passgen
├── cmd
│   └── root.go
├── LICENSE
└── main.go

There's nothing wrong with this file structure per se, but I'd prefer my main.go placed into a properly named subdirectory under cmd/. This will allow me to add other executables in the future, and it works nicely with the go tool. In fact, if you're a fan of meta references, you can check the source of Go tools themselves and see how they implements this pattern.

If we take a look inside the generated main.go file, it looks innocent enough:

// cmd/main.go
func main() {
	cmd.Execute()
}

The cmd.Execute() function call hints at trouble. And here's what the generated root.go file looks like:

// cmd/root.go
var cfgFile string // global state! 😧️

var rootCmd = &cobra.Command{
	/* command initialization */
} // more global state! 😖️

func Execute() {
	if err := rootCmd.Execute(); err != nil {
		fmt.Println(err)
		os.Exit(1) // I'd like main func to define exit code
	}
} // "static" func, uses all that global state 😓️

func init() {
	cobra.OnInitialize(initConfig)

	rootCmd.PersistentFlags().StringVar(&cfgFile, "config", "", "config file (default is $HOME/.passgen.yaml)")
	rootCmd.Flags().BoolP("toggle", "t", false, "Help message for toggle")
} // global state manipulation 😭️

func initConfig() { /* handling of the config file */ }

As you may have noticed, I don't like global state. Here are a few blogs that helped influence my opinions and explain the issue better than I could:

• A theory of modern Go, by Peter Bourgon
• Go, without package scoped variables, by Dave Cheney

In addition to objections to global state in general, there's one issue I'd like to emphasize here: readability. This code implements the registration pattern based on the init func. That makes it hard to unravel the entire CLI app's command structure by starting from the main function.

When adding new sub-commands using the Generator:

$ cobra add generate

newly generated sub-commands use the init func to register themselves with the root command. This is convenient as it allows the Generator to add new sub-commands without having to update any of the preexisting code. However it obscures the sub-command structure, as there's no single place where this strucutre is encoded in the source code.

Cobra's documentation offers an example of an alternative approach. In it all sub-commands are instantiated within programs main function. This is convenient for the documentation example that needs to fit on one page, but doesn't scale well for a larger app.

Better project structure

I structured the Passgen project a bit differently:

passgen
├── cmd
│   ├── passgen
│   │   └── main.go         # package main
│   ├── generate.go         # package cmd
│   ├── params.go           # package cmd
│   └── root.go             # package cmd
└── internal
    └── app
        ├── app.go          # package app
        ├── app_test.go     # package app_test
        └── words.go        # package app

With package dependencies:

+------+   +-----+   +-----+   +----------+
| main +-->+ cmd +-->+ app +<--+ app_test |
+------+   +-+-+-+   +-----+   +----------+
             | |
        +----+ +----+
        |           |
        v           v
    +-------+   +-------+
    | Cobra |   | Viper |
    +-------+   +-------+

This approach matches the hexagonal architecture. As such it's nothing fundamentally new. What I try to do here is apply those tried and true ideas to the particulars of the Cobra library and CLI app development in Go.

Package main

My main func is simple, but introduces a slight variation on the generated one:

// cmd/passgen/main.go
func main() {
	root := cmd.RootCmd() // creating new instance of command
	if err := root.Execute(); err != nil { // Execute is a method
		log.Fatal(err) // exit code is defined here in main
	}
}

This enables me to define the root command with:

// cmd/root.go
func RootCmd() *cobra.Command {
	cmd := &cobra.Command{ /* command initialization */ }
	cmd.AddCommand(
		generateCmd(), // sub-commands are listed explicitly
	)
	return cmd
}

I admit that listing sub-commands explicitly violates the open-closed principle because I have to update my root command whenever I add another top level sub-command. However, I find the simplicity of directly invoking methods preferable to the opaque self-registration based on the init function. This way you can follow method calls from the main func right down to the business logic.

Business logic

Speaking of the business logic, it's the interface between that logic housed in the app package and the cli package that's more interesting. Here I have two goals:

1. Keep Cobra and Viper dependencies confined to the cli package.
2. Inject other dependencies into the app package to enable easy testing.

With that in mind, here's what the app itself looks like:

// internal/app/app.go
type App struct {
	Params *Params
	Out io.Writer
	Random io.Reader
}

type Params struct {
	Min      int
	// other externally configurable parameters
}

func New() *App {
	return &App{ &Params{}, os.Stdout, rand.Reader }
}

func (a *App) Generate() error {
	// implementation that first validates a.Params
	// then it uses them and the provided a.Random
	// to generate a password that it writes to a.Out
}

The App struct encapsulates all application dependencies.

The custom Params struct defines all user-configurable input parameters. Note that the App neither knows nor cares how user provided those parameters. App does, however, handle parameter validation, making that part of the testable business logic.

App sends all its output to the provided io.Writer. This makes writing unit tests easier because they don't need to sniff out the standard output, and they can still check the output formatting. This makes output formatting another part of the business logic.

Lastly, the injected Random io.Reader is used as a source of randomness. The crypto/rand package from the standard library dictates this type. This is just one example of a dependency; in other applications you might want to inject HTTP client, database connection, etc.

With all this in place I can write classic table driven unit tests for all my business logic. It also means that I can write a CLI command like this:

// cmd/generate.go
func generateCmd() *cobra.Command {
	a := app.New() // App instance with production grade dependencies
	cmd := &cobra.Command{
		Use:     "generate",
		// documentation related fields...
		PreRunE: func(cmd *cobra.Command, args []string) error {
			// this reads parameters from command line
			// arguments, flags and config files:
			return initParams(cmd, a.Params)
		},
		RunE: func(cmd *cobra.Command, args []string) error {
			return a.Generate()
		},
	}
	cmd.Flags().IntVarP(&a.Params.Min, "min", "m", 16, "Min length")
	// the rest of the mapping from app.Props to flags
	return cmd
}

One good thing about this approach is that no matter how complex the app logic gets its Cobra command will always be more-less the same. The only code that I allow in the cmd package is used to set up the Cobra command.

Some extra benefits

In addition to easy testing, there are a few nice consequences of this design. Not all of them are immediately useful, and some of them may indeed never be used. I still find them interesting!

First of all, since I removed all the static init functions from the cli package, and since all commands are explicitly registered with their parent commands, it's now easy to create separate executables for a subset of those commands. All that's needed is to define a separate main function that executes not the root command, but one of the other higher level sub-commands.

Second, I use the same app.Props struct for all sub-commands. This helps me keep various input parameters consistent and unique on the application level. I can then read them from a config file, where a user can define defaults for any flag without specifying the individual command that the flag is used with.

Lastly, the App and it's logic is completely decoupled from the command line. That means it can easily be exposed behind other interfaces, like an HTTP server, or wrapped into a long running daemon process. This is usually not the first consideration when building CLI apps, but can become a handy benefit as project grows and evolves.

Conclusion

In conclusion, I find this project architecture allows me to test and scale my CLI apps and to get most out of libraries like Cobra and Viper. As previously mentioned, you can find the complete Passgen project on GitHub.

If all this command line talk has piqued your interest, I invite you to check my more general blog series on designing a CLI application:

1. CLI Development; Part 1 – Tower of Babel
2. CLI Development; Part 2 – Command line citizen
3. CLI Development; Part 3- Advanced use cases

And in case you are interested in seeing this kind of content from me in the future, you can:

• Subscribe to this blog's RSS feed.
• Follow @antolius@qua.name in fediverse.

Or just come chat with me @antolius on Mastodon