Add --format json support for verifycommit and audit commands

[ralphbean]

Summary

This PR adds machine-readable JSON output to the verifycommit and audit commands, making it significantly easier to extract data programmatically (e.g., with jq) instead of parsing text output with complex bash scripts.

This addresses the parsing challenges seen in konflux-ci/build-definitions#2867, where bash parsing was needed to extract SLSA levels from VSA output.

Changes

• New output framework (internal/cmd/output.go): Common OutputFormat enum and utilities for consistent JSON/text output
• verifycommit command: Added --format flag with JSON output support
• audit command: Added --format flag with JSON output support
• Comprehensive test coverage: 16 test cases across both commands

JSON Output Structure

verifycommit

{
  "success": true,
  "commit": "abc123",
  "ref": "main",
  "ref_type": "branch",
  "owner": "org",
  "repository": "repo",
  "verified_levels": ["SLSA_SOURCE_LEVEL_3"]
}

audit

{
  "owner": "org",
  "repository": "repo",
  "branch": "main",
  "latest_commit": "abc123",
  "commit_results": [
    {
      "commit": "abc123",
      "status": "passed",
      "verified_levels": ["SLSA_SOURCE_LEVEL_3"],
      "link": "https://github.com/org/repo/commit/abc123"
    }
  ],
  "summary": {
    "total_commits": 1,
    "passed_commits": 1,
    "failed_commits": 0
  }
}

Example Usage

# Extract SLSA level with jq
sourcetool verifycommit --format json --owner org --repo repo --branch main | jq -r '.verified_levels[0]'

# Get audit summary statistics
sourcetool audit --format json --owner org --repo repo --branch main | jq '.summary'

Test plan

• All existing tests pass
• New tests added for JSON output (16 test cases)
• Backward compatibility verified (text output is default)
• Binary builds successfully
• Help text displays new --format flag

🤖 Generated with Claude Code

[TomHennen]

Thanks for this change I see the value in it. I have a couple of suggestions that might make this easier to maintain long term.

[TomHennen]

I'm not entirely comfortable with this arrangement. If I'm reading it properly we've duplicated the audit logic with the textual output, that could lead them to diverge in the future.

Suggestion:

Instead of duplicating the logic, merge them by adding a 'processResult' function that either

1. calls printResult
2. calls convertAuditResultToJson & appends it to jsonResult.CommitResults.

Then, at the end of the process you can auditArgs.writeJSON(jsonResult) or similar as needed.

Some of the enhancements you made (like the audit summary) might be interesting for the text based output too.

[ralphbean]

Updated in 880805e.

[TomHennen]

This is currently comparing JSON output as strings, which isn't always stable.

It might be preferable to use something like DeepEqual which would ignore field order and formatting quirks.

Gemini provided this example

package main

import (
	"encoding/json"
	"reflect"
	"testing"
)

func TestJSONEquality(t *testing.T) {
	json1 := []byte(`{"name": "Alice", "age": 30, "hobbies": ["reading", "coding"]}`)
	json2 := []byte(`{"age": 30, "name": "Alice", "hobbies": ["coding", "reading"]}`) // Different order

	var data1, data2 interface{}

	err := json.Unmarshal(json1, &data1)
	if err != nil {
		t.Fatalf("failed to unmarshal json1: %v", err)
	}
	err = json.Unmarshal(json2, &data2)
	if err != nil {
		t.Fatalf("failed to unmarshal json2: %v", err)
	}

	if !reflect.DeepEqual(data1, data2) {
		t.Errorf("JSONs are not semantically equal.\nExpected: %v\nGot: %v", data1, data2)
	}
}

Stack Overflow has this one https://stackoverflow.com/a/32409106

[ralphbean]

Updated in b2217e6

[TomHennen]

This seems fine, but it also seems odd that opts both know if the output mode is JSON and then the user has to call writeText or writeJSON.

Could it work to have a generic writeResult method that, if the mode was 'JSON' just did what you're looking for, and if not not had specialized output based based on the type of the result (e.g. VerifyCommitResult)?

Perhaps that's too cute? Maybe @puerco has thoughts.

[TomHennen]

Oh, I just remembered, instead of doing all the specializing within opts it would be better to define a custom method on structs like VerifyCommitResult. If you name the function String it will work automatically with fmt. https://stackoverflow.com/a/13247979.

You can do a similar thing with MarshalJSON that will let you call functions as needed instead of having to define your own ToJSON conversion methods that wind up within the core logic of the code. This might save you some of the effort of defining those duplicate structs and provide a very easy way in opts to write the output as either JSON or Text as needed.

Here's the gemini example

package main

import (
	"encoding/json"
	"fmt"
	"log"
)

// User represents a user with first and last names.
type User struct {
	FirstName string
	LastName  string
}

// MarshalJSON provides a custom implementation for the json.Marshaler interface.
func (u User) MarshalJSON() ([]byte, error) {
	// Create a temporary struct to hold the data we want to marshal.
	// This avoids infinite recursion by not calling MarshalJSON on the same type.
	type alias User // Create a local alias to the type
	return json.Marshal(&struct {
		FullName string `json:"full_name"`
		*alias
	}{
		FullName: fmt.Sprintf("%s %s", u.FirstName, u.LastName),
		alias:    (*alias)(&u),
	})
}

func main() {
	user := User{
		FirstName: "Jane",
		LastName:  "Doe",
	}

	jsonData, err := json.Marshal(user)
	if err != nil {
		log.Fatalf("Error marshaling JSON: %v", err)
	}

	fmt.Printf("Custom JSON: %s\n", string(jsonData))
}

[ralphbean]

Updated in 019882f.

[puerco]

A few comments about following the command line architecture we are trying to keep across subcommands:

1. I would prefer the format option to be a simple string instead of fumbling to/from integers and strings.
2. Switching the writer to be returned from a method instead of keeping it as a property in the options set
3. Follow the pattern of other subcommands/option sets by adding AddFlags() and Validate() methods to outputOptions

(more details in the comments below 👇 )

[puerco]

OK, since we are introducing a new output options set , move the flag initialization to an AddFlags() method and call the other embedded lines above:

source-tool/internal/cmd/audit.go

Lines 73 to 74 in af7518e

	ao.branchOptions.AddFlags(cmd)
	ao.verifierOptions.AddFlags(cmd)

Also, simplify this call to a single one using StringVar() (see comment on the output options about turning this into a simple string):

Suggested change

	ao.format = OutputFormatText
	cmd.PersistentFlags().Var(&ao.format, "format", "Output format: 'text' (default) or 'json'")
	ao.format = OutputFormatText
	cmd.PersistentFlags().Var(&ao.format, "format", "Output format: 'text' (default) or 'json'")

[ralphbean]

Done in 826f7aa and 503729f.

[puerco]

I'm curious, why do we need to define a custom type for the flag value? Most of the logic in this file seems to handle converting to/from int <> string. Can't we just use a string `['json' | 'text' ] ? The we can drop most of the methods.

[ralphbean]

Dropped in 503729f.

[puerco]

Since this options set gets embedded in other options, this method should probably have a more descriptive name, for example:

Suggested change

	func (oo *outputOptions) isJSON() bool {
	func (oo *outputOptions) outputFormatIsJSON() bool {

[ralphbean]

Renamed in 403b1fa.

[puerco]

I would prefer the options set not to store the writer, since the usage pattern is always open -> write -> close, then we should probably switch the init() function to a getWriter() that returns os.Stdout by default. We can expand that to support outputting to a file later on.

[ralphbean]

Yeah, that makes sense. Changed in c5c5c5b.

[puerco]

Since this set will be embedded in other options set, add a Validate() method and call it from the options set embedding it, for example in audit:

source-tool/internal/cmd/audit.go

Lines 64 to 69 in af7518e

	func (ao *auditOpts) Validate() error {
	errs := []error{
	ao.branchOptions.Validate(),
	ao.verifierOptions.Validate(),
	}
	return errors.Join(errs...)

We should reject from the get go a value that is not json or text.

[ralphbean]

Ack - done in 826f7aa and 9300370.