Bug: unmarshalling CommandStep error when a : character is found in a string that's passed to buildkite-agent pipeline upload

[jamietanna]

I noticed today that, when trying to add an additional step to the pipeline to notify Slack, I received the following error:

2024-12-13 16:51:28 WARN   There were some issues with the pipeline input - pipeline upload will proceed, but might not succeed:
while unmarshaling the value for key "steps" into struct field "Steps"
  ↳ while unmarshaling step 1 of 1
      ↳ fell back using unknown type of step due to an unmarshaling error
          ↳ unmarshalling CommandStep: unmarshaling item at index 0 of 2: incompatible types: cannot unmarshal *ordered.Map[string,interface {}] into *string command=pipeline upload

This is given the following:

steps:
  - label: "Something to write here"
    command:
      - echo "Some context for the error message, blah: See https://github.com/...."
      - exit 1
    agents:
      image: busybox:latest
    notify:
      - slack:
          channels:
            - "#some-alerts-channel"

I noticed that when making the following change:

 steps:
   - label: "Something to write here"
     command:
-      - echo "Some context for the error message, blah: See https://github.com/...."
+      - echo "Some context for the error message, blah. See https://github.com/...."
       - exit 1
     agents:
       image: busybox:latest
     notify:
       - slack:
           channels:
             - "#some-alerts-channel"

Then the upload started working as expected.

It looks like there may be some additional YAML parsing/marshalling occurring, that doesn't seem to handle the : in the string.

[DrJosh9000]

Hi @jamietanna, thanks for raising this.

Unfortunately this looks like either a bug with the YAML parser library we're using (gopkg.in/yaml.v3), or a bug in the YAML spec itself, which means this could be difficult to fix properly.

Using a small Go program to exercise just the library:

package main

import (
	"os"

	"github.com/kr/pretty"
	"gopkg.in/yaml.v3"
)

func main() {
	var o any
	if err := yaml.NewDecoder(os.Stdin).Decode(&o); err != nil {
		println("Couldn't decode YAML:", err.Error())
		return
	}
	pretty.Println(o)
}

I got this output:

...snip...
            "command": []interface {}{
                map[string]interface {}{
                    "echo \"Some context for the error message, blah": "See https://github.com/....\"",
                },
                "exit 1",
            },
...snip... 

That is, gopkg.in/yaml.v3 interpreted echo "Some context for the error message, blah as a mapping key, : as the mapping separator, and the remainder as the value.

I also noticed that, with YAML highlighting, my text editor highlighted the YAML consistently with this interpretation: