k8s-objectmatcher User Guide

k8s-objectmatcher is a small, focused Go library that answers a single question for Kubernetes operator authors: "is the live object in the cluster meaningfully different from the object I want to apply?" It does this by computing a strategic-merge or JSON-merge patch between a desired object and the object the API server currently holds, then stripping the fields the server itself fills in (status, defaulted values, clusterIP, auto-injected metadata). What's left is a real, reconcilable diff — or an empty patch, which means "no update needed."

The library originated at Banzai Cloud and lives under cisco-open today. It is the diff engine inside several production operators and exposes a deliberately tiny surface: one PatchMaker.Calculate call, a handful of CalculateOption filters, and an annotation-based three-way merge so your controller can compare current state, desired state, and the last-applied state stored on the object.

This guide covers installation, the public API, the seven options most controllers use, and recipes for the patterns that come up daily in controller-runtime reconcile loops.

1. Quick Start

k8s-objectmatcher is a Go library. There is no binary, no daemon, no CLI. You add the import, instantiate a PatchMaker, and call Calculate(current, desired, opts...) from inside your controller's Reconcile method. If PatchResult.IsEmpty() returns true, skip the update; otherwise, apply the modified object and persist the new last-applied annotation.

Prerequisites: Go 1.22 or newer, a project that already imports k8s.io/apimachinery (every controller-runtime project does).

Install:

go get github.com/cisco-open/k8s-objectmatcher/patch

Minimal example — the canonical 10-line wiring:

package main

import (
	"context"
	"fmt"

	"github.com/cisco-open/k8s-objectmatcher/patch"
	appsv1 "k8s.io/api/apps/v1"
	"sigs.k8s.io/controller-runtime/pkg/client"
)

func reconcileDeployment(ctx context.Context, c client.Client, desired *appsv1.Deployment) error {
	current := &appsv1.Deployment{}
	if err := c.Get(ctx, client.ObjectKeyFromObject(desired), current); err != nil {
		return err
	}

	result, err := patch.DefaultPatchMaker.Calculate(current, desired, patch.IgnoreStatusFields())
	if err != nil {
		return err
	}
	if result.IsEmpty() {
		return nil // already in sync — skip the update
	}
	fmt.Printf("patch needed: %s\n", string(result.Patch))
	return c.Update(ctx, desired)
}

Expected behaviour:

• First reconcile of a new object: IsEmpty() returns false, you create it and call patch.DefaultAnnotator.SetLastAppliedAnnotation(desired) before the create so future diffs are three-way.
• Steady state: IsEmpty() returns true. No spurious updates, no fights with controllers that own status.

Tip

Skip per-controller wiring for the common case — patch.DefaultPatchMaker and patch.DefaultAnnotator are package-level globals that work out of the box. Build your own NewPatchMaker(...) only when you need a custom annotation key or you're injecting fakes for tests.

What to try next: Section 4 walks through the public types and every option. Section 5 has recipes for Service clusterIP, StatefulSet volumeClaimTemplates, and a wired-in controller-runtime reconcile.

2. Installation

The library is distributed as a Go module. There is nothing to compile ahead of time and no separate runtime to deploy.

2.1 Requirements

The library has zero CGO, no platform-specific code, and runs anywhere Go runs (Linux, macOS, Windows, ARM64, AMD64).

2.2 Add to a Go module

go get github.com/cisco-open/k8s-objectmatcher/patch
go mod tidy

go: added github.com/cisco-open/k8s-objectmatcher v0.x.y
go: added emperror.dev/errors v0.8.1
go: added github.com/evanphx/json-patch v5.9.0+incompatible
go: added github.com/json-iterator/go v1.1.12

Important

The importable subpackage is patch, not the module root. Always import github.com/cisco-open/k8s-objectmatcher/patch — the bare module path exposes nothing useful.

2.3 Vendoring (optional)

If your project vendors dependencies (controller-runtime projects often do), run a vendor refresh after go get:

go mod vendor

(no output on success)

2.4 Verify the install

A one-line smoke test is enough:

package main

import (
	"fmt"
	"github.com/cisco-open/k8s-objectmatcher/patch"
)

func main() {
	a := patch.NewAnnotator("example.com/last-applied")
	fmt.Println(a != nil) // true
}

go run ./...

true

Note

The default annotation key is banzaicloud.com/last-applied (exposed as patch.LastAppliedConfig). It is unchanged from the Banzai Cloud days for backward compatibility — existing operators that wrote that key still read correctly when their image is rebuilt against cisco-open.

2.5 Local development (for contributors)

The repo ships a Makefile, golangci-lint config (.golangci.yml), and a license-checker config (.licensei.toml). To run the test suite locally:

git clone https://github.com/cisco-open/k8s-objectmatcher
cd k8s-objectmatcher
go test ./patch/...

ok  	github.com/cisco-open/k8s-objectmatcher/patch	0.123s

The tests/ subdirectory is a separate module (own go.mod) that runs integration tests against k8s.io/client-go v0.21.3. Those tests need a working KUBECONFIG and a real or simulated API server — go test ./patch/... alone covers the unit tests.

3. Configuration

There is no config file, no environment variables, and no global state to tune. "Configuration" of k8s-objectmatcher is a matter of:

1. Picking the right annotation key.
2. Choosing which CalculateOptions to pass to Calculate.
3. Deciding whether to use DefaultPatchMaker / DefaultAnnotator or build your own with NewPatchMaker(...).

3.1 The annotation key

The Annotator stamps a JSON snapshot of the desired object onto the live object as an annotation, then reads it back later for the three-way merge. The key defaults to banzaicloud.com/last-applied:

const LastAppliedConfig = "banzaicloud.com/last-applied"
var DefaultAnnotator = NewAnnotator(LastAppliedConfig)

Choose your own key with NewAnnotator:

ann := patch.NewAnnotator("mycompany.io/last-applied")
maker := patch.NewPatchMaker(ann, &patch.K8sStrategicMergePatcher{}, &patch.BaseJSONMergePatcher{})

Tip

Pick a stable, unique key once per operator and never change it. Switching keys across releases makes existing live objects look like they have no last-applied snapshot, which forces a fresh three-way baseline (and may trigger one large reconcile).

3.2 Annotation storage format

The annotation value is gzipped + base64-encoded, with a fallback read path for plain (non-zipped) values from older operator versions. This matters for two reasons:

• Annotations have a 256 KiB limit per object in Kubernetes. The zip step roughly halves the JSON before encoding, so most CRDs fit comfortably; very large objects (StatefulSets with hundreds of mounts or ConfigMaps with embedded JSON) can still bump the ceiling.
• You will not see human-readable JSON if you kubectl get -o yaml the object — that's by design.

Note

The library transparently decodes both formats: pre-zip annotations (raw JSON) and modern zip+base64 annotations both round-trip through GetOriginalConfiguration. Upgrading does not require a flag day.

3.3 The CalculateOption set

Every option is a func([]byte, []byte) ([]byte, []byte, error) — it takes the current and modified JSON byte streams, mutates both symmetrically, and returns them. The shipped options are:

You can supply zero, one, or many options to a single Calculate call — they're applied left-to-right.

Important

Apply IgnoreStatusFields() to anything with a status subresource. Otherwise the API server's status updates (which your controller probably doesn't own) will show up as a non-empty patch on every reconcile and you'll spin in a tight update loop.

3.4 Using a custom PatchMaker

DefaultPatchMaker is built once at package load with the default annotator and the standard Kubernetes strategic-merge / JSON-merge implementations. To swap one out (e.g. inject a fake patcher in tests or attach PreconditionFuncs to the strategic-merge path):

maker := patch.NewPatchMaker(
	patch.NewAnnotator("mycompany.io/last-applied"),
	&patch.K8sStrategicMergePatcher{
		PreconditionFuncs: []mergepatch.PreconditionFunc{
			mergepatch.RequireKeyUnchanged("apiVersion"),
		},
	},
	&patch.BaseJSONMergePatcher{},
)

See the Admin Guide for a full breakdown of the strategic-merge / JSON- merge dispatch table and how to plug in a custom implementation.

4. Core Usage

The two-line mental model:

1. Calculate a PatchResult from (currentObject, desiredObject, opts...).
2. Branch on PatchResult.IsEmpty().

Everything else — annotation management, per-kind cleaners, the strategic-merge versus JSON-merge dispatch — happens inside Calculate.

4.1 The PatchResult shape

Calculate returns a *PatchResult:

type PatchResult struct {
	Patch    []byte // the diff, "{}" if no real change
	Current  []byte // current object as JSON, post-cleanup
	Modified []byte // desired object as JSON, post-cleanup
	Original []byte // last-applied snapshot from the annotation, may be nil
}

func (p *PatchResult) IsEmpty() bool      // true when string(Patch) == "{}"
func (p *PatchResult) String() string     // formatted dump for debugging

Treat IsEmpty() == true as the only signal that no update is needed. Inspect Patch only when you need to log what changed; never make control-flow decisions on the byte length, which can be 2 ({}) or larger even when there's no semantic change.

4.2 Calling Calculate

result, err := patch.DefaultPatchMaker.Calculate(currentObj, desiredObj,
	patch.IgnoreStatusFields(),
	patch.IgnoreField("metadata"),
)
if err != nil {
	return err
}
if !result.IsEmpty() {
	// real change — update or recreate
}

Internally, Calculate does the following in order:

1. JSON-marshals both objects.
2. Applies every option, left-to-right.
3. Calls DeleteNullInJson on both to drop nil fields and zero-valued IntOrString types (a long-standing source of false diffs).
4. Reads the last-applied annotation from currentObject.
5. Branches on the object's runtime type:
   • *unstructured.Unstructured → JSON-merge path (CreateThreeWayJSONMergePatch).
   • Everything else → strategic-merge path (CreateThreeWayMergePatch, then re-applies and re-diffs to collapse $setElementOrder directives).
6. Returns (Patch, Current, Modified, Original) for inspection.

Note

The strategic-merge path runs the patch twice — first to compute it, then to re-apply it locally and compute a two-way merge against the current state. This second pass eliminates spurious differences caused by $setElementOrder markers that strategic merge generates for arrays even when the contents are identical.

4.3 Annotation lifecycle

Three calls cover the full lifecycle for a single object:

You almost never call GetOriginalConfiguration or GetModifiedConfiguration directly — Calculate does that. The two calls you wire by hand are SetLastAppliedAnnotation (before create and before update).

4.4 The full public API

Tip

Need to mock the diff engine in unit tests? Implement the Maker interface (Calculate(current, modified, opts...) (*PatchResult, error)) in your test code and inject your fake. The library never hardcodes *PatchMaker anywhere it consumes Maker.

5. Common Tasks

The seven recipes below are the patterns that come up in nearly every controller. Each one stays inside the public API surface — no internal types, no reflection.

5.1 Diff a Deployment without spinning on status updates

The single most common pattern: reconcile a Deployment and stop fighting the deployment controller's status.observedGeneration / status.replicas writes.

import (
	"github.com/cisco-open/k8s-objectmatcher/patch"
	appsv1 "k8s.io/api/apps/v1"
	"sigs.k8s.io/controller-runtime/pkg/client"
)

func reconcileDeployment(ctx context.Context, c client.Client, desired *appsv1.Deployment) error {
	current := &appsv1.Deployment{}
	err := c.Get(ctx, client.ObjectKeyFromObject(desired), current)
	if errors.IsNotFound(err) {
		if err := patch.DefaultAnnotator.SetLastAppliedAnnotation(desired); err != nil {
			return err
		}
		return c.Create(ctx, desired)
	}
	if err != nil {
		return err
	}

	result, err := patch.DefaultPatchMaker.Calculate(current, desired, patch.IgnoreStatusFields())
	if err != nil {
		return err
	}
	if result.IsEmpty() {
		return nil
	}
	if err := patch.DefaultAnnotator.SetLastAppliedAnnotation(desired); err != nil {
		return err
	}
	return c.Update(ctx, desired)
}

Note

SetLastAppliedAnnotation runs once on create and again on every non-empty update. Skipping it leaves the live object's last-applied annotation stale, which downgrades subsequent diffs from three-way to two-way and re-introduces the spurious-update problem.

5.2 Diff a Service correctly (clusterIP, nodePort, targetPort)

Services pick up server-side defaults — the API server picks a clusterIP from the service CIDR, picks nodePorts on type NodePort/LoadBalancer, and resolves targetPort to a concrete value. None of those belong in your desired object, but they round-trip into your current object.

result, err := patch.DefaultPatchMaker.Calculate(currentSvc, desiredSvc, patch.IgnoreStatusFields())

IgnoreStatusFields() covers status.loadBalancer.ingress and friends. The server-set spec.clusterIP, spec.clusterIPs, and spec.ports[*].nodePort fields are already preserved by the strategic-merge logic the PatchMaker runs — leaving them off the desired object is enough.

Tip

If you must hardcode a clusterIP on a desired Service (for example, to pin a headless DNS service across rebuilds), set it explicitly on the desired object. The patch will not flip it back to empty because the desired side now matches the current side.

5.3 StatefulSet volumeClaimTemplates — the special case

spec.volumeClaimTemplates is one of the few places where the API server permanently rewrites your desired object — it stamps kind, apiVersion, and an initial status.phase=Pending on every PVC template. Without a filter, every reconcile sees a non-empty patch forever.

result, err := patch.DefaultPatchMaker.Calculate(currentSS, desiredSS,
	patch.IgnoreStatusFields(),
	patch.IgnoreVolumeClaimTemplateTypeMetaAndStatus(),
)

IgnoreVolumeClaimTemplateTypeMetaAndStatus walks spec.volumeClaimTemplates[*] on both sides and sets kind="", apiVersion="", and status={"phase":"Pending"}, normalising the two representations rather than deleting them.

Warning

IgnoreVolumeClaimTemplateTypeMetaAndStatus does not handle the nested selector.matchLabels / resources.requests.storage fields being mutated post-creation — Kubernetes treats those as immutable and will reject the update. If your reconciler is the one trying to change PVC capacity, you need a separate recreate-or-resize path; see issue #51 for an open report on edge cases.

5.4 Use IgnoreField to opt-out a top-level key

IgnoreField(field) deletes a top-level key from both byte streams before diffing. Useful for fields a third-party controller writes that you want to leave alone:

result, err := patch.DefaultPatchMaker.Calculate(curObj, desObj,
	patch.IgnoreStatusFields(),
	patch.IgnoreField("metadata"), // ignore everything in metadata (rare; usually too broad)
)

The argument is an exact, top-level JSON key. To ignore a nested field (say, spec.replicas), IgnoreField is not sufficient — that capability is open as #57. Use the workaround in 5.7 below.

5.5 Wire it into a controller-runtime Reconcile

Every healthy controller-runtime reconciler that uses this library has the same shape: get → calculate → branch → update + reannotate.

func (r *AppReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
	desired := r.buildDesired(req) // your construction logic

	current := &appsv1.Deployment{}
	err := r.Get(ctx, req.NamespacedName, current)

	switch {
	case errors.IsNotFound(err):
		if err := patch.DefaultAnnotator.SetLastAppliedAnnotation(desired); err != nil {
			return ctrl.Result{}, err
		}
		return ctrl.Result{}, r.Create(ctx, desired)

	case err != nil:
		return ctrl.Result{}, err
	}

	result, err := patch.DefaultPatchMaker.Calculate(current, desired,
		patch.IgnoreStatusFields(),
	)
	if err != nil {
		return ctrl.Result{}, err
	}
	if result.IsEmpty() {
		return ctrl.Result{}, nil
	}

	r.Log.V(1).Info("applying patch", "patch", string(result.Patch))
	if err := patch.DefaultAnnotator.SetLastAppliedAnnotation(desired); err != nil {
		return ctrl.Result{}, err
	}
	return ctrl.Result{}, r.Update(ctx, desired)
}

5.6 Debug spurious updates

If IsEmpty() keeps returning false and you can't see why:

1. Log result.String() — it dumps Patch, Current, Modified, and Original in one block.
2. Look at Patch. The keys present are exactly the keys disagreeing.
3. Compare against Original. If Original is empty, the live object has no last-applied snapshot — call SetLastAppliedAnnotation on create / on first reconcile.
4. If Patch contains $setElementOrder/... even for unchanged slices, you've found the rare residual the second pass didn't collapse. File an issue with the reproducer; the strategic-merge path already handles the common cases.
5. If Patch references a field your controller never sets but the API server fills in (spec.clusterIP, spec.podCIDR, defaulted enum values), use IgnoreField(...).

Tip

The fastest reproducer is result.String() printed from inside a failing reconcile. Paste that into the issue. A plain reflect.DeepEqual(current, desired) is not informative — you need the strategic-merge perspective the library uses.

5.7 Handle CRDs / unstructured objects

When currentObject is a *unstructured.Unstructured, Calculate automatically routes to the JSON-merge path. There is nothing to configure — the type assertion in the dispatch decides:

import "k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"

cur := &unstructured.Unstructured{}
cur.SetGroupVersionKind(myCRDGVK)
_ = c.Get(ctx, key, cur)

desired := &unstructured.Unstructured{Object: myDesiredMap}

result, err := patch.DefaultPatchMaker.Calculate(cur, desired)

The JSON-merge path also runs the same "apply locally, re-diff" trick to suppress no-op patches. CRDs without a registered Go type — the common case for dynamic operators — are a first-class citizen.

Note

The strategic-merge path needs a typed Go struct so it can look up patch metadata (patchStrategy / patchMergeKey struct tags). For unstructured objects there's no struct, so CreateThreeWayJSONMergePatch from k8s.io/apimachinery/pkg/util/jsonmergepatch is used instead. The trade-off: list-of-objects with merge-keys won't merge as intelligently. Test your CRD schema against real updates.

5.8 Opt out of an entire subtree (workaround for nested-field ignore)

Until nested-field ignoring lands (#57), the workaround is a custom CalculateOption — the type is just a function, so you can write your own:

import (
	json "github.com/json-iterator/go"
	"github.com/cisco-open/k8s-objectmatcher/patch"
)

// IgnoreSpecReplicas zeroes spec.replicas on both sides — useful when
// HPA owns replicas.
func IgnoreSpecReplicas() patch.CalculateOption {
	return func(current, modified []byte) ([]byte, []byte, error) {
		for i, b := range [][]byte{current, modified} {
			var m map[string]interface{}
			if err := json.Unmarshal(b, &m); err != nil {
				return nil, nil, err
			}
			if spec, ok := m["spec"].(map[string]interface{}); ok {
				delete(spec, "replicas")
			}
			out, err := json.ConfigCompatibleWithStandardLibrary.Marshal(m)
			if err != nil {
				return nil, nil, err
			}
			if i == 0 {
				current = out
			} else {
				modified = out
			}
		}
		return current, modified, nil
	}
}

You can stack any number of these alongside the shipped options.

6. Troubleshooting

The diagnostic decision tree:

Reconcile loops forever?
├── Calculate returns no error, IsEmpty() == false on every pass
│     → 6.1   spurious diff from server-set fields
│     → 6.2   stale or missing last-applied annotation
│     → 6.3   $setElementOrder noise
├── Calculate returns an error
│     → 6.4   "Failed to lookup patch meta" (unsupported type)
│     → 6.5   "could not unmarshal" (object isn't valid JSON)
│     → 6.6   "Failed to convert current object to byte sequence"
└── Annotation never appears on the live object
      → 6.7   forgot to call SetLastAppliedAnnotation before create/update

6.1 Error: every reconcile reports a non-empty patch on status

Cause: You're not passing IgnoreStatusFields(). The API server writes status for almost every workload type, and your reconciler sees that as a difference between desired (status: {}) and current (status: {observedGeneration: 7, ...}).

Solution:

result, err := patch.DefaultPatchMaker.Calculate(cur, des, patch.IgnoreStatusFields())

Prevention: Add IgnoreStatusFields() to every Calculate call by default. There's no downside on objects that don't have a status field — the option just deletes a key that's already absent.

6.2 Error: result.Original is nil and patches are huge

Cause: The live object has no last-applied annotation, so the three-way merge degrades into a two-way merge between Current and Modified. Every server-defaulted field shows up.

Solution:

1. Call patch.DefaultAnnotator.SetLastAppliedAnnotation(desired) immediately before c.Create(ctx, desired).
2. Call it again immediately before every c.Update(ctx, desired).
3. For existing objects in the cluster predating your operator, one-time backfill: read the live object, call SetLastAppliedAnnotation(live) against it as the "desired", and Update. The next reconcile starts clean.

Prevention: Move the SetLastAppliedAnnotation call into a small helper that wraps Create/Update so you can never forget it.

6.3 Error: patch contains $setElementOrder/containers for an unchanged Pod spec

Cause: Strategic merge generates $setElementOrder directives for mergeable list fields. The library's second pass (apply locally, re- diff with two-way merge) catches the common cases, but a few residuals slip through — see issue #9.

Solution: Confirm that the name keys on every list element match exactly between current and desired. Strategic merge uses those as merge keys; even a single rename re-orders the list and produces a real $setElementOrder directive — not a bug.

Prevention: Build desired Pod specs from a deterministic source (template + Helm-like rendering, not random map iteration) so the list order is stable.

6.4 Error: Failed to lookup patch meta

Cause: K8sStrategicMergePatcher.CreateThreeWayMergePatch calls strategicpatch.NewPatchMetaFromStruct(dataStruct). If dataStruct is a type with no strategic-merge tags (a CRD struct that doesn't register patchStrategy="merge" annotations), this errors.

Solution: Pass the object as *unstructured.Unstructured instead — Calculate will route it through the JSON-merge path, which doesn't need patch metadata.

u, err := runtime.DefaultUnstructuredConverter.ToUnstructured(myCRDObj)
// ...wrap and use as *unstructured.Unstructured

Prevention: For CRDs without proper struct tags, standardize on unstructured access in the reconciler.

6.5 Error: could not unmarshal byte sequence from a CalculateOption

Cause: Your custom option (or one of the shipped ones) was handed bytes that aren't valid JSON. Almost always: a previous option in the chain returned malformed bytes.

Solution: Inspect the option order. Each option must return valid JSON for both current and modified. If you're writing your own option, never return raw []byte("") on the success path — always round-trip through marshaling.

Prevention: Unit-test custom options with the unstructured test helpers from patch/patch_test.go (mustFromUnstructured / mustToUnstructured) as a template.

6.6 Error: Failed to convert current object to byte sequence

Cause: The runtime.Object you passed wraps a value that json-iterator refuses to marshal — typically a pointer cycle or a Go interface with no encoder. With Kubernetes types, this almost never happens; with custom embedded structs in your own CRD types, it can.

Solution: Round-trip the object through runtime.DefaultUnstructuredConverter.ToUnstructured and back. If that fails too, the bug is in your CRD type definition (an unexported field on the JSON path, a custom MarshalJSON that panics, etc.).

Prevention: Run json.Marshal(yourObj) once at startup as a smoke check.

6.7 Error: live object has no annotation after Create

Cause: SetLastAppliedAnnotation was called on the desired object after Create. The mutation lives in memory but never reaches the API server.

Solution: Move the SetLastAppliedAnnotation call before c.Create(ctx, desired). The mutation is on the in-memory struct and will be persisted as part of the create payload.

Prevention: Treat the annotator as a "stamp before send" operation, not a "stamp after send" one.

Warning

Calling SetLastAppliedAnnotation on the current (server-side) object inside a reconcile, then calling Update, will work — but only if you also reapply your other desired-state mutations to the live object first. Most controllers find it cleaner to mutate the desired object and Update that, never the live object.

7. FAQ

Q: Is this a CLI tool?

No — it's a Go library. There is no binary, no kubectl plugin, no container image. You import github.com/cisco-open/k8s-objectmatcher/patch into a Go program (typically a Kubernetes operator) and call its API.

Q: Does it work with controller-runtime / Operator SDK / Kubebuilder?

Yes — those all build on k8s.io/client-go and k8s.io/apimachinery, and the only types this library cares about are runtime.Object (which all of those use) and *unstructured.Unstructured. Drop it into any reconciler.

Q: What's the difference between IsEmpty() and string(Patch) == "{}"?

IsEmpty() is exactly that comparison. Use the method — it's clearer and survives a future internal change to representation.

Q: Why isn't there a kubectl apply --dry-run style mode?

Because that's the wrong abstraction for an operator. kubectl apply operates on a single object at a time and prints diffs to a human; an operator needs a programmatic answer ("should I update or not?") and a machine-readable patch. Calculate returns both — the answer (IsEmpty()) and the byte-level patch.

Q: Does it handle server-side apply?

Not directly. Server-side apply uses managed-fields tracking (metadata.managedFields), which is a different model from the client-side last-applied annotation this library implements. If your controller uses server-side apply, the API server itself does the "is this a real change" calculation — you don't need this library. Many older operators use the client-side model and that's where this library is the right fit.

Q: Can I use this without controller-runtime?

Yes. The library has zero dependency on controller-runtime — it only needs k8s.io/apimachinery types. Plain client-go informers and custom reconcile loops work fine.

Q: What about Pods?

Pods work fine, but most operators don't reconcile Pods directly — they reconcile workload controllers (Deployment, StatefulSet, Job) and let those manage their Pods. If you do diff a Pod, expect a lot of server-defaulted fields (securityContext, terminationGracePeriodSeconds, dnsPolicy, the auto-mounted ServiceAccount token volume) so use IgnoreField("status") and consider a custom option for the auto-injected volume.

Q: How do I ignore a label on Pods specifically?

This is open as #65. For now, write a custom CalculateOption that walks spec.template.metadata.labels and deletes the offending key on both sides. The recipe in 5.8 is the template.

Q: What's the deal with the banzaicloud.com/last-applied annotation key?

The library was originally built at Banzai Cloud (a Hungarian Kubernetes shop) and Cisco kept the key when they took over the repo. Customers depend on the exact value for backward compatibility — operators upgrading from Banzai Cloud builds to cisco-open builds need their existing live objects to round-trip correctly.

Q: Does it handle StatefulSet volumeClaimTemplates resize?

No — the library answers "is this a real change?", but StatefulSet PVC capacity is immutable in Kubernetes itself. Even with a real change, the API server will reject the update. Use a separate recreate-with-data-migration path. See issue #51 for related discussion.

Q: Where do I report a missing per-kind cleaner?

github.com/cisco-open/k8s-objectmatcher/issues. Issue #40 is the canonical "exclude specific fields from a StatefulSet" thread.

Q: Is the library safe to call from many goroutines?

Yes — DefaultPatchMaker and DefaultAnnotator are stateless beyond their immutable fields, and Calculate is a pure function over its inputs. The only shared state is the underlying meta.MetadataAccessor, which is itself goroutine-safe.

Q: How big can the last-applied annotation get?

Kubernetes caps annotations at 256 KiB per object. The library gzip-compresses the JSON before base64-encoding, so practical limits are roughly 1–2 MiB of raw JSON. ConfigMaps with embedded large payloads can hit the ceiling — for those, consider IgnoreField on the data key and reconcile-by-hash instead.