Remove legacy engine.steps field and add migration codemod#17106
Remove legacy engine.steps field and add migration codemod#17106
Conversation
…vel steps Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
|
@copilot add more codemod testing |
There was a problem hiding this comment.
Pull request overview
Removes the legacy engine.steps field from workflow parsing/validation/runtime detection and introduces a CLI codemod to migrate existing workflows to the supported top-level steps field.
Changes:
- Deleted
engine.stepsfromEngineConfig, schema validation, and runtime package/runtime detection paths. - Removed the shared
InjectCustomEngineSteps()helper and updated engines/tests accordingly. - Added codemod
engine-steps-to-top-level(introduced in0.11.0) with unit tests and registered it in the codemod list.
Reviewed changes
Copilot reviewed 21 out of 21 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| pkg/workflow/validation_test.go | Removes test coverage tied to legacy engine.steps package collection behavior. |
| pkg/workflow/threat_detection.go | Drops propagation of EngineConfig.Steps into threat-detection engine config. |
| pkg/workflow/runtime_validation.go | Removes package extraction from engine.steps (now only top-level steps + MCP). |
| pkg/workflow/runtime_setup_integration_test.go | Updates integration test frontmatter to use top-level steps. |
| pkg/workflow/runtime_detection.go | Removes runtime detection scanning of engine.steps (top-level steps already covered via CustomSteps). |
| pkg/workflow/pip_test.go | Updates comments to reflect package extraction source changes. |
| pkg/workflow/gemini_engine.go | Removes engine-step injection path (via InjectCustomEngineSteps) from Gemini execution steps. |
| pkg/workflow/engine_includes_test.go | Updates include-related tests to use top-level steps instead of engine.steps. |
| pkg/workflow/engine_helpers_shared_test.go | Deletes tests for the removed InjectCustomEngineSteps helper. |
| pkg/workflow/engine_helpers.go | Removes the InjectCustomEngineSteps helper implementation. |
| pkg/workflow/engine_config_test.go | Removes assertions and import usage related to EngineConfig.Steps. |
| pkg/workflow/engine.go | Removes Steps from EngineConfig and stops parsing engine.steps. |
| pkg/workflow/copilot_engine_execution.go | Removes engine-step injection path from Copilot execution steps. |
| pkg/workflow/collect_packages_test.go | Removes engine-steps package extraction tests; adjusts combined-source tests accordingly. |
| pkg/workflow/codex_engine.go | Removes engine-step injection path from Codex execution steps. |
| pkg/workflow/claude_engine.go | Removes engine-step injection path from Claude execution steps. |
| pkg/parser/schemas/main_workflow_schema.json | Removes steps property from engine_config schema. |
| pkg/cli/fix_codemods_test.go | Updates codemod count and expected ordering to include the new codemod. |
| pkg/cli/fix_codemods.go | Registers engine-steps-to-top-level in GetAllCodemods(). |
| pkg/cli/codemod_engine_steps_test.go | Adds unit tests for the new migration codemod. |
| pkg/cli/codemod_engine_steps.go | Implements the codemod that moves engine.steps to top-level steps and merges when needed. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| } | ||
| withoutEngineSteps = append(withoutEngineSteps, line) | ||
| } | ||
|
|
There was a problem hiding this comment.
The codemod removes the engine.steps lines but leaves the engine: key intact. If a workflow has an engine: block that only contained steps (or only comments besides steps), the result will include a dangling engine: with a null value, which is likely invalid per the workflow schema. Consider detecting when the engine block becomes empty after removal and removing the now-empty engine: key (and any adjacent blank lines) as part of the migration.
| // Pass 1b: if the engine block is now empty (only comments/blank lines), remove it | |
| cleanedWithoutEngine := make([]string, 0, len(withoutEngineSteps)) | |
| for i := 0; i < len(withoutEngineSteps); { | |
| line := withoutEngineSteps[i] | |
| trimmed := strings.TrimSpace(line) | |
| // Look for a top-level "engine:" key | |
| if isTopLevelKey(line) && strings.HasPrefix(trimmed, "engine:") { | |
| engineIndent := getIndentation(line) | |
| startIdx := i | |
| endIdx := i | |
| j := i + 1 | |
| onlyCommentsOrBlank := true | |
| // Walk the lines belonging to the engine block | |
| for ; j < len(withoutEngineSteps); j++ { | |
| l := withoutEngineSteps[j] | |
| t := strings.TrimSpace(l) | |
| // Stop at the next top-level key (same or smaller indentation) | |
| if isTopLevelKey(l) && len(getIndentation(l)) <= len(engineIndent) && t != "" { | |
| break | |
| } | |
| // Track the furthest extent of the engine block | |
| endIdx = j | |
| // Consider only blank lines or pure comments as "empty" | |
| if t == "" || strings.HasPrefix(t, "#") { | |
| continue | |
| } | |
| // Found non-comment, non-blank content under engine | |
| onlyCommentsOrBlank = false | |
| break | |
| } | |
| if onlyCommentsOrBlank { | |
| // Remove engine block and any adjacent blank lines around it | |
| removeStart := startIdx | |
| for removeStart-1 >= 0 && strings.TrimSpace(withoutEngineSteps[removeStart-1]) == "" { | |
| removeStart-- | |
| } | |
| removeEnd := endIdx | |
| for k := endIdx + 1; k < len(withoutEngineSteps) && strings.TrimSpace(withoutEngineSteps[k]) == ""; k++ { | |
| removeEnd = k | |
| } | |
| engineStepsCodemodLog.Print("Removed empty 'engine' block after migrating 'engine.steps'") | |
| // Skip over the removed region | |
| i = removeEnd + 1 | |
| // Ensure we don't drop lines already added before removeStart | |
| continue | |
| } | |
| } | |
| // Keep the current line | |
| cleanedWithoutEngine = append(cleanedWithoutEngine, line) | |
| i++ | |
| } | |
| withoutEngineSteps = cleanedWithoutEngine |
| // Find existing top-level steps block (if any) | ||
| topLevelStepsEndIdx := -1 | ||
| hasTopLevelSteps := false | ||
| if _, exists := frontmatter["steps"]; exists { | ||
| hasTopLevelSteps = true | ||
| engineStepsCodemodLog.Print("Found existing top-level 'steps'") | ||
| } | ||
|
|
||
| if hasTopLevelSteps { | ||
| // Find the end of the top-level steps block in the lines | ||
| for i, line := range frontmatterLines { |
There was a problem hiding this comment.
When merging into an existing top-level steps, the codemod only checks frontmatter["steps"] for presence, but doesn’t verify it’s actually a YAML sequence. If steps exists but is a scalar/object (invalid but possible in real files), this code will append list items after steps: <value>, producing invalid YAML. Consider type-checking frontmatter["steps"] as []any (or verifying the steps: line is followed by indented list items) before merging; otherwise, treat it as no top-level steps (insert a fresh steps: block) or return a clear error.
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Fixed in 5f5a9d9. |
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Fixed in 47d9311. The failures were in |
engine-steps-to-top-levelcodemod inpkg/cli/codemod_engine_steps.gopkg/cli/fix_codemods.gostepsfrom engine_config inpkg/parser/schemas/main_workflow_schema.jsonSteps []map[string]anyfromEngineConfigstruct and parsing inpkg/workflow/engine.goInjectCustomEngineStepsfunction frompkg/workflow/engine_helpers.goInjectCustomEngineStepscallsengine.stepsdetection frompkg/workflow/runtime_detection.goengine.stepspackage extraction frompkg/workflow/runtime_validation.goshared_workflow_test.go(use top-levelstepsinstead ofengine.steps)mcp_registry_live_test.go) to skip on 503/502/504 responses instead of failingfrontmatter["steps"]as[]anybefore treating as merge targetengine:block after removingstepswhen block has no remaining contentOriginal prompt
💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.