Skip to content

Comments

Dynamic template fetching from GitHub repos#279

Open
ejacquier wants to merge 90 commits intomainfrom
feature/dynamic-templates
Open

Dynamic template fetching from GitHub repos#279
ejacquier wants to merge 90 commits intomainfrom
feature/dynamic-templates

Conversation

@ejacquier
Copy link
Contributor

Dynamic template fetching from GitHub repos

Summary

  • Templates are no longer embedded in the CLI. They are standalone CRE projects hosted in GitHub repos, discovered via template.yaml metadata, downloaded as tarballs, and extracted into the user's workspace. No CLI release needed to add or update templates.
  • Two built-in Hello World templates (Go + TypeScript) remain embedded as offline fallbacks.
  • New cre templates list|add|remove commands to manage template repository sources, stored in ~/.cre/template.yaml.
  • template.yaml standard defines template metadata (kind, id, title, language, category, tags, capabilities, networks, workflows, projectDir, postInit, exclude).
  • Templates with projectDir set are treated as complete CRE projects — extracted as-is, skipping config generation and workflow renaming. Go templates with projectDir run go mod tidy post-scaffold.
  • Template list (1hr TTL) and tarballs (24hr TTL) are cached at ~/.cre/template-cache/. cre init --refresh bypasses cache.
  • Interactive Bubble Tea wizard for template selection with search, language filtering, and network RPC URL prompts.
  • CLI prompt helpers (ui.Confirm, ui.Input, ui.Select, ui.InputForm) consolidated from raw huh calls across all commands.

  - Create internal/ui/ package with centralized Lipgloss styles (styles.go)
  - Add output helpers for consistent styling: Title, Box, Success, Dim, etc.
  - Implement Bubble Tea spinner with reference counting for async operations
  - Add GlobalSpinner singleton for seamless spinner across CLI lifecycle
  - Update PersistentPreRunE to show spinner during initialization
  - Migrate cre init and cre whoami to use shared UI package
  - Add spinner during file generation (copying, generating templates, contracts)
  - Show spinner during Go dependencies installation
  - Display dependencies in styled box after spinner completes
  - Fix Next steps box spacing and formatting
  - Refactor initializeGoModule to return deps instead of printing
  - Add styled template functions (styleTitle, styleSection, styleCommand, styleDim, styleSuccess, styleCode)
  - Update help template to use Lipgloss styling
  - Style section headers, command names, tips, and URLs
  - Improve visual hierarchy and readability
  - Add complete Blocks palette constants (Gray, Blue, Green, Red, Orange, Yellow, Teal, Purple)
  - Use high-contrast colors for dark terminal readability
  - Style titles/commands with Blue 400-500 for visibility
  - Style secondary info with Gray 500 (dimmed)
  - Create custom Huh theme with Blocks colors for forms
  - Update spinner to use Blue 500
  - Add styled title and welcome message using Chainlink theme
  - Add Bubble Tea spinner with progress states throughout auth flow:
    - Preparing authentication
    - Opening browser
    - Waiting for authentication
    - Exchanging authorization code
    - Saving credentials
  - Show styled URL fallback when browser cannot open automatically
  - Display success message with next steps in branded box
  - Update spinner message during org membership retry flow
  - Update tests to include spinner in handler instantiations
  - Add SilenceErrors: true to root command to disable Cobra's default error output
  - Display all user-facing errors with styled ui.Error() in Execute()
  - Errors now show with red color and ✗ prefix consistent with Chainlink theme
  - Internal debug logging via zerolog remains unchanged
  - Add SilenceErrors: true to disable Cobra's default error output
  - Display errors with styled ui.Error() (red with ✗ prefix)
  - Set SilenceUsage in PersistentPreRunE to hide usage for runtime errors
  - Keep usage/suggestions visible for command typos and flag errors
    - Replaced prompt.YesNoPrompt with huh.NewConfirm forms
    - Removed stdin io.Reader parameter
  2. Updated cmd/creinit/creinit.go:
    - Updated call sites to match new function signatures
  3. Updated cmd/secrets/common/handler.go:
    - Replaced ~25 fmt.Print* calls with ui.* functions
  4. Updated cmd/workflow/simulate/telemetry_writer.go:
    - Replaced fmt.Printf with ui.Printf
    - Removed unused fmt import
  5. Deleted internal/prompt/ directory:
    - Removed entire old promptui-based package
  6. Cleaned cmd/common/utils.go:
    - Removed unused MustGetUserInputWithPrompt function
    - Removed unused bufio and errors imports
  7. Dependencies cleaned (go mod tidy):
    - Removed github.com/manifoldco/promptui
    - Removed github.com/chzyer/readline
…s.go) — An optional []string of chain names in template.yaml.

  Dynamic project.yaml RPC generation (internal/settings/):
  - project.yaml.tpl now uses a {{RPCsList}} placeholder instead of hardcoded chains
  - BuildRPCsListYAML() generates the rpcs YAML block from networks + URLs
  - Falls back to ethereum-testnet-sepolia with default public RPC when no networks specified

  New wizard step (cmd/creinit/wizard.go):
  - stepNetworkRPCs between template selection and workflow name
  - Prompts for each network's RPC URL sequentially
  - Empty input accepted (fill later), non-empty input validated as valid HTTP/HTTPS URL
  - Step is skipped if template has no networks or all RPCs provided via flags

  --rpc-url flag (cmd/creinit/creinit.go):
  - Repeatable StringArray flag: --rpc-url chain-name=url
  - Parsed in ResolveInputs(), merged with wizard results in Execute()
  - Flag values override wizard input
  - Add WorkflowDirEntry struct and Workflows/PostInit fields to
  TemplateMetadata in types.go
  - Rewrite renameWorkflowDir in registry.go to branch on workflow count:
   skip renaming for multi-workflow, rename single workflow, heuristic
  fallback for legacy
  - Update wizard to skip workflow name prompt for multi-workflow
  templates and default placeholder from Workflows[0].Dir
  - Default workflowName from Workflows[0].Dir in non-interactive mode
  - Loop GenerateWorkflowSettingsFile for each declared workflow dir in
  multi-workflow templates
  - Skip workflow.yaml generation when template already ships one
  (prevents overwriting custom config paths)
  - Rewrite printSuccessMessage to show per-workflow summary and postInit
   instructions
  - Add 6 new test cases covering multi-workflow, single-workflow
  defaults, rename with flag, and built-in backwards compat
@ejacquier ejacquier requested a review from a team as a code owner February 20, 2026 18:03
@github-actions
Copy link

👋 ejacquier, thanks for creating this pull request!

To help reviewers, please consider creating future PRs as drafts first. This allows you to self-review and make any final changes before notifying the team.

Once you're ready, you can mark it as "Ready for review" to request feedback. Thanks!

@github-actions
Copy link

⚠️ Abigen Fork Check - Update Available

The forked abigen package is outdated and may be missing important updates.

Version Value
Current Fork v1.16.0
Latest Upstream v1.17.0

Action Required

  1. Review abigen changes in upstream (only the accounts/abi/bind directory matters)
  2. Compare with our fork in cmd/generate-bindings/bindings/abigen/
  3. If relevant changes exist, sync them and update FORK_METADATA.md
  4. If no abigen changes, just update the version in FORK_METADATA.md to v1.17.0

Files to Review

  • cmd/generate-bindings/bindings/abigen/bind.go
  • cmd/generate-bindings/bindings/abigen/bindv2.go
  • cmd/generate-bindings/bindings/abigen/template.go

⚠️ Note to PR author: This is not something you need to fix. The Platform Expansion team is responsible for maintaining the abigen fork.

cc @smartcontractkit/bix-framework

@github-actions
Copy link

⚠️ Abigen Fork Check - Update Available

The forked abigen package is outdated and may be missing important updates.

Version Value
Current Fork v1.16.0
Latest Upstream v1.17.0

Action Required

  1. Review abigen changes in upstream (only the accounts/abi/bind directory matters)
  2. Compare with our fork in cmd/generate-bindings/bindings/abigen/
  3. If relevant changes exist, sync them and update FORK_METADATA.md
  4. If no abigen changes, just update the version in FORK_METADATA.md to v1.17.0

Files to Review

  • cmd/generate-bindings/bindings/abigen/bind.go
  • cmd/generate-bindings/bindings/abigen/bindv2.go
  • cmd/generate-bindings/bindings/abigen/template.go

⚠️ Note to PR author: This is not something you need to fix. The Platform Expansion team is responsible for maintaining the abigen fork.

cc @smartcontractkit/bix-framework

@github-actions
Copy link

⚠️ Abigen Fork Check - Update Available

The forked abigen package is outdated and may be missing important updates.

Version Value
Current Fork v1.16.0
Latest Upstream v1.17.0

Action Required

  1. Review abigen changes in upstream (only the accounts/abi/bind directory matters)
  2. Compare with our fork in cmd/generate-bindings/bindings/abigen/
  3. If relevant changes exist, sync them and update FORK_METADATA.md
  4. If no abigen changes, just update the version in FORK_METADATA.md to v1.17.0

Files to Review

  • cmd/generate-bindings/bindings/abigen/bind.go
  • cmd/generate-bindings/bindings/abigen/bindv2.go
  • cmd/generate-bindings/bindings/abigen/template.go

⚠️ Note to PR author: This is not something you need to fix. The Platform Expansion team is responsible for maintaining the abigen fork.

cc @smartcontractkit/bix-framework

var topLevelPrefix string

for {
header, err := tr.Next()

Check failure

Code scanning / CodeQL

Arbitrary file access during archive extraction ("Zip Slip") High

Unsanitized archive entry, which may contain '..', is used in a
file system operation
.
Unsanitized archive entry, which may contain '..', is used in a
file system operation
.
Unsanitized archive entry, which may contain '..', is used in a
file system operation
.

Copilot Autofix

AI about 6 hours ago

In general, the fix is to treat header.Name as untrusted, normalize the resulting output path, and ensure it stays within the intended destination directory. This typically involves: (1) cleaning the relative path (filepath.Clean), (2) rejecting any path that is absolute or that starts with .. (or contains .. path components), and (3) after joining with destDir, verifying that the resulting absolute path still has destDir as a prefix.

For this specific code, we can introduce a small helper that, given destDir and relPath, returns a safe targetPath or an error if the entry is unsafe. After computing relPath (and before any filesystem operation), we will:

  • cleanRel := filepath.Clean(relPath)
  • Skip entries where cleanRel == ".", filepath.IsAbs(cleanRel), or strings.HasPrefix(cleanRel, ".."+string(os.PathSeparator)) or cleanRel == "..".
  • Build targetPath := filepath.Join(destDir, cleanRel).
  • Compute absDest and absTarget with filepath.Abs, and ensure absTarget == absDest or strings.HasPrefix(absTarget, absDest+string(os.PathSeparator)). If not, skip or error.

This can be done inline without changing external behavior for normal, well-formed tarballs; only malicious or malformed paths will be ignored (or cause an error, depending on desired policy). We already import filepath, os, and strings, so no new imports are needed. The changes are all localized around the computation and use of relPath / targetPath in internal/templaterepo/client.go, ensuring all three CodeQL variants are addressed by validating before using targetPath in os.MkdirAll, filepath.Dir, and os.OpenFile.


Suggested changeset 1
internal/templaterepo/client.go

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/internal/templaterepo/client.go b/internal/templaterepo/client.go
--- a/internal/templaterepo/client.go
+++ b/internal/templaterepo/client.go
@@ -384,6 +384,17 @@
 			continue
 		}
 
+		// Normalize the relative path and prevent path traversal or escaping destDir.
+		relPath = filepath.Clean(relPath)
+		if relPath == "." {
+			continue
+		}
+		// Disallow absolute paths and any leading ".." segments.
+		if filepath.IsAbs(relPath) || relPath == ".." || strings.HasPrefix(relPath, ".."+string(os.PathSeparator)) {
+			c.logger.Warn().Msgf("Skipping unsafe archive entry path: %s", relPath)
+			continue
+		}
+
 		// Check standard ignores
 		if shouldIgnore(relPath, standardIgnores) {
 			continue
@@ -396,6 +407,20 @@
 
 		targetPath := filepath.Join(destDir, relPath)
 
+		// Ensure the final path is still within destDir.
+		absDest, err := filepath.Abs(destDir)
+		if err != nil {
+			return fmt.Errorf("failed to resolve destination directory: %w", err)
+		}
+		absTarget, err := filepath.Abs(targetPath)
+		if err != nil {
+			return fmt.Errorf("failed to resolve target path: %w", err)
+		}
+		if absTarget != absDest && !strings.HasPrefix(absTarget, absDest+string(os.PathSeparator)) {
+			c.logger.Warn().Msgf("Skipping archive entry escaping destination dir: %s -> %s", relPath, absTarget)
+			continue
+		}
+
 		switch header.Typeflag {
 		case tar.TypeDir:
 			c.logger.Debug().Msgf("Extracting dir: %s -> %s", name, targetPath)
EOF
@@ -384,6 +384,17 @@
continue
}

// Normalize the relative path and prevent path traversal or escaping destDir.
relPath = filepath.Clean(relPath)
if relPath == "." {
continue
}
// Disallow absolute paths and any leading ".." segments.
if filepath.IsAbs(relPath) || relPath == ".." || strings.HasPrefix(relPath, ".."+string(os.PathSeparator)) {
c.logger.Warn().Msgf("Skipping unsafe archive entry path: %s", relPath)
continue
}

// Check standard ignores
if shouldIgnore(relPath, standardIgnores) {
continue
@@ -396,6 +407,20 @@

targetPath := filepath.Join(destDir, relPath)

// Ensure the final path is still within destDir.
absDest, err := filepath.Abs(destDir)
if err != nil {
return fmt.Errorf("failed to resolve destination directory: %w", err)
}
absTarget, err := filepath.Abs(targetPath)
if err != nil {
return fmt.Errorf("failed to resolve target path: %w", err)
}
if absTarget != absDest && !strings.HasPrefix(absTarget, absDest+string(os.PathSeparator)) {
c.logger.Warn().Msgf("Skipping archive entry escaping destination dir: %s -> %s", relPath, absTarget)
continue
}

switch header.Typeflag {
case tar.TypeDir:
c.logger.Debug().Msgf("Extracting dir: %s -> %s", name, targetPath)
Copilot is powered by AI and may make mistakes. Always verify output.
@github-actions
Copy link

⚠️ Abigen Fork Check - Update Available

The forked abigen package is outdated and may be missing important updates.

Version Value
Current Fork v1.16.0
Latest Upstream v1.17.0

Action Required

  1. Review abigen changes in upstream (only the accounts/abi/bind directory matters)
  2. Compare with our fork in cmd/generate-bindings/bindings/abigen/
  3. If relevant changes exist, sync them and update FORK_METADATA.md
  4. If no abigen changes, just update the version in FORK_METADATA.md to v1.17.0

Files to Review

  • cmd/generate-bindings/bindings/abigen/bind.go
  • cmd/generate-bindings/bindings/abigen/bindv2.go
  • cmd/generate-bindings/bindings/abigen/template.go

⚠️ Note to PR author: This is not something you need to fix. The Platform Expansion team is responsible for maintaining the abigen fork.

cc @smartcontractkit/bix-framework

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant