Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 71 additions & 0 deletions .github/workflows/coverage.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
name: Repository Coverage

on:
pull_request:
types: [opened, synchronize, reopened, ready_for_review]
workflow_dispatch:

concurrency:
group: repository-coverage-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

permissions:
contents: read

jobs:
coverage:
name: Test coverage policy
runs-on: ubuntu-latest
timeout-minutes: 60

steps:
- name: Checkout code
uses: actions/checkout@v4
with:
submodules: recursive

- name: Set up JDK 17
uses: actions/setup-java@v4
with:
distribution: temurin
java-version: "17"
cache: maven

- name: Set up Python 3.11
uses: actions/setup-python@v5
with:
python-version: "3.11"
cache: pip
cache-dependency-path: |
python-ecosystem/rag-pipeline/requirements.txt
python-ecosystem/inference-orchestrator/src/requirements.txt
python-ecosystem/inference-orchestrator/src/requirements.test.txt

- name: Run repository coverage gate
run: bash tools/coverage/check-repository-coverage.sh

- name: Publish coverage summary
if: always()
shell: bash
run: |
if [ -f build/coverage/summary.md ]; then
cat build/coverage/summary.md >> "$GITHUB_STEP_SUMMARY"
else
echo "# Repository coverage gate" >> "$GITHUB_STEP_SUMMARY"
echo "" >> "$GITHUB_STEP_SUMMARY"
echo "The coverage runner stopped before it could create a summary." >> "$GITHUB_STEP_SUMMARY"
fi

- name: Upload coverage and test reports
if: always()
uses: actions/upload-artifact@v4
with:
name: repository-coverage-${{ github.run_id }}
if-no-files-found: warn
retention-days: 14
path: |
build/coverage/
java-ecosystem/**/target/site/jacoco/
analysis-plugins/**/target/site/jacoco/
java-ecosystem/**/target/surefire-reports/
java-ecosystem/**/target/failsafe-reports/
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ Thumbs.db
coverage.xml
htmlcov/
.ci-test-results/
build/coverage/
.coverage-venvs/

# Python environments and packaging output
.venv/
Expand Down
15 changes: 8 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,8 @@ These features are platform-independent and available through the CodeCrow web U

| Method | Bitbucket Cloud | GitHub | GitLab |
| :----------------------- | :-------------: | :---------------------------------: | :---------------------------------------: |
| OAuth / App Installation | ✅ (OAuth) | ✅ (GitHub App with OAuth fallback) | ✅ (OAuth, including self-managed GitLab) |
| OAuth / App Installation | ✅ (OAuth) | ✅ (GitHub App with OAuth fallback) | ✅ (GitLab.com only) |
| Self-managed VCS | — | — | ✅ (personal or project access token) |
| Manual Webhook | ✅ | ✅ | ✅ |
| CI Pipeline Action | ✅ | — | — |

Expand Down Expand Up @@ -234,12 +235,12 @@ for the detailed invariants and failure behavior.
## Self-Hosting and Build Verification

The interactive setup configures secrets and chooses OpenRouter or Ollama for
embeddings. The local production build synchronizes the pinned frontend
submodule, rejects local frontend drift, recreates the two isolated Python 3.11
CI environments, and runs the same Python, plugin-boundary, Maven `verify`, and
observable-image Buildx gates as CI/CD. Only after every gate passes does it
replace the local Compose services with those validated images and wait for
health checks.
embeddings. The local production build fetches and checks out the latest commit
from the frontend submodule's configured `main` branch, rejects local frontend
drift, recreates the two isolated Python 3.11 CI environments, and runs the same
Python, plugin-boundary, Maven `verify`, and observable-image Buildx gates as
CI/CD. Only after every gate passes does it replace the local Compose services
with those validated images and wait for health checks.

```bash
cd deployment
Expand Down
17 changes: 16 additions & 1 deletion deployment/config/rag-pipeline/.env.sample
Original file line number Diff line number Diff line change
Expand Up @@ -33,20 +33,35 @@ SERVICE_SECRET=change-me-to-a-random-secret
# OLLAMA_MAX_CHARS=24000
# OLLAMA_MAX_RETRIES=3
# OLLAMA_RETRY_BASE_DELAY=1.0
# OPENROUTER_BATCH_SIZE=100
# OPENROUTER_BATCH_SIZE=50
# OPENROUTER_TIMEOUT=300
# OPENROUTER_MAX_CHARS=24000
# Maximum parallel batches for one index. Set to 1 for serial rollback behavior.
# OPENROUTER_INDEX_CONCURRENCY=8
# Cross-worker capacity cap. Redis coordinates the normal path; a Redis outage
# degrades this performance-only limiter to the process-local cap.
# OPENROUTER_MAX_IN_FLIGHT=16
# Use "price" to retain OpenRouter's default price-oriented routing.
# OPENROUTER_INDEX_PROVIDER_SORT=throughput
# OPENROUTER_QUERY_PROVIDER_SORT=latency

# === Qdrant Storage ===
# QDRANT_URL=http://qdrant:6333
# QDRANT_API_KEY=
# QDRANT_COLLECTION_PREFIX=codecrow
# QDRANT_VECTORS_ON_DISK=true
# QDRANT_UPSERT_BATCH_SIZE=128

# === Queue and Server Runtime ===
# REDIS_URL=redis://redis:6379/1
# MAX_CONCURRENT_RAG_JOBS=2
# UVICORN_WORKERS=4
# Project mutation coordination is correctness-critical in multi-worker setups.
# RAG_MUTATION_LEASE_SECONDS=300
# RAG_MUTATION_ACQUIRE_TIMEOUT_SECONDS=5
# Expired pending collections are retained for six hours by default.
# RAG_PENDING_COLLECTION_MAX_AGE_SECONDS=21600
# RAG_PENDING_JANITOR_INTERVAL_SECONDS=3600

# === Repository and API Safety Limits ===
# Root directory that repo_path arguments may resolve under.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -332,6 +332,7 @@ private Map<String, Object> buildSerializableRequestPayload(AiAnalysisRequest re
payload.put("ragEnabled", request.getRagEnabled());
payload.put("analysisType", request.getAnalysisType());
payload.put("vcsProvider", request.getVcsProvider());
payload.put("vcsBaseUrl", request.getVcsBaseUrl());
payload.put("prTitle", request.getPrTitle());
payload.put("prDescription", request.getPrDescription());
payload.put("taskContext", request.getTaskContext());
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -179,7 +179,8 @@ public record SummarizeRequest(
String accessToken,
boolean supportsMermaid,
Integer maxAllowedTokens,
String vcsProvider) {
String vcsProvider,
String vcsBaseUrl) {
}

/**
Expand All @@ -203,6 +204,7 @@ public record AskRequest(
String accessToken,
Integer maxAllowedTokens,
String vcsProvider,
String vcsBaseUrl,
String analysisContext,
java.util.List<String> issueReferences) {
}
Expand Down Expand Up @@ -244,7 +246,8 @@ public record ReviewRequest(
String oAuthSecret,
String accessToken,
Integer maxAllowedTokens,
String vcsProvider) {
String vcsProvider,
String vcsBaseUrl) {
}

/**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,8 @@ public interface AiAnalysisRequest {

String getVcsProvider();

default String getVcsBaseUrl() { return null; }

String getPrTitle();

String getPrDescription();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ public class AiAnalysisRequestImpl implements AiAnalysisRequest {
protected final String targetBranchName;
protected final String sourceBranchName;
protected final String vcsProvider;
protected final String vcsBaseUrl;
protected final String rawDiff;

// Incremental analysis fields
Expand Down Expand Up @@ -98,6 +99,7 @@ protected AiAnalysisRequestImpl(Builder<?> builder) {
this.targetBranchName = builder.targetBranchName;
this.sourceBranchName = builder.sourceBranchName;
this.vcsProvider = builder.vcsProvider;
this.vcsBaseUrl = builder.vcsBaseUrl;
this.rawDiff = builder.rawDiff;
// Incremental analysis fields
this.analysisMode = builder.analysisMode != null ? builder.analysisMode : AnalysisMode.FULL;
Expand Down Expand Up @@ -231,6 +233,11 @@ public String getVcsProvider() {
return vcsProvider;
}

@Override
public String getVcsBaseUrl() {
return vcsBaseUrl;
}

public String getRawDiff() {
return rawDiff;
}
Expand Down Expand Up @@ -311,6 +318,7 @@ public static class Builder<T extends Builder<T>> {
private String targetBranchName;
private String sourceBranchName;
private String vcsProvider;
private String vcsBaseUrl;
private String rawDiff;
// Incremental analysis fields
private AnalysisMode analysisMode;
Expand Down Expand Up @@ -606,6 +614,11 @@ public T withVcsProvider(String vcsProvider) {
return self();
}

public T withVcsBaseUrl(String vcsBaseUrl) {
this.vcsBaseUrl = vcsBaseUrl;
return self();
}

public T withRawDiff(String rawDiff) {
this.rawDiff = rawDiff;
return self();
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
package org.rostilos.codecrow.analysisengine.processor.analysis;

import okhttp3.OkHttpClient;
import org.rostilos.codecrow.analysisengine.aiclient.AiAnalysisClient;
import org.rostilos.codecrow.commitgraph.dag.CommitRangeContext;
import org.rostilos.codecrow.analysisengine.processor.VcsRepoInfoImpl;
Expand All @@ -21,7 +20,6 @@
import org.rostilos.codecrow.analysisengine.service.PullRequestStatusSyncService;
import org.rostilos.codecrow.commitgraph.service.CommitCoverageService;
import org.rostilos.codecrow.analysisengine.service.vcs.VcsAiClientService;
import org.rostilos.codecrow.analysisengine.service.vcs.VcsOperationsService;
import org.rostilos.codecrow.analysisengine.service.vcs.VcsServiceFactory;
import org.rostilos.codecrow.analysisapi.rag.RagOperationsService;
import org.rostilos.codecrow.analysisengine.util.ProjectVcsInfoRetriever;
Expand Down Expand Up @@ -195,9 +193,8 @@ && matchCache(request, existingBranchOpt, project, consumer)) {
"Branch analysis started for branch: " + request.getTargetBranchName());

VcsRepoInfoImpl vcsRepoInfoImpl = ProjectVcsInfoRetriever.getVcsInfo(project);
OkHttpClient client = vcsClientProvider.getHttpClient(vcsRepoInfoImpl.vcsConnection());
VcsClient client = vcsClientProvider.getClient(vcsRepoInfoImpl.vcsConnection());
EVcsProvider provider = ProjectVcsInfoRetriever.getVcsProvider(project);
VcsOperationsService operationsService = vcsServiceFactory.getOperationsService(provider);

// ── Commit range resolution ───────────────────────────────────
CommitRangeContext rangeCtx = branchCommitService.resolveCommitRange(project,
Expand All @@ -209,7 +206,7 @@ && matchCache(request, existingBranchOpt, project, consumer)) {
EventNotificationEmitter.emitStatus(consumer, "fetching_diff", "Fetching diff for analysis");

// ── PR number resolution ─────────────────────────────────────────
Long prNumber = resolvePrNumber(request, operationsService, client, vcsRepoInfoImpl);
Long prNumber = resolvePrNumber(request, client, vcsRepoInfoImpl);
List<String> prLookupCommitCandidates = new ArrayList<>();
if (request.getCommitHash() != null && !request.getCommitHash().isBlank()) {
prLookupCommitCandidates.add(request.getCommitHash());
Expand Down Expand Up @@ -242,8 +239,8 @@ && matchCache(request, existingBranchOpt, project, consumer)) {
String sourceParent = headCommits.get(0).parentHashes().get(1);
prLookupCommitCandidates.add(sourceParent);
try {
prNumber = operationsService.findPullRequestForCommit(
client, vcsRepoInfoImpl.workspace(),
prNumber = client.findPullRequestForCommit(
vcsRepoInfoImpl.workspace(),
vcsRepoInfoImpl.repoSlug(), sourceParent);
if (isValidPrNumber(prNumber)) {
log.info("Found PR #{} from merge commit's second parent {}",
Expand Down Expand Up @@ -309,7 +306,7 @@ && matchCache(request, existingBranchOpt, project, consumer)) {
// ── Multi-tier diff strategy ─────────────────────────────────────
Long diffPrNumber = mergedPrNumbers.size() > 1 ? null : prNumber;
String repositoryDiff = branchDiffFetcher.fetchDiff(request, existingBranchOpt.orElse(null), rangeCtx,
operationsService, client, vcsRepoInfoImpl, diffPrNumber, unanalyzedCommits);
client, vcsRepoInfoImpl, diffPrNumber, unanalyzedCommits);
String rawDiff = AnalysisScopeFilter.filterDiff(repositoryDiff, project);

Set<String> changedFiles = DiffParsingUtils.parseFilePathsFromDiff(rawDiff);
Expand Down Expand Up @@ -518,16 +515,15 @@ private boolean matchCache(BranchProcessRequest request, Optional<Branch> existi
* This handles cases where branch analysis is triggered by push events.
*/
private Long resolvePrNumber(BranchProcessRequest request,
VcsOperationsService operationsService,
OkHttpClient client, VcsRepoInfoImpl vcsRepoInfoImpl) {
VcsClient client, VcsRepoInfoImpl vcsRepoInfoImpl) {
Long prNumber = request.getSourcePrNumber();
if (!isValidPrNumber(prNumber)) {
prNumber = null;
}
if (prNumber == null && request.getCommitHash() != null) {
try {
prNumber = operationsService.findPullRequestForCommit(
client, vcsRepoInfoImpl.workspace(), vcsRepoInfoImpl.repoSlug(),
prNumber = client.findPullRequestForCommit(
vcsRepoInfoImpl.workspace(), vcsRepoInfoImpl.repoSlug(),
request.getCommitHash());
if (isValidPrNumber(prNumber)) {
log.info("Found PR #{} for commit {} via API lookup", prNumber,
Expand Down
Loading
Loading