diff --git a/catalog/Frameworks/Official-DotNet-ASPNetCore/skills/configuring-opentelemetry-dotnet/SKILL.md b/catalog/Frameworks/Official-DotNet-ASPNetCore/skills/configuring-opentelemetry-dotnet/SKILL.md deleted file mode 100644 index c99fe5c5..00000000 --- a/catalog/Frameworks/Official-DotNet-ASPNetCore/skills/configuring-opentelemetry-dotnet/SKILL.md +++ /dev/null @@ -1,289 +0,0 @@ ---- -name: configuring-opentelemetry-dotnet -description: Configure OpenTelemetry distributed tracing, metrics, and logging in ASP.NET Core using the .NET OpenTelemetry SDK. Use when adding observability, setting up OTLP exporters, creating custom metrics/spans, or troubleshooting distributed trace correlation. -license: MIT ---- - -# Configuring OpenTelemetry in .NET - -## When to Use - -- Adding distributed tracing to an ASP.NET Core application -- Setting up OpenTelemetry exporters (OTLP is the primary protocol; Jaeger accepts OTLP natively; Prometheus OTLP ingestion requires explicit opt-in) -- Creating custom metrics or trace spans for business operations -- Troubleshooting distributed trace context propagation across services - -## When Not to Use - -- The user wants application-level logging only (use ILogger, Serilog) -- The user is using Application Insights SDK directly (different API) -- The user needs APM with a commercial vendor's proprietary SDK - -## Inputs - -| Input | Required | Description | -|-------|----------|-------------| -| ASP.NET Core project | Yes | The application to instrument | -| Observability backend | No | Where to export: OTLP collector, Aspire dashboard, Jaeger (accepts OTLP natively) | - -## Workflow - -### Step 1: Install the correct packages - -**There are many OpenTelemetry NuGet packages. Install exactly these:** - -```bash -# Core SDK + ASP.NET Core instrumentation + logging integration -dotnet add package OpenTelemetry.Extensions.Hosting -dotnet add package OpenTelemetry.Instrumentation.AspNetCore -dotnet add package OpenTelemetry.Instrumentation.Http - -# Exporter -dotnet add package OpenTelemetry.Exporter.OpenTelemetryProtocol # OTLP exporter for traces, metrics, AND logs - -# Optional — dev/local debugging only (do NOT include in production deployments) -# dotnet add package OpenTelemetry.Exporter.Console -``` - -**Do NOT install `OpenTelemetry` alone** — you need `OpenTelemetry.Extensions.Hosting` for proper DI integration. - -#### Optional: additional auto-instrumentation packages - -Install only the packages that match the libraries your application uses: - -```bash -dotnet add package OpenTelemetry.Instrumentation.SqlClient # SQL Server queries -dotnet add package OpenTelemetry.Instrumentation.EntityFrameworkCore # EF Core -dotnet add package OpenTelemetry.Instrumentation.GrpcNetClient # gRPC calls -dotnet add package OpenTelemetry.Instrumentation.Runtime # GC, thread pool metrics -``` - -### Step 2: Configure all signals in Program.cs - -```csharp -using OpenTelemetry.Resources; -using OpenTelemetry.Trace; -using OpenTelemetry.Metrics; -using OpenTelemetry.Logs; - -var builder = WebApplication.CreateBuilder(args); - -builder.Services.AddOpenTelemetry() - .ConfigureResource(resource => resource - .AddService(serviceName: builder.Environment.ApplicationName)) - .WithTracing(tracing => tracing - .AddAspNetCoreInstrumentation(options => - { - // Filter out health check endpoints from traces - options.Filter = httpContext => - !httpContext.Request.Path.StartsWithSegments("/healthz"); - }) - .AddHttpClientInstrumentation(options => - { - options.RecordException = true; - }) - // Optional: add SQL instrumentation if using SqlClient directly - // .AddSqlClientInstrumentation(options => - // { - // options.SetDbStatementForText = true; - // options.RecordException = true; - // }) - // Custom activity sources (must match ActivitySource names in your code) - .AddSource("MyApp.Orders") - .AddSource("MyApp.Payments") - .AddSource("MyApp.Messaging")) - .WithMetrics(metrics => metrics - .AddAspNetCoreInstrumentation() - .AddHttpClientInstrumentation() - // Optional: .AddRuntimeInstrumentation() for GC and thread pool metrics - // (requires OpenTelemetry.Instrumentation.Runtime package) - // Custom meters (must match Meter names in your code) - .AddMeter("MyApp.Metrics")) - .WithLogging(logging => - { - logging.IncludeScopes = true; - // logging.IncludeFormattedMessage = true; // Enable if you need the formatted message string in log exports - }) - // Single OTLP exporter for all signals — reads OTEL_EXPORTER_OTLP_ENDPOINT - // env var (defaults to http://localhost:4317). Override via environment variable - // or appsettings.json configuration. - .UseOtlpExporter(); -``` - -### Step 3: Understanding log–trace correlation - -The `.WithLogging()` call in Step 2 integrates ILogger with OpenTelemetry: - -- Each log entry automatically includes TraceId and SpanId for correlation with traces -- The service resource from `.ConfigureResource()` propagates to logs automatically -- `UseOtlpExporter()` applies to logs alongside traces and metrics -- No additional packages or separate `SetResourceBuilder` call needed - -### Step 4: Create custom spans (Activities) for business operations - -```csharp -using System.Diagnostics; -using Microsoft.Extensions.Logging; - -public class OrderService -{ - // Create an ActivitySource matching what you registered in Step 2 - private static readonly ActivitySource ActivitySource = new("MyApp.Orders"); - private readonly ILogger _logger; - - public OrderService(ILogger logger) => _logger = logger; - - public async Task ProcessOrderAsync(CreateOrderRequest request) - { - // Start a new span - using var activity = ActivitySource.StartActivity("ProcessOrder"); - - // Add attributes (tags) to the span - activity?.SetTag("order.customer_id", request.CustomerId); - activity?.SetTag("order.item_count", request.Items.Count); - - try - { - // Child span for validation - using (var validationActivity = ActivitySource.StartActivity("ValidateOrder")) - { - await ValidateOrderAsync(request); - validationActivity?.SetTag("validation.result", "passed"); - } - - // Child span for payment - using (var paymentActivity = ActivitySource.StartActivity("ProcessPayment", - ActivityKind.Client)) // Client = outgoing call - { - paymentActivity?.SetTag("payment.method", request.PaymentMethod); - await ProcessPaymentAsync(request); - } - - var order = new Order { Id = Guid.NewGuid(), CustomerId = request.CustomerId, Status = "Completed" }; - - activity?.SetTag("order.status", "completed"); - activity?.SetStatus(ActivityStatusCode.Ok); - - return order; - } - catch (Exception ex) - { - activity?.SetStatus(ActivityStatusCode.Error, ex.Message); - // Log via ILogger — OpenTelemetry captures this with trace correlation. - // Prefer logging over activity.RecordException() as OTel is deprecating - // span events for exception recording in favor of log-based exceptions. - _logger.LogError(ex, "Order processing failed for customer {CustomerId}", request.CustomerId); - throw; - } - } -} -``` - -**Critical: `ActivitySource` name must match `AddSource("...")` in configuration.** Unmatched sources are silently ignored — this is the #1 debugging issue. - -### Step 5: Create custom metrics - -Use `IMeterFactory` (injected via DI) to create meters — this ensures proper lifetime management and testability. - -```csharp -using System.Diagnostics; -using System.Diagnostics.Metrics; - -public class OrderMetrics -{ - private readonly Counter _ordersProcessed; - private readonly Histogram _orderProcessingDuration; - private readonly UpDownCounter _activeOrders; - - public OrderMetrics(IMeterFactory meterFactory) - { - // Meter name must match AddMeter("...") in configuration - var meter = meterFactory.Create("MyApp.Metrics"); - - // Counter — use for things that only go up - _ordersProcessed = meter.CreateCounter( - "orders.processed", "orders", "Total orders successfully processed"); - - // Histogram — use for measuring distributions (latency, sizes) - _orderProcessingDuration = meter.CreateHistogram( - "orders.processing_duration", "ms", "Time to process an order"); - - // UpDownCounter — use for things that go up AND down - _activeOrders = meter.CreateUpDownCounter( - "orders.active", "orders", "Currently processing orders"); - } - - public void RecordOrderProcessed(string region, double durationMs) - { - // Tags enable dimensional filtering (by region, status, etc.) - var tags = new TagList - { - { "region", region }, - { "order.type", "standard" } - }; - - _ordersProcessed.Add(1, tags); - _orderProcessingDuration.Record(durationMs, tags); - } -} -``` - -Register `OrderMetrics` in DI: - -```csharp -builder.Services.AddSingleton(); -``` - -### Step 6: Configure context propagation for distributed scenarios - -Trace context propagation is automatic for HTTP calls when using `AddHttpClientInstrumentation()`. For non-HTTP scenarios: - -```csharp -using System; -using System.Collections.Generic; -using System.Diagnostics; -using OpenTelemetry.Context.Propagation; - -// ActivitySource should be static — register via .AddSource("MyApp.Messaging") in Step 2 -private static readonly ActivitySource MessageSource = new("MyApp.Messaging"); - -// Manual context propagation (e.g., across message queues) -// On the SENDING side: -var propagator = Propagators.DefaultTextMapPropagator; -var activityContext = Activity.Current?.Context ?? default; -var context = new PropagationContext(activityContext, Baggage.Current); -var carrier = new Dictionary(); - -propagator.Inject(context, carrier, (dict, key, value) => dict[key] = value); -// Send carrier dictionary as message headers - -// On the RECEIVING side: -var parentContext = propagator.Extract(default, carrier, - (dict, key) => dict.TryGetValue(key, out var value) ? new[] { value } : Array.Empty()); - -Baggage.Current = parentContext.Baggage; -using var activity = MessageSource.StartActivity("ProcessMessage", - ActivityKind.Consumer, - parentContext.ActivityContext); // Links to parent trace! -``` - -## Validation - -- [ ] Traces appear in the observability backend (Jaeger, Aspire dashboard, etc.) -- [ ] HTTP requests automatically create spans with correct verb, URL, status code -- [ ] Custom `ActivitySource` names match `AddSource()` registrations -- [ ] Custom `Meter` names match `AddMeter()` registrations -- [ ] Logs include TraceId and SpanId for correlation -- [ ] Health check endpoints are filtered from traces -- [ ] Exception details appear on error spans - -## Common Pitfalls - -| Pitfall | Solution | -|---------|----------| -| `ActivitySource.StartActivity` returns null | Source name doesn't match any `AddSource()` — names must match exactly | -| Traces not appearing in exporter | Check OTLP endpoint: gRPC uses port 4317, HTTP uses 4318 | -| Missing HTTP client spans | Ensure `AddHttpClientInstrumentation()` is registered; it works for both `IHttpClientFactory`/DI and `new HttpClient()` (use `IHttpClientFactory` for lifetime management) | -| High cardinality tags | Don't use user IDs, request IDs, or UUIDs as metric tags — explodes storage | -| OTLP gRPC vs HTTP mismatch | Default is gRPC (port 4317); if collector only accepts HTTP, set `OtlpExportProtocol.HttpProtobuf` | -| `Meter` / `ActivitySource` lifecycle | `ActivitySource` should be static; create `Meter` via `IMeterFactory` from DI (not `new Meter()`) for proper lifetime management and testability | diff --git a/catalog/Frameworks/Official-DotNet-ASPNetCore/skills/configuring-opentelemetry-dotnet/manifest.json b/catalog/Frameworks/Official-DotNet-ASPNetCore/skills/configuring-opentelemetry-dotnet/manifest.json deleted file mode 100644 index e224a034..00000000 --- a/catalog/Frameworks/Official-DotNet-ASPNetCore/skills/configuring-opentelemetry-dotnet/manifest.json +++ /dev/null @@ -1,6 +0,0 @@ -{ - "version": "0.1.1", - "category": "Web", - "compatibility": "Requires an ASP.NET Core project or solution.", - "package_prefix": "OpenTelemetry" -} diff --git a/catalog/Frameworks/Official-DotNet-ASPNetCore/skills/minimal-api-file-upload/SKILL.md b/catalog/Frameworks/Official-DotNet-ASPNetCore/skills/minimal-api-file-upload/SKILL.md deleted file mode 100644 index 6ec5fc80..00000000 --- a/catalog/Frameworks/Official-DotNet-ASPNetCore/skills/minimal-api-file-upload/SKILL.md +++ /dev/null @@ -1,235 +0,0 @@ ---- -name: minimal-api-file-upload -description: File upload endpoints in ASP.NET minimal APIs (.NET 8+) -license: MIT ---- - -# Implementing File Uploads in ASP.NET Core Minimal APIs - -## When to Use -- File upload endpoints in ASP.NET Core minimal APIs (.NET 8+) -- Handling IFormFile or IFormFileCollection parameters -- When you need size limits, content type validation, or streaming large files - -## When Not to Use -- MVC controllers → `[FromForm] IFormFile` works directly with attributes -- Simple JSON body → no file upload needed -- Very large files (> 1GB) → use streaming with `MultipartReader` instead - -## Inputs - -| Input | Required | Description | -|-------|----------|-------------| -| File parameter(s) | Yes | IFormFile or IFormFileCollection | -| Size limits | Yes | Max file/request size | -| Allowed types | No | Content type or extension restrictions | - -## Workflow - -### Step 1: CRITICAL — Understand IFormFile Binding in Minimal APIs - -```csharp -// In .NET 8+ minimal APIs, IFormFile binds automatically from multipart/form-data -// when it is the only complex parameter. -app.MapPost("/upload", (IFormFile file) => ...); - -// CRITICAL: When you mix files with other form fields, use [FromForm] on all -// form-bound parameters (or group them into a single [FromForm] DTO). -app.MapPost("/upload-with-metadata", - ([FromForm] IFormFile file, [FromForm] string description) => -{ - return Results.Ok(new { file.FileName, Description = description }); -}); - -// Multiple files: IFormFileCollection also binds automatically from multipart/form-data. -// You only need [FromForm] if you mix it with other form fields, as shown above. -app.MapPost("/upload-multiple", (IFormFileCollection files) => -{ - return Results.Ok(files.Select(f => new { f.FileName, f.Length })); -}); -``` - -### Step 2: CRITICAL — File Size Limits Are Separate from Request Size Limits - -```csharp -// CRITICAL: There are TWO different size limits and you need to configure BOTH - -// 1. Request body size limit (Kestrel level) — default is 30MB -builder.WebHost.ConfigureKestrel(options => -{ - options.Limits.MaxRequestBodySize = 10 * 1024 * 1024; // 10 MB -}); - -// 2. Form options — multipart body length limit — default is 128MB -builder.Services.Configure(options => -{ - options.MultipartBodyLengthLimit = 10 * 1024 * 1024; // 10 MB - options.ValueLengthLimit = 1024 * 1024; // 1 MB for form values - options.MultipartHeadersLengthLimit = 16384; // 16 KB for section headers -}); - -// COMMON MISTAKE: Only increasing Kestrel MaxRequestBodySize -// upload still fails because FormOptions.MultipartBodyLengthLimit is exceeded - -// COMMON MISTAKE: Only increasing FormOptions -// upload fails with "Request body too large" from Kestrel before reaching form parsing - -// CRITICAL: Per-endpoint override with RequestSizeLimit attribute -app.MapPost("/upload-large", [RequestSizeLimit(200_000_000)] (IFormFile file) => -{ - return Results.Ok(new { file.FileName, file.Length }); -}); - -// CRITICAL: To disable the limit entirely (for streaming): -app.MapPost("/upload-unlimited", [DisableRequestSizeLimit] async (HttpContext context) => -{ - // Handle manually -}); -``` - -### Step 3: CRITICAL — Anti-Forgery Auto-Validates Form Uploads in .NET 8+ - -```csharp -// CRITICAL: In .NET 8+ with UseAntiforgery(), ALL form-bound endpoints -// automatically validate anti-forgery tokens, INCLUDING file uploads - -builder.Services.AddAntiforgery(); -var app = builder.Build(); -app.UseAntiforgery(); - -// This endpoint now REQUIRES an anti-forgery token: -app.MapPost("/upload", (IFormFile file) => Results.Ok(file.FileName)); -// Without the token → 400 Bad Request - -// CRITICAL: For API-only file uploads (no anti-forgery needed), opt out: -app.MapPost("/api/upload", (IFormFile file) => Results.Ok(file.FileName)) - .DisableAntiforgery(); // CRITICAL: Must explicitly opt out - -// COMMON MISTAKE: Getting 400 errors on file uploads and not realizing -// it's because UseAntiforgery() is in the pipeline - -// WARNING: DisableAntiforgery() is safe for unauthenticated endpoints and -// endpoints using JWT bearer authentication. However, for endpoints -// authenticated with cookies, disabling antiforgery removes CSRF protection -// and exposes the endpoint to cross-site request forgery attacks. -// For cookie-authenticated endpoints, include a valid antiforgery token instead. -``` - -### Step 4: CRITICAL — Validate File Content, Not Just Extension - -```csharp -app.MapPost("/upload", async (IFormFile file) => -{ - // CRITICAL: Check content type AND file signature (magic bytes) - // NEVER trust file extension alone — it can be spoofed - - // Allow only JPEG/PNG by default. To support more (e.g., GIF), - // add the MIME type here AND validate its magic bytes below. - var allowedTypes = new[] { "image/jpeg", "image/png" }; - if (!allowedTypes.Contains(file.ContentType, StringComparer.OrdinalIgnoreCase)) - return Results.BadRequest("File type not allowed"); - - // CRITICAL: Check magic bytes for file type verification - using var stream = file.OpenReadStream(); - var header = new byte[8]; - var bytesRead = await stream.ReadAsync(header, 0, header.Length); - if (bytesRead < 4) - return Results.BadRequest("File content is too short or invalid"); - - // JPEG: FF D8 FF - // PNG: 89 50 4E 47 - var isJpeg = header[0] == 0xFF && header[1] == 0xD8 && header[2] == 0xFF; - var isPng = header[0] == 0x89 && header[1] == 0x50 && header[2] == 0x4E && header[3] == 0x47; - - // Determine the actual content type from magic bytes - string? detectedContentType = isJpeg ? "image/jpeg" : isPng ? "image/png" : null; - if (detectedContentType is null) - return Results.BadRequest("File content is not a supported image format (only JPEG and PNG are allowed)."); - - // Ensure the declared Content-Type matches what the magic bytes detected - if (!string.Equals(file.ContentType, detectedContentType, StringComparison.OrdinalIgnoreCase)) - return Results.BadRequest("File content type does not match the declared ContentType header."); - - // CRITICAL: Never use the user-provided filename directly for the save path — it can - // contain path traversal characters (e.g., "../../../etc/passwd"). - // Generate a safe filename; derive the extension from validated content, not user input. - var extension = detectedContentType == "image/jpeg" ? ".jpg" : ".png"; - var safeFileName = $"{Guid.NewGuid()}{extension}"; - // NEVER: var path = Path.Combine("uploads", file.FileName); // Path traversal! - - var filePath = Path.Combine("uploads", safeFileName); - Directory.CreateDirectory("uploads"); - stream.Position = 0; - using var fileStream = File.Create(filePath); - await stream.CopyToAsync(fileStream); - - return Results.Ok(new { FileName = safeFileName, file.Length }); -}); -``` - -### Step 5: CRITICAL — Streaming Large Files Without Buffering - -```csharp -// CRITICAL: IFormFile relies on multipart form parsing that buffers content in memory -// (up to a threshold) then spills to temp files on disk. For very large uploads, -// this overhead is unnecessary if you can process the data in chunks. -// Use MultipartReader to stream directly — e.g., to a final storage location — -// without buffering the entire file first. - -app.MapPost("/upload-stream", - [DisableRequestSizeLimit] - async (HttpContext context) => -{ - // Extract the multipart boundary from the Content-Type header - var contentType = context.Request.ContentType; - if (contentType == null) - return Results.BadRequest("Missing Content-Type"); - - // Safely parse the Content-Type header to avoid FormatException from MediaTypeHeaderValue.Parse - if (!MediaTypeHeaderValue.TryParse(contentType, out var mediaType)) - return Results.BadRequest("Invalid Content-Type"); - - var boundary = HeaderUtilities.RemoveQuotes(mediaType.Boundary).Value; - if (string.IsNullOrWhiteSpace(boundary)) - return Results.BadRequest("Not a multipart request"); - - var reader = new MultipartReader(boundary, context.Request.Body); - - // CRITICAL: ReadNextSectionAsync returns null when there are no more sections - while (await reader.ReadNextSectionAsync() is { } section) - { - // Parse Content-Disposition to identify file sections - if (!ContentDispositionHeaderValue.TryParse(section.ContentDisposition, out var contentDisposition)) - continue; - - if (contentDisposition.DispositionType.Equals("form-data") - && !string.IsNullOrEmpty(contentDisposition.FileName.Value)) - { - // Sanitize the user-provided filename to prevent path traversal - var originalFileName = contentDisposition.FileName.Value ?? string.Empty; - var sanitizedFileName = Path.GetFileName(originalFileName.Trim('"')); - var safeFile = $"{Guid.NewGuid()}"; - - // CRITICAL: Stream directly to disk — avoids buffering in memory - Directory.CreateDirectory("uploads"); - using var fileStream = File.Create(Path.Combine("uploads", safeFile)); - await section.Body.CopyToAsync(fileStream); - } - } - - return Results.Ok("Uploaded"); -}).DisableAntiforgery(); - -// COMMON MISTAKE: Using IFormFile for very large files -// Multipart form parsing can buffer large uploads and consume memory/disk. -// Use MultipartReader for streaming directly to storage. -``` - -## Common Mistakes - -1. **Only configuring one size limit**: Must configure BOTH Kestrel `MaxRequestBodySize` AND `FormOptions.MultipartBodyLengthLimit`. -2. **400 errors from anti-forgery**: In .NET 8+, `UseAntiforgery()` auto-validates form uploads. Use `.DisableAntiforgery()` for API endpoints (safe for JWT/unauthenticated; do NOT disable for cookie-authenticated endpoints). -3. **Trusting file.FileName**: User-provided filename can contain path traversal. Generate a safe filename with `Guid.NewGuid()` and derive the extension from validated content. -4. **Trusting Content-Type only**: Content type is client-spoofable. Always check magic bytes for actual file type verification. -5. **Using IFormFile for very large files**: Multipart form parsing buffers with a memory threshold and spills to temp files. Use `MultipartReader` to stream data in chunks directly to storage without buffering the entire file. -6. **Deriving file extension from user input**: Prefer deriving the extension from the validated content type or magic bytes rather than `Path.GetExtension(file.FileName)`. If the original extension must be preserved, validate it against the detected content type. diff --git a/catalog/Frameworks/Official-DotNet-ASPNetCore/skills/minimal-api-file-upload/manifest.json b/catalog/Frameworks/Official-DotNet-ASPNetCore/skills/minimal-api-file-upload/manifest.json deleted file mode 100644 index c019b3bf..00000000 --- a/catalog/Frameworks/Official-DotNet-ASPNetCore/skills/minimal-api-file-upload/manifest.json +++ /dev/null @@ -1,6 +0,0 @@ -{ - "version": "0.1.1", - "category": "Web", - "compatibility": "Requires an ASP.NET Core project or solution.", - "package_prefix": "Microsoft.AspNetCore" -} diff --git a/external-sources/upstreams/astro/packages/astro/package.json b/external-sources/upstreams/astro/packages/astro/package.json index a0461ca9..1a4e1966 100644 --- a/external-sources/upstreams/astro/packages/astro/package.json +++ b/external-sources/upstreams/astro/packages/astro/package.json @@ -141,7 +141,7 @@ "devalue": "^5.8.1", "diff": "^9.0.0", "dset": "^3.1.4", - "es-module-lexer": "^2.0.0", + "es-module-lexer": "^3.0.2", "esbuild": "^0.28.0", "find-proc": "0.2.0", "flattie": "^1.1.1", @@ -150,7 +150,6 @@ "github-slugger": "^2.0.0", "html-escaper": "3.0.3", "http-cache-semantics": "^4.2.0", - "js-yaml": "^4.3.2", "jsonc-parser": "^3.3.1", "magic-string": "^1.0.0", "magicast": "^0.5.2", @@ -197,7 +196,6 @@ "@types/aria-query": "^5.0.4", "@types/html-escaper": "3.0.4", "@types/http-cache-semantics": "^4.2.0", - "@types/js-yaml": "^4.0.9", "@types/parse-srcset": "^1.0.0", "@types/picomatch": "^4.0.2", "@types/yargs-parser": "^21.0.3", @@ -215,7 +213,7 @@ "remark-code-titles": "^0.1.2", "sass": "^1.98.0", "typescript": "^6.0.3", - "undici": "^8.11.0", + "undici": "7.29.0", "vitest": "^4.1.11" }, "engines": { diff --git a/external-sources/upstreams/dotnet-skills/dotnet-aspnetcore/skills/configuring-opentelemetry-dotnet/SKILL.md b/external-sources/upstreams/dotnet-skills/dotnet-aspnetcore/skills/configuring-opentelemetry-dotnet/SKILL.md deleted file mode 100644 index c99fe5c5..00000000 --- a/external-sources/upstreams/dotnet-skills/dotnet-aspnetcore/skills/configuring-opentelemetry-dotnet/SKILL.md +++ /dev/null @@ -1,289 +0,0 @@ ---- -name: configuring-opentelemetry-dotnet -description: Configure OpenTelemetry distributed tracing, metrics, and logging in ASP.NET Core using the .NET OpenTelemetry SDK. Use when adding observability, setting up OTLP exporters, creating custom metrics/spans, or troubleshooting distributed trace correlation. -license: MIT ---- - -# Configuring OpenTelemetry in .NET - -## When to Use - -- Adding distributed tracing to an ASP.NET Core application -- Setting up OpenTelemetry exporters (OTLP is the primary protocol; Jaeger accepts OTLP natively; Prometheus OTLP ingestion requires explicit opt-in) -- Creating custom metrics or trace spans for business operations -- Troubleshooting distributed trace context propagation across services - -## When Not to Use - -- The user wants application-level logging only (use ILogger, Serilog) -- The user is using Application Insights SDK directly (different API) -- The user needs APM with a commercial vendor's proprietary SDK - -## Inputs - -| Input | Required | Description | -|-------|----------|-------------| -| ASP.NET Core project | Yes | The application to instrument | -| Observability backend | No | Where to export: OTLP collector, Aspire dashboard, Jaeger (accepts OTLP natively) | - -## Workflow - -### Step 1: Install the correct packages - -**There are many OpenTelemetry NuGet packages. Install exactly these:** - -```bash -# Core SDK + ASP.NET Core instrumentation + logging integration -dotnet add package OpenTelemetry.Extensions.Hosting -dotnet add package OpenTelemetry.Instrumentation.AspNetCore -dotnet add package OpenTelemetry.Instrumentation.Http - -# Exporter -dotnet add package OpenTelemetry.Exporter.OpenTelemetryProtocol # OTLP exporter for traces, metrics, AND logs - -# Optional — dev/local debugging only (do NOT include in production deployments) -# dotnet add package OpenTelemetry.Exporter.Console -``` - -**Do NOT install `OpenTelemetry` alone** — you need `OpenTelemetry.Extensions.Hosting` for proper DI integration. - -#### Optional: additional auto-instrumentation packages - -Install only the packages that match the libraries your application uses: - -```bash -dotnet add package OpenTelemetry.Instrumentation.SqlClient # SQL Server queries -dotnet add package OpenTelemetry.Instrumentation.EntityFrameworkCore # EF Core -dotnet add package OpenTelemetry.Instrumentation.GrpcNetClient # gRPC calls -dotnet add package OpenTelemetry.Instrumentation.Runtime # GC, thread pool metrics -``` - -### Step 2: Configure all signals in Program.cs - -```csharp -using OpenTelemetry.Resources; -using OpenTelemetry.Trace; -using OpenTelemetry.Metrics; -using OpenTelemetry.Logs; - -var builder = WebApplication.CreateBuilder(args); - -builder.Services.AddOpenTelemetry() - .ConfigureResource(resource => resource - .AddService(serviceName: builder.Environment.ApplicationName)) - .WithTracing(tracing => tracing - .AddAspNetCoreInstrumentation(options => - { - // Filter out health check endpoints from traces - options.Filter = httpContext => - !httpContext.Request.Path.StartsWithSegments("/healthz"); - }) - .AddHttpClientInstrumentation(options => - { - options.RecordException = true; - }) - // Optional: add SQL instrumentation if using SqlClient directly - // .AddSqlClientInstrumentation(options => - // { - // options.SetDbStatementForText = true; - // options.RecordException = true; - // }) - // Custom activity sources (must match ActivitySource names in your code) - .AddSource("MyApp.Orders") - .AddSource("MyApp.Payments") - .AddSource("MyApp.Messaging")) - .WithMetrics(metrics => metrics - .AddAspNetCoreInstrumentation() - .AddHttpClientInstrumentation() - // Optional: .AddRuntimeInstrumentation() for GC and thread pool metrics - // (requires OpenTelemetry.Instrumentation.Runtime package) - // Custom meters (must match Meter names in your code) - .AddMeter("MyApp.Metrics")) - .WithLogging(logging => - { - logging.IncludeScopes = true; - // logging.IncludeFormattedMessage = true; // Enable if you need the formatted message string in log exports - }) - // Single OTLP exporter for all signals — reads OTEL_EXPORTER_OTLP_ENDPOINT - // env var (defaults to http://localhost:4317). Override via environment variable - // or appsettings.json configuration. - .UseOtlpExporter(); -``` - -### Step 3: Understanding log–trace correlation - -The `.WithLogging()` call in Step 2 integrates ILogger with OpenTelemetry: - -- Each log entry automatically includes TraceId and SpanId for correlation with traces -- The service resource from `.ConfigureResource()` propagates to logs automatically -- `UseOtlpExporter()` applies to logs alongside traces and metrics -- No additional packages or separate `SetResourceBuilder` call needed - -### Step 4: Create custom spans (Activities) for business operations - -```csharp -using System.Diagnostics; -using Microsoft.Extensions.Logging; - -public class OrderService -{ - // Create an ActivitySource matching what you registered in Step 2 - private static readonly ActivitySource ActivitySource = new("MyApp.Orders"); - private readonly ILogger _logger; - - public OrderService(ILogger logger) => _logger = logger; - - public async Task ProcessOrderAsync(CreateOrderRequest request) - { - // Start a new span - using var activity = ActivitySource.StartActivity("ProcessOrder"); - - // Add attributes (tags) to the span - activity?.SetTag("order.customer_id", request.CustomerId); - activity?.SetTag("order.item_count", request.Items.Count); - - try - { - // Child span for validation - using (var validationActivity = ActivitySource.StartActivity("ValidateOrder")) - { - await ValidateOrderAsync(request); - validationActivity?.SetTag("validation.result", "passed"); - } - - // Child span for payment - using (var paymentActivity = ActivitySource.StartActivity("ProcessPayment", - ActivityKind.Client)) // Client = outgoing call - { - paymentActivity?.SetTag("payment.method", request.PaymentMethod); - await ProcessPaymentAsync(request); - } - - var order = new Order { Id = Guid.NewGuid(), CustomerId = request.CustomerId, Status = "Completed" }; - - activity?.SetTag("order.status", "completed"); - activity?.SetStatus(ActivityStatusCode.Ok); - - return order; - } - catch (Exception ex) - { - activity?.SetStatus(ActivityStatusCode.Error, ex.Message); - // Log via ILogger — OpenTelemetry captures this with trace correlation. - // Prefer logging over activity.RecordException() as OTel is deprecating - // span events for exception recording in favor of log-based exceptions. - _logger.LogError(ex, "Order processing failed for customer {CustomerId}", request.CustomerId); - throw; - } - } -} -``` - -**Critical: `ActivitySource` name must match `AddSource("...")` in configuration.** Unmatched sources are silently ignored — this is the #1 debugging issue. - -### Step 5: Create custom metrics - -Use `IMeterFactory` (injected via DI) to create meters — this ensures proper lifetime management and testability. - -```csharp -using System.Diagnostics; -using System.Diagnostics.Metrics; - -public class OrderMetrics -{ - private readonly Counter _ordersProcessed; - private readonly Histogram _orderProcessingDuration; - private readonly UpDownCounter _activeOrders; - - public OrderMetrics(IMeterFactory meterFactory) - { - // Meter name must match AddMeter("...") in configuration - var meter = meterFactory.Create("MyApp.Metrics"); - - // Counter — use for things that only go up - _ordersProcessed = meter.CreateCounter( - "orders.processed", "orders", "Total orders successfully processed"); - - // Histogram — use for measuring distributions (latency, sizes) - _orderProcessingDuration = meter.CreateHistogram( - "orders.processing_duration", "ms", "Time to process an order"); - - // UpDownCounter — use for things that go up AND down - _activeOrders = meter.CreateUpDownCounter( - "orders.active", "orders", "Currently processing orders"); - } - - public void RecordOrderProcessed(string region, double durationMs) - { - // Tags enable dimensional filtering (by region, status, etc.) - var tags = new TagList - { - { "region", region }, - { "order.type", "standard" } - }; - - _ordersProcessed.Add(1, tags); - _orderProcessingDuration.Record(durationMs, tags); - } -} -``` - -Register `OrderMetrics` in DI: - -```csharp -builder.Services.AddSingleton(); -``` - -### Step 6: Configure context propagation for distributed scenarios - -Trace context propagation is automatic for HTTP calls when using `AddHttpClientInstrumentation()`. For non-HTTP scenarios: - -```csharp -using System; -using System.Collections.Generic; -using System.Diagnostics; -using OpenTelemetry.Context.Propagation; - -// ActivitySource should be static — register via .AddSource("MyApp.Messaging") in Step 2 -private static readonly ActivitySource MessageSource = new("MyApp.Messaging"); - -// Manual context propagation (e.g., across message queues) -// On the SENDING side: -var propagator = Propagators.DefaultTextMapPropagator; -var activityContext = Activity.Current?.Context ?? default; -var context = new PropagationContext(activityContext, Baggage.Current); -var carrier = new Dictionary(); - -propagator.Inject(context, carrier, (dict, key, value) => dict[key] = value); -// Send carrier dictionary as message headers - -// On the RECEIVING side: -var parentContext = propagator.Extract(default, carrier, - (dict, key) => dict.TryGetValue(key, out var value) ? new[] { value } : Array.Empty()); - -Baggage.Current = parentContext.Baggage; -using var activity = MessageSource.StartActivity("ProcessMessage", - ActivityKind.Consumer, - parentContext.ActivityContext); // Links to parent trace! -``` - -## Validation - -- [ ] Traces appear in the observability backend (Jaeger, Aspire dashboard, etc.) -- [ ] HTTP requests automatically create spans with correct verb, URL, status code -- [ ] Custom `ActivitySource` names match `AddSource()` registrations -- [ ] Custom `Meter` names match `AddMeter()` registrations -- [ ] Logs include TraceId and SpanId for correlation -- [ ] Health check endpoints are filtered from traces -- [ ] Exception details appear on error spans - -## Common Pitfalls - -| Pitfall | Solution | -|---------|----------| -| `ActivitySource.StartActivity` returns null | Source name doesn't match any `AddSource()` — names must match exactly | -| Traces not appearing in exporter | Check OTLP endpoint: gRPC uses port 4317, HTTP uses 4318 | -| Missing HTTP client spans | Ensure `AddHttpClientInstrumentation()` is registered; it works for both `IHttpClientFactory`/DI and `new HttpClient()` (use `IHttpClientFactory` for lifetime management) | -| High cardinality tags | Don't use user IDs, request IDs, or UUIDs as metric tags — explodes storage | -| OTLP gRPC vs HTTP mismatch | Default is gRPC (port 4317); if collector only accepts HTTP, set `OtlpExportProtocol.HttpProtobuf` | -| `Meter` / `ActivitySource` lifecycle | `ActivitySource` should be static; create `Meter` via `IMeterFactory` from DI (not `new Meter()`) for proper lifetime management and testability | diff --git a/external-sources/upstreams/dotnet-skills/dotnet-aspnetcore/skills/minimal-api-file-upload/SKILL.md b/external-sources/upstreams/dotnet-skills/dotnet-aspnetcore/skills/minimal-api-file-upload/SKILL.md deleted file mode 100644 index 6ec5fc80..00000000 --- a/external-sources/upstreams/dotnet-skills/dotnet-aspnetcore/skills/minimal-api-file-upload/SKILL.md +++ /dev/null @@ -1,235 +0,0 @@ ---- -name: minimal-api-file-upload -description: File upload endpoints in ASP.NET minimal APIs (.NET 8+) -license: MIT ---- - -# Implementing File Uploads in ASP.NET Core Minimal APIs - -## When to Use -- File upload endpoints in ASP.NET Core minimal APIs (.NET 8+) -- Handling IFormFile or IFormFileCollection parameters -- When you need size limits, content type validation, or streaming large files - -## When Not to Use -- MVC controllers → `[FromForm] IFormFile` works directly with attributes -- Simple JSON body → no file upload needed -- Very large files (> 1GB) → use streaming with `MultipartReader` instead - -## Inputs - -| Input | Required | Description | -|-------|----------|-------------| -| File parameter(s) | Yes | IFormFile or IFormFileCollection | -| Size limits | Yes | Max file/request size | -| Allowed types | No | Content type or extension restrictions | - -## Workflow - -### Step 1: CRITICAL — Understand IFormFile Binding in Minimal APIs - -```csharp -// In .NET 8+ minimal APIs, IFormFile binds automatically from multipart/form-data -// when it is the only complex parameter. -app.MapPost("/upload", (IFormFile file) => ...); - -// CRITICAL: When you mix files with other form fields, use [FromForm] on all -// form-bound parameters (or group them into a single [FromForm] DTO). -app.MapPost("/upload-with-metadata", - ([FromForm] IFormFile file, [FromForm] string description) => -{ - return Results.Ok(new { file.FileName, Description = description }); -}); - -// Multiple files: IFormFileCollection also binds automatically from multipart/form-data. -// You only need [FromForm] if you mix it with other form fields, as shown above. -app.MapPost("/upload-multiple", (IFormFileCollection files) => -{ - return Results.Ok(files.Select(f => new { f.FileName, f.Length })); -}); -``` - -### Step 2: CRITICAL — File Size Limits Are Separate from Request Size Limits - -```csharp -// CRITICAL: There are TWO different size limits and you need to configure BOTH - -// 1. Request body size limit (Kestrel level) — default is 30MB -builder.WebHost.ConfigureKestrel(options => -{ - options.Limits.MaxRequestBodySize = 10 * 1024 * 1024; // 10 MB -}); - -// 2. Form options — multipart body length limit — default is 128MB -builder.Services.Configure(options => -{ - options.MultipartBodyLengthLimit = 10 * 1024 * 1024; // 10 MB - options.ValueLengthLimit = 1024 * 1024; // 1 MB for form values - options.MultipartHeadersLengthLimit = 16384; // 16 KB for section headers -}); - -// COMMON MISTAKE: Only increasing Kestrel MaxRequestBodySize -// upload still fails because FormOptions.MultipartBodyLengthLimit is exceeded - -// COMMON MISTAKE: Only increasing FormOptions -// upload fails with "Request body too large" from Kestrel before reaching form parsing - -// CRITICAL: Per-endpoint override with RequestSizeLimit attribute -app.MapPost("/upload-large", [RequestSizeLimit(200_000_000)] (IFormFile file) => -{ - return Results.Ok(new { file.FileName, file.Length }); -}); - -// CRITICAL: To disable the limit entirely (for streaming): -app.MapPost("/upload-unlimited", [DisableRequestSizeLimit] async (HttpContext context) => -{ - // Handle manually -}); -``` - -### Step 3: CRITICAL — Anti-Forgery Auto-Validates Form Uploads in .NET 8+ - -```csharp -// CRITICAL: In .NET 8+ with UseAntiforgery(), ALL form-bound endpoints -// automatically validate anti-forgery tokens, INCLUDING file uploads - -builder.Services.AddAntiforgery(); -var app = builder.Build(); -app.UseAntiforgery(); - -// This endpoint now REQUIRES an anti-forgery token: -app.MapPost("/upload", (IFormFile file) => Results.Ok(file.FileName)); -// Without the token → 400 Bad Request - -// CRITICAL: For API-only file uploads (no anti-forgery needed), opt out: -app.MapPost("/api/upload", (IFormFile file) => Results.Ok(file.FileName)) - .DisableAntiforgery(); // CRITICAL: Must explicitly opt out - -// COMMON MISTAKE: Getting 400 errors on file uploads and not realizing -// it's because UseAntiforgery() is in the pipeline - -// WARNING: DisableAntiforgery() is safe for unauthenticated endpoints and -// endpoints using JWT bearer authentication. However, for endpoints -// authenticated with cookies, disabling antiforgery removes CSRF protection -// and exposes the endpoint to cross-site request forgery attacks. -// For cookie-authenticated endpoints, include a valid antiforgery token instead. -``` - -### Step 4: CRITICAL — Validate File Content, Not Just Extension - -```csharp -app.MapPost("/upload", async (IFormFile file) => -{ - // CRITICAL: Check content type AND file signature (magic bytes) - // NEVER trust file extension alone — it can be spoofed - - // Allow only JPEG/PNG by default. To support more (e.g., GIF), - // add the MIME type here AND validate its magic bytes below. - var allowedTypes = new[] { "image/jpeg", "image/png" }; - if (!allowedTypes.Contains(file.ContentType, StringComparer.OrdinalIgnoreCase)) - return Results.BadRequest("File type not allowed"); - - // CRITICAL: Check magic bytes for file type verification - using var stream = file.OpenReadStream(); - var header = new byte[8]; - var bytesRead = await stream.ReadAsync(header, 0, header.Length); - if (bytesRead < 4) - return Results.BadRequest("File content is too short or invalid"); - - // JPEG: FF D8 FF - // PNG: 89 50 4E 47 - var isJpeg = header[0] == 0xFF && header[1] == 0xD8 && header[2] == 0xFF; - var isPng = header[0] == 0x89 && header[1] == 0x50 && header[2] == 0x4E && header[3] == 0x47; - - // Determine the actual content type from magic bytes - string? detectedContentType = isJpeg ? "image/jpeg" : isPng ? "image/png" : null; - if (detectedContentType is null) - return Results.BadRequest("File content is not a supported image format (only JPEG and PNG are allowed)."); - - // Ensure the declared Content-Type matches what the magic bytes detected - if (!string.Equals(file.ContentType, detectedContentType, StringComparison.OrdinalIgnoreCase)) - return Results.BadRequest("File content type does not match the declared ContentType header."); - - // CRITICAL: Never use the user-provided filename directly for the save path — it can - // contain path traversal characters (e.g., "../../../etc/passwd"). - // Generate a safe filename; derive the extension from validated content, not user input. - var extension = detectedContentType == "image/jpeg" ? ".jpg" : ".png"; - var safeFileName = $"{Guid.NewGuid()}{extension}"; - // NEVER: var path = Path.Combine("uploads", file.FileName); // Path traversal! - - var filePath = Path.Combine("uploads", safeFileName); - Directory.CreateDirectory("uploads"); - stream.Position = 0; - using var fileStream = File.Create(filePath); - await stream.CopyToAsync(fileStream); - - return Results.Ok(new { FileName = safeFileName, file.Length }); -}); -``` - -### Step 5: CRITICAL — Streaming Large Files Without Buffering - -```csharp -// CRITICAL: IFormFile relies on multipart form parsing that buffers content in memory -// (up to a threshold) then spills to temp files on disk. For very large uploads, -// this overhead is unnecessary if you can process the data in chunks. -// Use MultipartReader to stream directly — e.g., to a final storage location — -// without buffering the entire file first. - -app.MapPost("/upload-stream", - [DisableRequestSizeLimit] - async (HttpContext context) => -{ - // Extract the multipart boundary from the Content-Type header - var contentType = context.Request.ContentType; - if (contentType == null) - return Results.BadRequest("Missing Content-Type"); - - // Safely parse the Content-Type header to avoid FormatException from MediaTypeHeaderValue.Parse - if (!MediaTypeHeaderValue.TryParse(contentType, out var mediaType)) - return Results.BadRequest("Invalid Content-Type"); - - var boundary = HeaderUtilities.RemoveQuotes(mediaType.Boundary).Value; - if (string.IsNullOrWhiteSpace(boundary)) - return Results.BadRequest("Not a multipart request"); - - var reader = new MultipartReader(boundary, context.Request.Body); - - // CRITICAL: ReadNextSectionAsync returns null when there are no more sections - while (await reader.ReadNextSectionAsync() is { } section) - { - // Parse Content-Disposition to identify file sections - if (!ContentDispositionHeaderValue.TryParse(section.ContentDisposition, out var contentDisposition)) - continue; - - if (contentDisposition.DispositionType.Equals("form-data") - && !string.IsNullOrEmpty(contentDisposition.FileName.Value)) - { - // Sanitize the user-provided filename to prevent path traversal - var originalFileName = contentDisposition.FileName.Value ?? string.Empty; - var sanitizedFileName = Path.GetFileName(originalFileName.Trim('"')); - var safeFile = $"{Guid.NewGuid()}"; - - // CRITICAL: Stream directly to disk — avoids buffering in memory - Directory.CreateDirectory("uploads"); - using var fileStream = File.Create(Path.Combine("uploads", safeFile)); - await section.Body.CopyToAsync(fileStream); - } - } - - return Results.Ok("Uploaded"); -}).DisableAntiforgery(); - -// COMMON MISTAKE: Using IFormFile for very large files -// Multipart form parsing can buffer large uploads and consume memory/disk. -// Use MultipartReader for streaming directly to storage. -``` - -## Common Mistakes - -1. **Only configuring one size limit**: Must configure BOTH Kestrel `MaxRequestBodySize` AND `FormOptions.MultipartBodyLengthLimit`. -2. **400 errors from anti-forgery**: In .NET 8+, `UseAntiforgery()` auto-validates form uploads. Use `.DisableAntiforgery()` for API endpoints (safe for JWT/unauthenticated; do NOT disable for cookie-authenticated endpoints). -3. **Trusting file.FileName**: User-provided filename can contain path traversal. Generate a safe filename with `Guid.NewGuid()` and derive the extension from validated content. -4. **Trusting Content-Type only**: Content type is client-spoofable. Always check magic bytes for actual file type verification. -5. **Using IFormFile for very large files**: Multipart form parsing buffers with a memory threshold and spills to temp files. Use `MultipartReader` to stream data in chunks directly to storage without buffering the entire file. -6. **Deriving file extension from user input**: Prefer deriving the extension from the validated content type or magic bytes rather than `Path.GetExtension(file.FileName)`. If the original extension must be preserved, validate it against the detected content type. diff --git a/external-sources/vendir.lock.yml b/external-sources/vendir.lock.yml index 4bb625b8..334e708f 100644 --- a/external-sources/vendir.lock.yml +++ b/external-sources/vendir.lock.yml @@ -2,20 +2,21 @@ apiVersion: vendir.k14s.io/v1alpha1 directories: - contents: - git: - commitTitle: Add reusable build failure analysis workflow (#1217)... - sha: a55fbf42c36a37b94ec07291bd79cb6b6bf3a04d + commitTitle: Remove weak dotnet-aspnetcore skills (#1210)... + sha: 8599a06757aabf411ae28e0559063342d70386ef tags: - - skill-validator-nightly-1-ga55fbf4 + - skill-validator-nightly-1-g8599a06 path: dotnet-skills - git: commitTitle: Add Cursor rules that reference the existing skill docs... sha: af2319bd01bb7cc881267a9ef42cafdaf5e9029d path: webgpu-claude-skill - git: - commitTitle: Update dependency @netlify/functions to v6 (#18127) - sha: 3f3d580b83b0cb9f14d451b86dab547ccffe4eab + commitTitle: Add .ts and .js to server environment optimizeDeps entries glob + (#18133)... + sha: faac481dc86efdd2e4987069a7298f2aea3f1c6c tags: - - astro@7.3.5-7-g3f3d580b83 + - astro@7.3.5-26-gfaac481dc8 path: astro path: upstreams kind: LockConfig