[Feedback request] Experimental Python 3.14 OCI images with thread-based multi-concurrency for Lambda Managed Instances

[vip-amzn]

⚠️ Experimental - not for production use. These images are provided for feedback and testing only. They should not be used for production workloads. Functions based on these images are not covered by AWS Technical Support and are not eligible for the Lambda SLA. These images may be modified or discontinued without notice.

We have published experimental Python OCI images that add thread-based multi-concurrency as an opt-in alternative to the default process-based model on Lambda Managed Instances (LMI). These images let you choose how your function handles concurrent requests: via processes, threads, or a hybrid of both.

Note: Thread and hybrid concurrency modes require Lambda Managed Instances. On standard (on-demand) Lambda, multi-concurrency is not supported. The image behaves identically to the production-ready python:3.14 image.

Images available now:

public.ecr.aws/lambda/python:3.14-experimental            # multi-arch manifest
public.ecr.aws/lambda/python:3.14-experimental-x86_64
public.ecr.aws/lambda/python:3.14-experimental-arm64

---

Why thread-based concurrency?

The default process model on LMI (documented at Python runtime for Lambda Managed Instances) spawns N separate Python processes. This is safe and simple, but has limitations:

Problem	Impact
Memory multiplication	N processes × handler memory (e.g., 16 × 300MB = 4.8GB)
Duplicate initialization	Each process imports handler independently
No shared in-memory state	Caches, models, connections cannot be shared between requests

Thread mode addresses these by running N concurrent requests as threads within a single process, sharing memory, handler code, and any module-level state.

---

What's different from the production-ready python:3.14 image?

Aspect	Production-ready python:3.14	Experimental python:3.14-experimental
Production-ready?	Yes	No - experimental, not for production workloads
Concurrency model	Process-only (N processes × 1 thread)	Process, Thread, or Hybrid (configurable)
AWS_LAMBDA_CONCURRENCY_MODE env var	Not recognized	Accepts process / thread / hybrid
Pre-fork hooks (@register_pre_fork)	Not available	Available for process/hybrid modes
Default behavior (no env var set)	N processes × 1 thread	Identical to production-ready image
Python binary	Python 3.14 (standard, GIL-enabled)	Python 3.14 (standard, GIL-enabled)

Important: If you do NOT set AWS_LAMBDA_CONCURRENCY_MODE, the experimental image behaves exactly like the production-ready image. Thread mode is strictly opt-in.

Things to know before opting in (thread and hybrid modes)

Your code needs to be thread-safe

Thread and hybrid modes run concurrent requests as threads within a single process. This means all threads share memory, module-level objects, and file descriptors. Code that works correctly in process mode (where each request gets its own isolated process) may need changes:

• Shared mutable state: All threads share process memory. Patterns like shared boto3.Session(), shared boto3.resource(), requests.Session with header mutation, writing to /tmp with fixed filenames, and global_counter += 1 will break. Use threading.local(), threading.Lock(), or per-request file paths as appropriate.
• Database connections: Connections and sockets opened at module level are shared across threads. Use connection pools or per-thread connections. (Pre-created boto3.client() operations are safe; the client handles concurrency internally.)

Known platform limitations

The following observability features do not fully work in thread/hybrid modes yet. These are temporary limitations of the experimental image. The production-ready release will include full trace propagation and recursion detection support.

• Trace ID propagation: Downstream AWS service calls won't carry your Lambda invocation's trace ID. Traces appear disconnected in X-Ray.
• Recursion detection: The platform can't detect and stop recursive loops (e.g., Lambda → SQS → Lambda). Use process mode for these workloads.
• Custom segment generation (Application Signals / ADOT): The one-click Application Signals console feature, ADOT auto-instrumentation, and custom X-Ray subsegments won't produce connected traces.

---

How to use it

1. Pull the experimental image

FROM public.ecr.aws/lambda/python:3.14-experimental

COPY requirements.txt .
RUN pip install -r requirements.txt

COPY app.py ${LAMBDA_TASK_ROOT}
CMD ["app.handler"]

2. Configure concurrency mode

N is the maximum number of concurrent requests routed to a single execution environment, set by your function's PerExecutionEnvironmentMaxConcurrency configuration.

Set the AWS_LAMBDA_CONCURRENCY_MODE environment variable on your Lambda function or in your Dockerfile (ENV AWS_LAMBDA_CONCURRENCY_MODE=thread):

Mode	Environment Variable	What happens
Process (default)	AWS_LAMBDA_CONCURRENCY_MODE=process	N processes × 1 thread each. Identical to production-ready image.
Thread	AWS_LAMBDA_CONCURRENCY_MODE=thread	1 process × N threads. All threads share memory and handler.
Hybrid	AWS_LAMBDA_CONCURRENCY_MODE=hybrid	K processes × M threads. K = allocated vCPUs, M = floor(N / K).

Note (hybrid mode): M is calculated as floor(N / K). If N doesn't divide evenly by K, some concurrency slots may be unused. For example, N=16 on 3 vCPUs → 3 processes × 5 threads = 15 concurrent handlers (1 slot unused).

3. Example: Thread mode with shared S3 download

import boto3
import threading
import json
import os

# Module-level initialization runs ONCE (before threads start)
# This is the key advantage: expensive setup happens once, shared by all threads

s3 = boto3.client('s3')  # Pre-created client is thread-safe for operations
_lock = threading.Lock()
_shared_data = None

def _load_data():
    """Download large file from S3 once, share across all threads."""
    global _shared_data
    if _shared_data is None:
        with _lock:
            if _shared_data is None:  # Double-checked locking
                response = s3.get_object(Bucket='my-bucket', Key='large-config.json')
                _shared_data = json.loads(response['Body'].read())
    return _shared_data

def handler(event, context):
    # All concurrent threads share the same _shared_data
    data = _load_data()
    result = process(event, data)
    return {"statusCode": 200, "body": json.dumps(result)}

4. Example: Hybrid mode

Hybrid mode automatically calculates K (processes) from your allocated vCPUs and M (threads per process) from floor(N / K). This gives you a balance of fault isolation and memory sharing.

# On a 4-vCPU instance with N=16 (PerExecutionEnvironmentMaxConcurrency):
# hybrid → 4 processes × 4 threads = 16 concurrent handlers
AWS_LAMBDA_CONCURRENCY_MODE=hybrid

Each process independently loads the handler, then spawns M threads that share that process's memory.

---

Pre-fork hooks (@register_pre_fork)

In process and hybrid modes, each child process re-imports the handler independently (separate INIT per process). Pre-fork hooks let you run setup once in the parent before workers spawn. This is mainly useful for starting inference servers, downloading model files to /tmp, or other one-time setup that all children can benefit from.

import subprocess
from awslambdaric.lambda_concurrency_hooks import register_pre_fork

@register_pre_fork
def start_inference_server():
    """Start model server once; all worker processes send requests to it."""
    subprocess.Popen(["python", "serve.py", "--port", "8000"])

Notes:

• Only fires in process/hybrid modes. In thread mode, use module-level code instead (runs once before threads start).
• In hybrid mode, pre-fork hooks only execute when vCPU count > 1 (otherwise num_processes = 1, which behaves exactly like thread mode).
• Children are spawned with multiprocessing.get_context("spawn"). Each child gets a fresh Python interpreter with no inherited objects or file descriptors. Pre-fork hooks are for setup tasks, not for sharing in-memory state.

---

Compatibility

Environment	Works?	Notes
Lambda Managed Instances (LMI)	✅	Full thread/hybrid support with PerExecutionEnvironmentMaxConcurrency
AWS SAM CLI sam local invoke	✅	Same as production-ready image
Standard Lambda (non-LMI)	⚠️	Image works but multi-concurrency modes (thread/hybrid) are not supported. Behaves identically to python:3.14.

---

We want your feedback!

We'd love to hear from you:

1. What's your use case? (I/O-bound API calls, ML inference, data processing, etc.)
2. Which mode are you using? (thread, hybrid, process)
3. Are you using pre-fork hooks? If so, what for?
4. What libraries are you using with thread mode? Did anything break?
5. Memory/performance comparisons - did thread mode reduce your memory footprint vs process mode?
6. Would Copy-on-Write (CoW) memory sharing between processes be useful for your workload? (Currently, each child process re-imports independently with no shared memory from parent.)
7. Any issues, unexpected behavior, or suggestions for improvement?

Please share your experience in the comments below. Your feedback directly shapes whether and how this ships to the production-ready images.

---

Links

• Lambda Managed Instances overview
• Python runtime for LMI (current process-based docs)
• LMI Core Concepts
• Getting started with LMI
• LMI Scaling
• Understanding the LMI execution environment