Highlights
- Added the Python Runner API for packaging local Python projects and exposing functions, classes, or object instances as Flame services.
- Added
flmrun, a built-in Runner template application used by Runner-managed services. - Fixed Runner package downloads for multi-object-cache deployments by preserving the cache endpoint returned from package upload.
- Reworked the Python SDK around synchronous client APIs, object-cache helpers, service sessions, and the
flamepy.runnermodule. - Simplified the Rust SDK with top-level connection/session helpers, typed task messages, host-service helpers, object-cache helpers, and service macros.
- Added a standalone object cache with versioned object references, pluggable storage, upload/download support, incremental object fetch, and fast paths for tabular/numpy/Arrow data.
- Added
flmadminstallation profiles and support for installing Flame components, SDKs, examples, and multiple Python SDK versions. - Added
flmctl deployand object helper commands for application deployment and object-cache workflows. - Added a Helm chart and Kind-based Kubernetes E2E workflow for static Flame installs on Kubernetes.
- Updated Helm/Kubernetes defaults for filesystem session-manager storage and the
drf/gangpolicy set. - Fixed Flame CLI version metadata so binaries report their package version.
- Added scheduler policy work for priority scheduling, GPU-aware DRF, dynamic policy configuration, gang/batch flows, and resource requirements.
- Added session/runtime reliability improvements including TLS, executor recovery, session bind failure recovery, task watching, node/executor persistence, and notifier-based wakeups.
- Expanded system, E2E, Runner, cache, Python SDK, Rust SDK, and storage test coverage.
Upgrade Notes
- Python SDK packaging is now versioned as
0.6.0, requires Python 3.9 or newer, and advertises Python 3.9 through 3.12. - Runner imports should use
flamepy.runner; olderflamepy.rlnaming was replaced. flmadm installnow requires an explicit profile flag such as--all,--control-plane,--worker,--cache, or--client.- Cluster policy configuration now supports multiple policies through the
policieslist. - The Helm chart defaults now configure session-manager storage as
fs:///var/lib/flame/sessionand enable only thedrfandgangpolicies. - Object cache references are versioned. Clients can use
get_object,update_object,patch_object,upload_object, anddownload_objectinstead of passing raw object payloads through sessions. flmexecsupports explicit runtime selection for scripts, including Python runtime selection throughFLAME_PYTHON_VERSION. Its default Python runtime policy is owned byflmexecrather than the Rust SDK.
Python SDK And Runner
- Added
flamepy.Runnerandflamepy.runnerto package local projects and run Python execution objects remotely (#298, #341, #342). - Added Runner helpers for waiting, selecting, resolving object references, and fetching result objects (#300, #363).
- Added explicit Runner defaults for stateless functions/classes and stateful object instances (#474).
- Fixed Runner packaging for ad hoc scripts and dependency metadata generation (#471).
- Fixed Runner package URL generation to use the returned object-cache endpoint, so executor-manager downloads packages from the cache pod that stored them in multi-cache deployments (#482).
- Added support for Runner dependencies and Python-version selection in Runner/flmexec workflows (#468, #469, #473).
- Added
flmrunas a built-in application template for Runner services (#285). - Added Python service-session helpers and renamed the FlamePy agent API to service sessions (#454).
- Added custom session IDs and
open_sessionenhancements for Python clients and Runner services (#351, #353, #354). - Added
Session.list_tasksand task watching improvements for Python clients (#359, #371). - Aligned
flamepypackage metadata for the 0.6 release, including__version__, Python 3.9+ tooling targets, and Python 3.12 classifier support (#477).
Rust SDK
- Simplified the
flame-rsAPI with direct helpers for connecting, creating/opening sessions, running typed tasks, and writing services (#456). - Added typed service macros and typed task payload support through
FlameMessage(#456). - Added Rust object-cache helpers for putting, getting, updating, patching, uploading, downloading, and deleting objects (#427, #430, #456).
- Kept
flmexecPython runtime default policy local toflmexecand removed that default from the public Rust SDK surface (#475). - Fixed Rust SDK logger initialization when
RUST_LOGis unset (#343).
Object Cache And Storage
- Added the standalone
flame-object-cacheservice and object-cache client helpers (#244, #321). - Added common-data/object-cache integration and object references for shared session state (#258, #259, #269, #296).
- Added LRU eviction and per-application cache behavior (#367, #358).
- Added pluggable cache storage backends, object versioning, and a standalone cache binary (#419, #427).
- Added cache upload/download support with multi-scheme downloaders (#430).
- Added incremental object get, native tabular cache path, and fast-path serialization for numpy/Arrow data (#444, #446, #464).
- Added filesystem, HTTP, and none storage engines for session manager and package/cache workflows (#339, #344, #377, #395).
Scheduling And Runtime Management
- Added batch-session support and gang-related E2E coverage (#401, #407).
- Added priority scheduling and dynamic scheduler policy configuration (#428, #432).
- Added GPU-aware DRF scheduling with resource requirements (#434).
- Added resource requirement support in the priority plugin (#442).
- Added configurable scheduling interval and moved executor limits into
limits(#373, #396). - Added application URL support, installer metadata, and application cache-key improvements (#287, #421, #425).
CLI, Installation, And Local Development
- Added
flmadmfor installation, profile-based component selection, systemd integration, user-local installs, examples, and uninstall flows (#334, #338, #421, #468). - Added
flmctl deployand object helper commands (#459). - Added JSON output for session commands and improved CLI display behavior (#215, #439).
- Changed Flame CLI metadata to derive
--versionoutput from package versions and useXFLOPS <[email protected]>as the author (#488). - Added Podman support and local development helpers (#212).
- Added Docker, compose, and local system-test workflows for multi-component clusters (#345, #347, #466, #470, #472).
Kubernetes And Helm
- Added
charts/flame, a static Helm chart for installing Flame's session manager, executor manager, object cache, client configuration, ServiceAccount, services, persistent volumes, and Helm test resources (#479). - Added chart values and schema coverage for images, service ports, storage, runtime volumes, TLS Secret mounting, client config, component enablement, and static object-cache replica counts (#479).
- Updated chart defaults and Kubernetes E2E overrides to use filesystem session-manager storage and only the
drfandgangscheduler policies (#488). - Added a Kind-based Kubernetes E2E workflow that builds Flame images, installs the Helm chart, runs Helm tests, and verifies
flmctl,flmping, and the Python Pi Runner example from an in-cluster console pod (#479).
Reliability, Recovery, And Observability
- Added TLS support and certificate-generation fixes (#388, #411).
- Added Flame recovery flows and fixed executor state recovery behavior (#386, #403, #465).
- Added node/executor persistence for
flmctland session-manager recovery paths (#383). - Added notifier-based wakeups to reduce busy waiting (#417).
- Added task/event management and task event recording in Rust and Python services (#238, #239, #241, #242).
- Improved executor working-directory validation, stdout/stderr log placement, and cleanup on release (#360, #361, #362).
Examples And Documentation
- Added API reference pages, SDK guides, Runner setup guide, Rust API tutorial, and local-development guide (#451, #461).
- Added and updated examples for Python/Rust Pi, OpenAI agents, LangChain agents, SRA, crawler, Candle, RL, replay buffer, and TorchRL DQN (#264, #299, #424, #441, #450, #453, #462).
- Added the Python Pi example to
flmadm --with-examplesinstalls and made its workload configurable for fast E2E validation (#479). - Added design documents for the major 0.6 feature areas, including Runner, cache, storage, scheduler, TLS, recovery, app installer, Helm installation, and SDK simplification.
- Updated README positioning to "A Distributed Engine for AI" and refreshed project badges (#443).
Testing
- Added Python SDK unit coverage for cache, core, Runner, and service behavior.
- Added Runner storage regression coverage for returned object-cache endpoints and TLS endpoint normalization (#482).
- Added Rust SDK tests for typed services, macros, object cache, and integration flows.
- Added E2E coverage for sessions, cache, Runner,
flmexec, applications, system flows, and scheduler behavior. - Added Kubernetes install coverage for the Helm chart, multi-replica object cache, in-cluster client configuration, and Runner-based Python Pi execution (#479).
- Added release-branch E2E coverage for the Python Runner command path (#485).
- Added daily system tests and aligned the system workflow with the CI job format (#466, #470, #472).
Fixes
- Fixed fair-share allocation and priority-plugin session desired values (#276, #436).
- Fixed cache/session state updates, local instance behavior, task failure reporting, and shutdown queue handling (#214, #235, #243, #315).
- Fixed package/dependency caching through
UV_CACHE_DIRand Runner dependency handling (#356, #357). - Fixed Runner package downloads in multi-replica object-cache deployments by using the uploaded object's returned cache endpoint (#482).
- Fixed Docker image rebuild behavior and added missing tooling to the executor-manager image (#348, #385).
- Fixed Unix socket path length handling for host shim communication (#405).
- Fixed
flmpingoutput and README/example drift (#217, #256, #328).
Contributors
Thanks to everyone who contributed changes in this release cycle: