Auditable identity compiler for evidence-grounded synthetic people
schema v0.24 diagrams
System Overview
Generation Pipeline
Training Flywheel
Web Tier
Clients
Serving
Generation Pipeline
Training Flywheel
Storage / Atlas
External APIs
Concourse Agent EconomyTraining Flywheel auto at600 soulsStorageExternal APIsAtlas LayerGeneration PipelineSoul Engine engine/loop.pyDashboard port 8403FastAPI port 8402Web Frontend Next.js 15port 3000Clientspassed soulupsertindexappendportraitpage themeWeb BrowserAI Agent / Claude CodeRemote Harvester/ Home Feed/p/seed Profile/arena /wall /globePOST /api/generateGET /api/jobs/idPOST /api/mcp JSON-RPC/api/ingest/* admin-tokenDrizzle ORMserve/server.pyPortrait GenerationPage Theme GenerationMCP Server FastMCPglobe / telemetry / flywheelclaim job → generate→ score → storeDay-partitioned event logFile-based job queuesampler.py SeedSpecSamplingatlas/retrieve.py ContextBundleteacher/prompts.py PromptAssemblyMlxTeacher Qwen3-4B or9B plus LoRAAnthropicTeacherclaude-sonnet-5compiler/compile.pycompile_soulconstitution.py SafetyChecksgrounding.py Atlas Auditrubric/axes.py 8-axis Scoreatlas.db SQLite 11 tablesbuild_free.py World Bank +Wikidatabuild_x_official.py X TwitterAPIbuild_x402.py x402 paidfeedsWorld Bank APIWikidata SPARQLX Twitter Official APIagentcash x402 paid feedsOpen-Meteo WeatherAnthropic APIclaude-sonnet-5PostgreSQL souls jobseventsS3 / MinIO Portraitscatalog.db Soul indexdata/souls/ Gallery JSONdata/state/ config + eventscorpus/raw/ JSONL logmake_sft_dataset.pymake_dpo_pairs.pymlx_lm.lora SFT trainingmlx_lm_lora DPO trainingeval/harness.py A/B evalDeploy best adapterprovision_soul Ed25519keyharvest room transcripts
Input
Sampling
Atlas Bundle
Generation
Compilation
Scoring
Output
OutputScoring rubric/axes.pyCompilationcompiler/compile.pyGenerationteacher/generate.pyAtlas Bundleatlas/retrieve.pySampling sampler.pyInputpassfailfailfailpasspasspasspasspasspassNatural language promptFree run randomPostgres job queueparse_prompt_clamp~1000 demographic cellssample_cell weightedrandomindividuate age / occupation/ languages / cognition /hometownMusic age-gated slotsTrends X API or scrapeNames 12 freq-weightedEvents / texture / weather /showsContextBundle Markdownwith atlas:// refsSYSTEM_TEACHER 3Ktokens shared viaPromptPrefixCacheuser_prompt bundle + seed+ banned openersMlxTeacher Qwen3 optionalLoRA optional grammarconstraintsAnthropicTeacherclaude-sonnet-5 temp 1.0Raw JSON strip fences fillpipeline fieldsSanitize strip dashesRepair shapes fixenvelopesPydantic v2 schemavalidateConstitution SFW /domestic fear / stereotype /name collisionGrounding Atlas refs / artistmatch / provenancemanifestRubricScore 8 weightedaxes 0 to 1Quality gates age drift /hometown / distinctiveness0.45data/souls/ gallery JSONPostgreSQL upsertPortrait FLUX local or x402Page theme agent designsown CSScorpus/raw/ JSONL appendConcourse Ed25519provisionRetry with feedback temp0.7 to 0.9 up to 3x
Accumulate
Data Prep
Training
Evaluation
Deploy
DeployEval eval/harness.pyTraining mlx_lmData PrepAccumulate engine/loop.pyappend rowstudent winsno improvementused for next generationSoul Engine continuousloopcorpus/raw/ JSONL logsouls_passed counterTrigger at gap of 600 soulsmake_sft_dataset.py filter /dedup / formatmake_dpo_pairs.py highestvs lowest rubric per cellcorpus/sft/ train valid testcorpus/dpo/ chosenrejectedSFT mlx_lm.loraadapters/sft_vNDPO mlx_lm_loraadapters/dpo_vNTeacher arm base model noadapterStudent arm base modelplus LoRApass rate / skeleton rate /near-dup rate / groundinginvariantsz-test 95% student mustbeat teacherScorecarddata/state/eval/run_id.jsonDeploy best adapterconfig.student_adapterKeep current no significantimprovement
Clients
Auth
Pages
Public API
Ingest API
Database
Database Drizzle ORMPostgreSQLPublic API RoutesAuth lib/gate.tsBrowser ClientsIngest API engine writesPOST /api/ingest/soulsupsertPOST/api/ingest/souls/seed/portraitGET /api/ingest/statusPOST /api/ingest/eventsPOST /api/ingest/jobs/claimFOR UPDATE SKIPLOCKED 900s leasePOST /api/ingest/jobs/idfinishNext.js Pages App Router/ Home Feed paginatedsoul grid/p/seed Profilesoul-designed CSS theme/arena Soul fight arena/wall Real-time wall/globe 3D globe/telemetry /flywheel /skinsUser BrowserAI Agent / Claude CodePortal gate HMAC cookieLAN bypass 30-day TTLAdmin gate Bearer tokenMOSAIC_ADMIN_TOKENPOST /api/generate insertjobGET /api/jobs/id poll statusGET /api/status engine rowGET /api/events recenteventsPOST /api/mcp JSON-RPCfind_me_a_soulsouls seed PK / name /country / score / portrait /page jsonb / payload jsonbjobs uuid PK / kind / state /lease_expires_atevents bigserial / ts / type /seedengine_status singleton rowtexture place / kind / snippet