ARCHITECTURAL BLUEPRINT: INTEGRATING QWEN3.6 + MCP WITH THINKING PRESERVATION INTO SIA

Document ID: SI-AB-2026-SIA04

Author: Sentinel Systems Architecture (Otto)

Date: July 04, 2026

Strategic Target: SIA Codebase Stabilization & Weight Optimization (sia/weights/)

Deployment Environment: Local Inference Node (Apple M4 Pro Mac mini | 24GB Unified VRAM | macOS)

1. STRATEGIC OBJECTIVE

The Self-Improving AI (sia) framework requires deep relational planning, precise tool execution, and high-concurrency iteration loops. Running these agent loops on local consumer hardware (Local Inference Node's 24GB unified memory) introduces a critical latency and compute bottleneck when models generate extensive Chain-of-Thought (CoT) tokens across multi-turn sessions.

Standardizing sia on the Qwen3.6-35B-A3B (MoE) model paired with Model Context Protocol (MCP) and SGLang Prefix Caching solves this. This blueprint outlines how to:

1. Launch SGLang with prefix caching optimized for Apple Silicon on Local Inference Node.

2. Implement an MCP Client Session within the sia agent runner.

3. Use Thinking Preservation (preserve_thinking=True) to recycle multi-turn KV caches.

4. Adapt the sia/weights/dataset_builder.py to output CoT-Aware SFT Trajectories for training local Gemma/Qwen models.

2. MIRA DEPLOYMENT: HOSTING QWEN3.6 VIA SGLANG

To run the Qwen3.6-35B-A3B Mixture of Experts model in Q4 quantization (fitting comfortably inside ~18GB of Local Inference Node's unified VRAM), we deploy the high-throughput SGLang engine. SGLang’s RadixAttention dynamically manages and caches prefix KV states across multiple concurrent sessions.

Launch command on macOS (Local Inference Node)

Run SGLang with Metal acceleration, prefix caching, and the native context window:

python3 -m sglang.launch_server \
    --model-path Qwen/Qwen3.6-35B-A3B-Instruct-GGUF \
    --quantization q4_k_m \
    --host 127.0.0.1 \
    --port 11434 \
    --enable-prefix-caching \
    --context-length 65536 \
    --mem-fraction-static 0.75

Note: --enable-prefix-caching is the critical flag. It enables RadixAttention to instantly recognize matching instruction headers, system prompts, and prior conversation histories, bypassing redundant prompt evaluation.

3. MULTI-TURN PYTHON RUNNER: IMPLEMENTING THINKING PRESERVATION

In a multi-turn agent session, the model generates reasoning inside ... blocks on every turn. In typical pipelines, this history is sent back to the model, forcing the server to re-evaluate all prior reasoning tokens.

By setting preserve_thinking=True (or passing the custom extra_body parameters to SGLang), SGLang keeps the exact token representation of those thinking blocks inside the active KV cache.

python Implementation (`sia/agent/runner.py` template)

import json
import httpx
from typing import List, Dict, Any

class SIALocalRunner:
    def __init__(self, api_url: str = "http://127.0.0.1:11434/v1"):
        self.api_url = api_url
        self.client = httpx.Client(timeout=180.0)

    def execute_turn(self, messages: List[Dict[str, Any]], thinking_enabled: bool = True) -> Dict[str, Any]:
        """
        Execute an agent turn against local SGLang.
        Leverages custom extra_body params for Thinking Preservation and Radix prefix caching.
        """
        payload = {
            "model": "default",
            "messages": messages,
            "temperature": 0.2 if thinking_enabled else 0.0,
            "stream": False,
            # SGLang extra body parameters for Qwen3.6 reasoning control
            "extra_body": {
                "preserve_thinking": True,
                "thinking_budget": 2048 if thinking_enabled else 0
            }
        }
        
        response = self.client.post(f"{self.api_url}/chat/completions", json=payload)
        response.raise_for_status()
        res_data = response.json()
        
        # SGLang returns the standard chat completions payload
        choice = res_data["choices"][0]
        message = choice["message"]
        
        return {
            "role": "assistant",
            "content": message.get("content", ""),
            "tool_calls": message.get("tool_calls", None)
        }

4. MCP TOOL DECOUPLING IN SIA

To prevent codebase pollution by custom Python wrappers, we transition the target tools used during sia generations (such as file reading, editing, and sandbox command execution) to standard MCP stdio servers.

Standardizing target execution via MCP

Instead of hardcoding tool execution in sia/agent/tools.py, the runner initiates an asynchronous mcp.ClientSession over stdio to call the tools:

import asyncio
from mcp import ClientSession, StdioServerParameters
from mcp.client.stdio import stdio_client

async def run_mcp_tool_call(server_script: str, tool_name: str, arguments: dict):
    """
    Connect to a local standard MCP server over stdio and execute a tool.
    Decouples tool mechanics entirely from model-specific wrapper code.
    """
    server_params = StdioServerParameters(
        command="python3",
        args=[server_script]
    )
    
    async with stdio_client(server_params) as (read_stream, write_stream):
        async with ClientSession(read_stream, write_stream) as session:
            # Initialize connection
            await session.initialize()
            
            # Execute tool call
            result = await session.call_tool(tool_name, arguments)
            return result.content

5. TRAJECTORY PROCESSING: COT-AWARE SFT DATASET BUILDER

During local model fine-tuning with LLaMA Factory on Local Inference Node, we want to train our target models (e.g. Gemma-2B/7B or Qwen-7B) to perform Chain-of-Thought reasoning before generating tool calls or final responses.

We must refine dataset_builder.py to:

1. Identify and retain thinking blocks (...) in high-scoring trajectories instead of stripping them.

2. Format them natively in the output SFT ChatML dataset.

Code Patch for `sia/weights/dataset_builder.py`

We append a specific parser that extracts and standardizes CoT blocks during the format_to_sft_chatml transformation:

    def format_to_sft_chatml(self, trajectory: list[dict[str, Any]], flatten: bool = True) -> dict[str, Any]:
        """
        Format a raw trajectory list of turns into a CoT-Aware standard ChatML dataset schema.
        Preserves <think> blocks in assistant content to train local models to reason.
        """
        messages = []
        for turn in trajectory:
            role = turn.get("role")
            if not role:
                continue

            msg_entry = {"role": role}
            raw_content = turn.get("content", "")
            
            # Extract content string
            content_str = self.flatten_content(raw_content)
            
            # For assistant turns, make sure we do not strip reasoning
            if role == "assistant":
                # Ensure the <think> blocks are preserved
                # If the raw log has a separate reasoning field, wrap it
                reasoning = turn.get("reasoning", "")
                if reasoning and "<think>" not in content_str:
                    content_str = f"<think>\n{reasoning}\n</think>\n{content_str}"
            
            msg_entry["content"] = content_str

            if "tool_calls" in turn:
                msg_entry["tool_calls"] = turn["tool_calls"]
            if "tool_call_id" in turn:
                msg_entry["tool_call_id"] = turn["tool_call_id"]

            messages.append(msg_entry)

        return {"messages": messages}

6. SFT ALIGNMENT WORKFLOW ON MIRA (LLAMA FACTORY)

Once dataset_builder.py outputs the cot_sft_dataset.json, execute standard SFT training on Local Inference Node utilizing LoRA fine-tuning.

SFT Configuration (`llama_factory_mira_config.yaml`)

### model
model_name_or_path: Qwen/Qwen2.5-Coder-7B-Instruct

### method
stage: sft
do_train: true
finetuning_type: lora
lora_target: all

### dataset
dataset: sia_cot_sft
dataset_dir: data
template: qwen
cutoff_len: 8192
max_samples: 1000
overwrite_cache: true
preprocessing_num_workers: 4

### output
output_dir: saves/Qwen2.5-Coder-7B-SIA-LoRA
logging_steps: 10
save_steps: 100
plot_loss: true
overwrite_output_dir: true

### train
per_device_train_batch_size: 1
gradient_accumulation_steps: 8
learning_rate: 1.0e-4
num_train_epochs: 3.0
lr_scheduler_type: cosine
warmup_ratio: 0.1
fp16: true

### eval
val_size: 0.1
per_device_eval_batch_size: 1
evaluation_strategy: steps
eval_steps: 100

This guarantees the fine-tuned model replicates the precise step-by-step reasoning found in our highest-reward trajectories, driving local model self-improvement autonomously.