01 / Converse
Read once. Keep talking.
Use small-model prefill to start a long-context conversation, then reuse the target's cached history for follow-ups.
Introducing LatentPort-4+27B · Research preview
Small model reads.
Large model answers.
A hybrid composite model that transfers working state from a 4B reader to a 27B generator. The larger model starts answering without processing the original prompt.
Run it locally. Keep the conversation cached. Save a context and branch it into different tasks.
Apache-2.0 · Python runtime · GGUF · Windows / RTX 5090 preview
2.19×Faster TTFT at ~16K context
23 vs 11Completed tasks versus native 4B, out of 64
0Prompt tokens processed by the 27B
E120 composite evaluation, pinned NF4/BF16 stack on an RTX 5090. TTFT: 95% CI 2.15–2.23×. Task counts: independent confirmation panel. These are research-configuration results, not GGUF quality scores. See the comparison and scope.
A different way to compose models
Pass the working context.
Skip the second prefill.
Qwen3.5-4B reads the complete prompt, including your question. A learned bridge translates its full hybrid inference state into Qwen3.8-27B's state space. The 27B then generates from that state.
4B readerProcess the entire prompt
→
LatentPort bridgeTranslate H · K/V · C · R
→
27B generatorAnswer without prompt replay
The bridge transfers attention cache, recurrent memory, convolution history and terminal hidden state. It is a learned, model-pair-specific translation—not a text summary. This preview supports one direction: 4B → 27B.
The LatentPort Python runtime
Context becomes something you can build with.
Start with composite inference. Then give your context a lifetime beyond one request: capture it, translate it, save it, and create independent continuations.
capturetranslatesaveloadforkappendimport latentport as lp
with lp.load(backend="gguf") as port:
port.warm()
chat = port.composite.session()
answer = chat.ask("Analyze this document: ...")
print(answer.text, end="")
while not answer.finished:
answer = chat.generate()
print(answer.text, end="")The first turn uses the 4B and bridge. Later messages append to the 27B's retained state. Choose strict mode to keep every user input on the 4B instead.
02 / Branch
One context. Different questions.
Prepare a repository once. Give a security review and a performance review independent branches of the translated context.
03 / Resume
Pick up where you left off.
Save the complete state, release the models, and restore the context later with matching model and runtime identities.
04 / Extend
Bring new information in.
Append logs or tool output to a source state and translate a fresh snapshot when needed. Earlier facts are not edited or merged automatically.
Branches execute through a single-GPU queue; forking does not imply concurrent agent execution. Saved state grows with context and must be protected like the original data. Additional bridges and distributed serving are future work.
Measured composite performance
More completed tasks than 4B.
Less time to the first token than 27B.
The same frozen bridge was evaluated against both native models. The confirmation panel showed an 18.75 percentage-point gain over native 4B; native 27B remained stronger overall.
| Task panel | Native 4B | LatentPort-4+27B | Native 27B |
|---|---|---|---|
| A · 64 tasks | 9 / 64 | 26 / 64 | 29 / 64 |
| Confirmation B · 64 tasks | 11 / 64 | 23 / 64 | 29 / 64 |
| Prompt | Native 27B median | Composite median | Paired speedup · 95% CI |
|---|---|---|---|
| Short | 0.603 s | 0.513 s | 1.06× · 0.87–1.29 |
| ~2K | 2.148 s | 1.295 s | 1.65× · 1.61–1.69 |
| ~8K | 8.871 s | 4.459 s | 2.01× · 1.96–2.05 |
| ~16K | 24.663 s | 11.087 s | 2.19× · 2.15–2.23 |
E120: pinned NF4/BF16 research stack, RTX 5090, resident models. Eight independent requests per length; three timing repetitions. Paired ratios are not ratios of medians. Overall 32-request speedup: 1.67× (1.49–1.86). Confirmation quality gain: +18.75 points (95% interval +0.74 to +33.92).
TTFT includes reader work, translation and installation, but excludes cold loading. The first token may be a reasoning/control token. Faster complete answers are not established. Tasks are adapted public and authored cases, not official leaderboard scores; prose uses blinded AI grading. GGUF quality requires separate evaluation. Inspect the result data.
Start building
Your first composite model.
The preview runs locally on Windows x64 with Python 3.11, an RTX 5090 (32 GB), CUDA 13.0 runtime libraries and a compatible driver. The pinned GGUF models and native runtime download from Hugging Face.
$env:LATENTPORT_HOME = 'F:\LatentPort\runtime'
py -3.11 -m venv "$env:LATENTPORT_HOME\venv"
& "$env:LATENTPORT_HOME\venv\Scripts\Activate.ps1"
python -m pip install torch==2.11.0 --index-url https://download.pytorch.org/whl/cu128
python -m pip install "https://huggingface.co/latentport/latentport-4-27b-research-preview-GGUF/resolve/1415eb9591dfaf15355d8bba5ed38f64792026e0/runtime/latentport-0.1.0.dev1-py3-none-any.whl"
python -m latentport install-native
python -m latentport doctor
python -m latentport downloadChoose your own cache disk; allow ~40 GB free. The first model download is ~22 GB. Then run the Python example above. No repository clone or native compilation is needed for the supported binary. Other GPUs, Linux/macOS and vLLM are not qualified or integrated in this preview.
Where it began
Research into portable inference state.
The first LatentPort paper studied 4B → 9B recurrent-state transfer. LatentPort-4+27B and the inference runtime extend the project toward practical composite models and reusable context.