NeuralFoundry AI is the developer cockpit to custom-train, compress, and optimize open-source LLMs. Distill heavy teacher weights into efficient edge models, customize adapter hyperparameters, and export GGUF formats instantly.
Switch tabs to preview simulated Fine-Tuning and Distillation pipelines.
Forget brittle commands and massive cluster setups. NeuralFoundry AI bundles the absolute state-of-the-art parameter tuning options inside a production-grade workspace.
Target specific parameter projection modules (q_proj, v_proj, gate_proj) with specialized LoRA layers. Tune learning rates, dropouts, and batch allocations dynamically.
Distill dense model intelligence (e.g. Llama 3 8B) into ultra-portable student models using Kullback-Leibler (KL) divergence distillation temperaments.
Keep track of live metrics, loss calculations, data ingestion speed, and convergence criteria with a built-in interactive streaming developer console.
Export adapters or distilled student models straight into GGUF formatting. Ready to run locally on your mac via llama.cpp or LM Studio.
Connect your Hugging Face credentials and push custom adapters or distilled models back into public repositories with a single click.
All dataset parsing, tokenization validation, and parameter checks happen securely. Ensure models compile properly before execution.
NeuralFoundry AI abstracts complex training loops while giving full visibility into telemetry parameters. Fine-tuning builds custom adapter checkpoints, and distillation uses relative KL divergence losses to enforce similarity metrics with teacher outputs.
{
"model_type": "adapter_distillation",
"teacher_model": "meta-llama/Llama-3.1-8B-Instruct",
"student_model": "Qwen/Qwen2.5-1.5B-Instruct",
"hyperparameters": {
"kl_alpha": 0.9,
"temperature": 2.0,
"lora": {
"r": 32,
"alpha": 32,
"dropout": 0.1
},
"optimizer": "adamw_8bit"
},
"export": [
"GGUF",
"adapters_only"
]
}
Access the distillation and fine-tuning suite in a secure sandbox workspace. Optimize your open-source LLMs in minutes.