15,762 technical terms and definitions
knowledge distillation efficiency, knowledge distillation, distillation efficiency
Knowledge Distillation Efficiency Introduction & Motivation Knowledge Distillation Efficiency addresses a central problem in ml-training process control and student-teacher-gap…
knowledge distillation, teacher student, soft labels, logit distillation, feature distillation
Model distillation trains a smaller student to reproduce useful behavior or internal representations of a larger teacher. It converts expensive model capability into…
Model Distillation into Interpretable Surrogates Introduction & Motivation Model Distillation into Interpretable Surrogates addresses a central problem in interpretability…
Model Distillation into Interpretable Surrogates (Extended Study) Introduction & Motivation Model Distillation into Interpretable Surrogates (Extended Study) addresses a central…
Model Distillation into Interpretable Surrogates (Extended Study) Introduction & Motivation Model Distillation into Interpretable Surrogates (Extended Study) addresses a central…
teacher student training, dark knowledge transfer, logit distillation, feature distillation
Knowledge Distillation is the model compression technique where a smaller student network is trained to replicate the behavior of a larger teacher network — learning not just…
teacher student network, knowledge transfer distillation, soft label distillation, distillation training
Knowledge Distillation is the model compression technique where a smaller "student" network is trained to mimic the behavior of a larger, more capable "teacher" network…
Rule Distillation from Complex Models Introduction & Motivation Rule Distillation from Complex Models addresses a central problem in explainability interpretation and model…
data drift, concept drift, covariate shift, population stability index, PSI, Kolmogorov-Smirnov test, model decay
Model drift is the gradual loss of a deployed machine learning model's predictive quality because the world it was trained on has changed: the inputs it receives no longer look…
model training
Model editing directly updates specific weights to fix factual errors or modify behaviors without full retraining. Motivation: Models contain factual errors, knowledge becomes…
model editing knowledge editing large language models, model editing, knowledge editing large, ceo llm memit mend mlp peft, model editing & knowledge editing in large language models
Model Editing & Knowledge Editing in Large Language Models 1. Introduction & Motivation Large language models memorize an enormous number of facts during pretraining, but that…
model ensemble inference and resource sharing overhead, model ensemble inference, ensemble inference resource, inference resource sharing, resource sharing overhead, model ensemble
Model Ensemble Inference and Resource Sharing Overhead Introduction & Motivation Model Ensemble Inference and Resource Sharing Overhead addresses a central problem in inference…
reinforcement learning advanced
Model ensemble RL is reinforcement-learning approaches that use multiple models or policies to improve robustness and uncertainty handling - Ensembles aggregate predictions…
llm evaluation, ai evaluation, mmlu, humaneval, gsm8k, benchmarking, model judge, human evaluation, goodhart law
Model evaluation measures whether a model is actually good at the thing it's being deployed to do, and the field's hardest-won lesson is that a single number almost never answers…
capability elicitation, few shot prompting evaluation, benchmark contamination
LLM Capability Elicitation and Evaluation is the systematic process of measuring what a language model can and cannot do — including prompt engineering for evaluation, avoiding…
llm evaluation framework, evaluation harness, benchmark contamination, llm benchmark design
LLM Evaluation and Benchmarking is the systematic methodology for measuring language model capabilities across diverse tasks — encompassing academic benchmarks (MMLU, HumanEval…
Aggregating Multiple Explanations Introduction & Motivation Aggregating Multiple Explanations addresses a central problem in explainability interpretation and model understanding…
interoperability, framework portability
ONNX for Model Interoperability What is ONNX? ONNX (Open Neural Network Exchange) is an open format for representing machine learning models, enabling interoperability between…
stealing, query
Model Extraction (Model Stealing) is the adversarial attack where an adversary reconstructs a functional copy of a proprietary machine learning model by systematically querying…
interpretability
Model Extraction is an attack that approximates a target model by repeatedly querying its prediction API - It can replicate decision behavior and expose intellectual property…
ai safety
Model extraction attack (also called model stealing) is a security attack where an adversary aims to recreate a proprietary ML model by systematically querying it and using…
model extraction, extraction knowledge, api ip kl, model extraction & knowledge theft, model extraction knowledge, extraction knowledge theft, knowledge theft
Model Extraction & Knowledge Theft Introduction & Motivation Model extraction: steal model knowledge via queries. Recover models via API access. Applications: security…
model extraction stealing, model extraction, extraction stealing
Model Extraction Stealing Introduction & Motivation Model Extraction Stealing addresses a central problem in systems process control and extraction-attack-query-based prediction…
unique, identify
Model Fingerprinting is the technique of identifying or verifying a machine learning model's identity based on its behavioral characteristics — using carefully crafted probe…
mfu, optimization
Model FLOPs utilization is the efficiency metric that estimates how much useful model computation is achieved relative to hardware capability - it separates productive model math…
Model-Free vs Model-Based Learning Introduction & Motivation Model-Free vs Model-Based Learning addresses a central problem in reinforcement policy learning…
model governance audit trail prediction fab ml systems, model governance, audit trail prediction, ml, model governance audit, governance audit trail, trail prediction fab
Model Governance and Audit Trail Prediction for Fab ML Systems Introduction & Motivation Model Governance and Audit Trail Prediction for Fab ML Systems addresses a central problem…
Hugging Face Hub, model registry, model cards, Mitchell 2019, from_pretrained, model versioning, TensorFlow Hub, PyTorch Hub
A model hub is an online registry where trained machine learning models are published, discovered, versioned and downloaded, in the way a package index such as PyPI serves code…
model inference energy efficiency prediction, model inference energy, inference energy efficiency, energy efficiency prediction, model inference, inference energy
Model Inference Energy Efficiency Prediction with Machine Learning Introduction & Motivation Model Inference Energy Efficiency Prediction with Machine Learning addresses a central…
portable model format, framework exchange
ONNX (Open Neural Network Exchange) is an open-source format for representing machine learning models, enabling interoperability between different frameworks and deployment…
gradient attribution saliency, shap lime explanation, attention visualization model, feature importance neural
Model interpretability and explainability are often used interchangeably but can be usefully distinguished: interpretability generally refers to a model being inherently…
shap shapley values, grad cam saliency, attention visualization, feature attribution method
Model explainability techniques applied to complex, opaque models like deep neural networks generate an approximate, human-readable account of why the model produced a particular…
interpretability
Model Inversion is an attack that reconstructs sensitive input features from model outputs or gradients - It exposes privacy risk by inferring private attributes from accessible…
ai safety
Model inversion attacks reconstruct training data from model parameters or prediction outputs. Attack types: Class representative: Reconstruct average input for a class - what…
privacy
Model Inversion Attacks are privacy attacks that reconstruct private training data (or representative features) from a trained model — exploiting the model's predictions…
privacy
Model inversion defense encompasses techniques to prevent attackers from reconstructing training data by querying or analyzing a trained machine learning model. Model inversion…
model inversion membership, model inversion, inversion membership
Model Inversion Membership Introduction & Motivation Model Inversion Membership addresses a central problem in systems process control and membership-inference-attack prediction…
model training
Model merging combines weights from multiple fine-tuned models to create a single model with combined capabilities. Methods: Linear interpolation (weighted average of weights)…
model averaging, slerp merging, weight interpolation, model fusion
Model Merging is a technique that combines multiple fine-tuned LLMs into a single model by interpolating or adding their weight spaces — creating models with combined capabilities…
model soup, slerp merge, ties merge, dare merge
Model Merging is the technique of combining the weights of multiple independently fine-tuned models into a single model without additional training — creating models that inherit…
model soup, TIES merging, DARE merging, frankenmerge, weight interpolation
Model Merging is the technique of combining the weights of multiple fine-tuned models into a single model without additional training — by interpolating, averaging, or selectively…
weight averaging, model soups, task arithmetic, federated averaging
Model Merging and Weight Averaging — Combining Neural Networks Without Retraining Model merging combines the parameters of multiple trained neural networks into a single model…
model soup, model averaging, task arithmetic merging, slerp merging
Model Merging is the technique of combining the weights of multiple independently-trained or fine-tuned neural networks into a single model that inherits the capabilities of all…
model merging task arithmetic combining fine tuned models without retraining, model merging, task arithmetic combining, cpu dare glue gpu slerp ties, merging task
Model Merging & Task Arithmetic: Combining Fine-Tuned Models Without Retraining Introduction & Motivation Fine-tuning a large pretrained model on a new task or dataset…
ml model monitoring, production model monitoring, ml observability, model performance monitoring
Model monitoring definition and system boundary. Model monitoring is the continuous measurement of a deployed model, its inputs and outputs, the serving system, and eventual…
model monitoring and drift detection, model monitoring drift, monitoring drift detection, model monitoring, monitoring drift, drift detection
Model Monitoring and Drift Detection Introduction & Motivation Model Monitoring and Drift Detection addresses a central problem in ml-ops process control…
model monitoring drift detection, model monitoring drift, monitoring drift detection, model monitoring, monitoring drift, drift detection
Model Monitoring Drift Detection Introduction & Motivation Model Monitoring Drift Detection addresses a central problem in systems process control…
Output-Level Explanation Methods Introduction & Motivation Output-Level Explanation Methods addresses a central problem in explainability interpretation and model understanding…
model training
Model parallelism splits model layers across devices, enabling training models too large for single GPU memory. Motivation: Models like GPT-3 (175B) exceed single GPU memory. Must…
tensor parallelism, pipeline parallelism, sequence parallelism, expert parallelism, megatron
Model parallelism partitions model parameters, activations, or layer computation across accelerators. It enables models and contexts that cannot fit on one GPU and can raise…