data parallelism
model training
Data parallelism replicates the model on each device and processes different data batches in parallel. How it works: Copy complete model to each GPU, each processes different…
15,752 technical terms and definitions
model training
Data parallelism replicates the model on each device and processes different data batches in parallel. How it works: Copy complete model to each GPU, each processes different…
distributed data parallel, ddp training
Data parallelism is the simplest and most common way to scale training across many GPUs: replicate the entire model on every device, give each replica a different slice of…
data parallelism batch distribution and gradient synchronization, data parallelism batch, parallelism batch distribution, batch distribution gradient, data parallelism
Data Parallelism Batch Distribution and Gradient Synchronization Introduction & Motivation Data Parallelism Batch Distribution and Gradient Synchronization addresses a central…
ml data pipeline, etl, elt, data orchestration, airflow, dagster, prefect, spark, flink, kafka
Data pipeline is an automated, observable flow that ingests, validates, transforms, stores and delivers data products to downstream consumers. ML systems depend on reproducible…
input pipeline, prefetching data, data loader, io bound training
ML Data Pipeline is the system that efficiently loads, preprocesses, and batches training data — a bottleneck that can reduce GPU utilization from 100% to < 30% if poorly…
training, malicious
Data Poisoning is the adversarial attack that corrupts machine learning models by injecting malicious examples into training data — exploiting the fundamental dependence of ML…
ai safety
Data poisoning injects malicious samples into training data to corrupt model behavior. Attack goals: Untargeted: Degrade overall model performance. Targeted: Make model misbehave…
data prefetching accuracy prediction, data prefetching accuracy, prefetching accuracy prediction, data prefetching, prefetching accuracy, accuracy prediction
Data Prefetching Accuracy Prediction with Machine Learning Introduction & Motivation Data Prefetching Accuracy Prediction with Machine Learning addresses a central problem in data…
training data extraction attack, membership inference attack
Data privacy, in the context of AI, is the problem of keeping information about real individuals from leaking out of a system that touched their data — not just during collection…
Data Profiling and Quality Assessment Introduction & Motivation Data Profiling and Quality Assessment addresses a central problem in (subject) assurance and (target) management…
training
Data proportions is the explicit percentage share of each dataset component within the final training corpus - Proportion settings control how often each data type contributes…
Composite Data Quality Index DQ-Score Formulation Introduction & Motivation Composite Data Quality Index DQ-Score Formulation addresses a central problem in (subject) assurance…
Multi-Dimensional Data Quality Assessment Introduction & Motivation Multi-Dimensional Data Quality Assessment addresses a central problem in (subject) assurance and (target)…
data quality validation, data quality, quality validation
Data Quality Validation Introduction & Motivation Data Quality Validation addresses a central problem in ml-ops process control and validation-latency-per-record prediction: how…
Data Reconciliation and Matching Introduction & Motivation Data Reconciliation and Matching addresses a central problem in (subject) assurance and (target) management: how…
data reordering and prefetch generation in compiler, data reordering prefetch, reordering prefetch generation, prefetch generation compiler, data reordering, reordering prefetch
Data Reordering and Prefetch Generation in Compiler Introduction & Motivation Data Reordering and Prefetch Generation in Compiler addresses a central problem in ml-compiler…
training
Data replay is reintroduction of selected past data during later training phases to preserve learned capabilities - Replay buffers protect important knowledge when models continue…
training techniques
Data Retention is policy framework that defines how long data is stored before deletion or archival - It is a core method in modern semiconductor AI serving and trustworthy-ML…
data scientist, Drew Conway data science Venn diagram, John Tukey future of data analysis, DJ Patil Jeff Hammerbacher, data science vs machine learning, data science workflow
Data science names the practice of extracting usable knowledge and decisions from data by combining three things that, on their own, are each insufficient: statistics to reason…
Data Science in Medicine & Biomedical Analysis: Statistical Learning and Clinical Applications 1. Epidemiology and Statistical Analysis Epidemiology studies disease distribution…
data serialization formats, data serialization, serialization formats
Data Serialization Formats Introduction & Motivation Data Serialization Formats addresses a central problem in programming process control and format-overhead-size prediction: how…
distributed training
Data shuffling at scale is the large-distributed randomization of sample order to prevent correlation bias during training - it must balance statistical randomness quality…
Data Smells Automated Antipattern Static Analysis Introduction & Motivation Data Smells Automated Antipattern Static Analysis addresses a central problem in (subject) assurance…
Data Standardization and Normalization Introduction & Motivation Data Standardization and Normalization addresses a central problem in (subject) assurance and (target) management…
arrays vs linked lists, hash table explained, stacks and queues, intro trees and graphs
A data structure is simply a particular way of arranging the addresses from the previous topic so that a specific set of operations — adding an item, finding one, removing one…
Data Type and Format Consistency Introduction & Motivation Data Type and Format Consistency addresses a central problem in (subject) assurance and (target) management: how…
Data Completeness Validation Introduction & Motivation Data Completeness Validation addresses a central problem in (subject) assurance and (target) management: how to Improve…
Data Validation Rules and Constraints Introduction & Motivation Data Validation Rules and Constraints addresses a central problem in (subject) assurance and (target) management…
Data Valuation for Active Learning Introduction & Motivation Data Valuation for Active Learning addresses a central problem in (subject) learning and efficient (target)…
data versioning and reproducibility, data versioning reproducibility, data versioning, versioning reproducibility
Data Versioning and Reproducibility Introduction & Motivation Data Versioning and Reproducibility addresses a central problem in ml-ops process control…
data warehouse olap, data warehouse, warehouse olap
Data Warehouse OLAP Introduction & Motivation Data Warehouse OLAP addresses a central problem in systems process control and query-latency-optimization prediction: how to Predict…
data2vec General Self-Supervised Learning Across Modalities Introduction & Motivation data2vec General Self-Supervised Learning Across Modalities addresses a central problem…
database backup recovery plan, database backup recovery, backup recovery plan, database backup, backup recovery, recovery plan
Database Backup Recovery Plan Introduction & Motivation Database Backup Recovery Plan addresses a central problem in cloud process control and backup-strategy-full-incremental…
database connection pooling, database connection, connection pooling
Database Connection Pooling Introduction & Motivation Database Connection Pooling addresses a central problem in ml-inference process control and connection-acquisition-latency…
database migration strategy, database migration, migration strategy
Database Migration Strategy Introduction & Motivation Database Migration Strategy addresses a central problem in ml-ops process control and migration-time-hours prediction: how…
database migration strategy, database migration, migration strategy
Database Migration Strategy Introduction & Motivation Database Migration Strategy addresses a central problem in cloud process control and migration-downtime-zero-downtime…
database replication consistency, database replication, replication consistency
Database Replication Consistency Introduction & Motivation Database Replication Consistency addresses a central problem in ml-inference process control…
database replication consistency, database replication, replication consistency
Database Replication Consistency Introduction & Motivation Database Replication Consistency addresses a central problem in cloud process control…
database sharding strategy, database sharding, sharding strategy
Database Sharding Strategy Introduction & Motivation Database Sharding Strategy addresses a central problem in cloud process control and shard-key-distribution-balance prediction…
data modeling, relational tables, SQL, primary key, index, transaction, ACID
A database is the organized, persistent storage sitting behind the HTTP response from the previous topic, and this very article is a real row inside one. Data modeling is…
dataflow accelerator architecture design, dataflow accelerator architecture, accelerator architecture design, dataflow accelerator, accelerator architecture, architecture design
Dataflow Accelerator Architecture Design Introduction & Motivation Dataflow Accelerator Architecture Design addresses a central problem in ml-hardware process control…
dataflow analysis in compilers, dataflow analysis compilers, dataflow analysis, analysis compilers
Dataflow Analysis in Compilers Introduction & Motivation Dataflow Analysis in Compilers addresses a central problem in compiler-optimization process control and analysis-precision…
spatial computing hardware, coarse grain reconfigurable, cgra dataflow, dataflow processor design
Dataflow Architecture Computing is the processor design paradigm where instructions execute as soon as their input operands are available (data-driven execution) rather than…
dataflow computing and architectures, dataflow computing architectures, dataflow computing, computing architectures
Dataflow Computing and Architectures Introduction & Motivation Dataflow Computing and Architectures addresses a central problem in dataflow-computing process control…
wave computing, spatial architecture computing, coarse grain reconfigurable array cgra, stream dataflow architecture
Dataflow Processor Architecture: Spatial Computing via Coarse-Grained Reconfigurable Arrays — compute elements directly mapped to hardware nodes with data-driven execution model…
corpus, training data
Training Data for LLMs Pretraining Datasets Large language models are pretrained on massive text corpora—often trillions of tokens from diverse sources. Common Pretraining Sources…
Dataset Cartography and Data Maps for Training Dynamics Introduction & Motivation Dataset Cartography and Data Maps for Training Dynamics addresses a central problem…
distributed training
Dataset sharding is the partitioning of training data into non-overlapping subsets assigned across distributed workers - it ensures balanced workload distribution, minimizes…
Date-Time Parsing and Normalization Introduction & Motivation Date-Time Parsing and Normalization addresses a central problem in (subject) assurance and (target) management: how…
d2d variation, daily drift
Day-to-Day Variation (D2D) in semiconductor manufacturing refers to process parameter fluctuations between production days caused by environmental, equipment, or operational…