In artificial intelligence and machine learning, a parameter is a numerical value within a model that is learned from training data. Parameters define the internal configuration of a model: they are the weights and biases in neural networks that determine how inputs are transformed into outputs. During training, an optimization algorithm adjusts these parameters iteratively to minimize the difference between the model's predictions and the actual target values, a process known as learning.
The number of parameters in an AI model is often used as a rough indicator of its capacity and capability:
- Small models: Millions of parameters, suitable for specific tasks like sentiment analysis or image classification.
- Medium models: Hundreds of millions of parameters, capable of handling more complex tasks with greater nuance.
- Large language models (LLMs): Billions to trillions of parameters, powering systems like GPT-4, Claude, and Gemini that demonstrate broad reasoning and generation abilities.
It's important to distinguish parameters from hyperparameters. Parameters are learned automatically during training (e.g., neural network weights), while hyperparameters are set manually before training begins (e.g., learning rate, batch size, number of layers). Hyperparameters control how the training process works, whereas parameters represent what the model has actually learned from the data.
More parameters generally means greater model capability but also higher computational costs for training and inference, increased memory requirements, and potential risks like overfitting. Choosing the right model size means balancing performance needs against infrastructure costs and latency requirements to get practical results in production.