1. Keywords

• Machine Learning (ML)

  The technology of automatically acquiring knowledge from data.

• Neural Network (NN)

  A model that imitates the structure and learning mechanism of biological neural networks; one of the branches of machine learning.

  The structure of a neural network consists of an input layer, hidden layers, and an output layer.

• Deep Neural Network (DNN)

  The hidden layers are usually more than 2.

  The outstanding performance of DNNs comes from using statistical methods to extract high-level features from raw sensory data.

• Recurrent Neural Network (RNN)

  A DNN’s network has no memory — its output is independent of previous network inputs. An RNN has intrinsic memory, allowing long-term dependencies to influence the output.

  The state values of the intermediate computations in an RNN are stored in the network and used to process subsequent input computations.

• Cost Function, also called Loss Function

  A function that measures the training or prediction error of a neural network under the current parameters.

  The goal of model training: during training, the value of the loss function must be continuously reduced via gradient descent until it converges to a stable value.

  Common loss functions include mean squared error, cross-entropy, and log-cosh.

• Gradient Descent

  A parameter optimization algorithm that moves along the gradient direction — like descending from a mountaintop into a valley — eventually reaching some local minimum.

  Its drawbacks are that convergence may be slow and it may get stuck at a local minimum.

  A gradient can be understood as a partial derivative, a rate of change; along the negative gradient direction the function decreases fastest.

• Learning rate

  A hyperparameter that controls the step size of parameter updates during gradient descent.

• Supervised learning

  Labels are used in the training data.

• Self-supervised Learning

  Labels are generated automatically from unlabeled data and used for training.

• Reinforcement learning

  A reward mechanism is introduced during training to simulate the way humans learn through trial and error, perceiving the state of the environment and maximizing accumulated reward.

• Transfer Learning

  Knowledge learned in a source task is transferred to a target task, thereby improving the learning efficiency of the target task. The source task’s model parameters serve as the starting point for initializing the target task.

• Large Model

  A machine learning and deep learning model with a very large parameter scale, on the order of tens of billions of parameters.

• Small Model

  A model with a parameter scale below one hundred million, a model size under 100MB, and the ability to train efficiently on a single device.

• Self-Attention

  Lets the model learn the correlations between tokens; this is the core mechanism of the Transformer model.

  It enables parallel modeling of sequences.

• Transformer

  A neural network structure based on the attention mechanism.

  It allows the model to attend to information at different positions simultaneously.

  Compared with RNNs, it has a significant advantage in parallel computation.

• Alignment

  Large models’ training data carries bias and cannot stay consistent with ideal human values.

  Various means are used to ensure that the model’s output conforms to the mainstream.

• RLHF (Recursive Human Feedback)

  Improving the alignment of large models through user feedback.

• Fine-tuning

  With the help of a model trained on another task, fine-tuning on the target task dataset converts a pretrained model into a custom model.

• Large model quantization

  The technique of converting model parameters from floating-point numbers to low-bit integers.

  By sacrificing precision, it lowers device requirements and improves speed.

  Full precision (32-bit float) - half precision quantization (16-bit float) - 8-bit integer - 4-bit integer.

• Embedding

  Maps high-dimensional discrete inputs to low-dimensional dense vectors, improving model efficiency.

  The distance between word vectors represents the relationship between words.

• Low-dimensional dense vectors

  Representing each sample with fewer dimensions than the original representation, so that the distance between samples better reflects the relationship between them.

• Vocabulary expansion

  Avoids unencoded words being set to [UNK], which loses semantic information.

• batchSize

  Defines how many training samples a neural network feeds into the model per iteration.

  Larger values consume more resources and are faster; smaller values are less stable but save resources.

  Number of samples = batchSize * number of iterations

  Typically you start experimenting with powers of 2; the typical range is between 16 and 512.

• Perplexity

  Perplexity measures the uncertainty or confusion of the model in predicting the next word given a test sample.

  Perplexity is an evaluation metric for language models; smaller is better.

2. Processes and Steps

2.1 The Training Process of a Neural Network

1. Initialize the network parameters (weights and biases)
2. Run a forward computation on the training dataset with the current parameters to obtain the loss function
3. Compute the gradient of the loss function with respect to the current parameters
4. Update the network parameters using gradient descent combined with the learning rate
5. Repeat 2~4 for many rounds of iteration, updating the parameters
6. When the loss function value converges or a preset number of iterations is reached, end training

2.2 The Training Process of a Large Model

The main process of training a Large Model includes:

1. Data preparation: collect and consolidate massive amounts of training data.
2. Pretraining: pretrain the model on a large dataset and initialize the model’s parameters. At present, pretraining is mainly done in a self-supervised manner.
3. Model construction: design the model’s network structure, such as Transformer, ResNet, and other structures. structure Affect Indicators.
4. Distributed training: train in a distributed environment with multiple GPUs or multiple nodes to improve speed. Techniques such as data parallelism need to be implemented.
5. Hyperparameter optimization: tune hyperparameters such as batchSize and learning rate, and run many rounds of training to get the best result.
6. Model compression: use methods such as knowledge distillation to compress the model size for practical deployment.
7. Fine-tuning and deployment: use fine-tuning on downstream tasks to convert the pretrained general-purpose model into a specialized model, then deploy the service.

2.3 The Process of Setting the Learning Rate

1. First pick a relatively large learning rate as the starting value, for example 0.01
2. Train for a while with this initial learning rate and observe how the loss function changes
3. If the loss function fluctuates a lot or diverges, reduce the learning rate, for example by 10x to 0.001
4. Find a learning rate that steadily reduces the loss function, then keep training until it converges

2.4 How Self-Attention Is Computed

1. Convert the input sequence X into Query (Q), Key (K), and Value (V) matrices.
2. Compute the dot product of Query and Key, then divide by a scaling factor to obtain the attention score matrix (Attention Score). This reflects the relevance of each token’s relative position.
3. Apply softmax to the attention score matrix to obtain the attention weights (Attention Weight). The weights sum to 1.
4. Multiply V by the attention weights, i.e. take the weighted sum of Value, to obtain the output representation Z.
5. Z serves as the output of this module and can be passed to the next layer.

3. Questions

3.1 Comparison of Various Neural Networks

3.2 Transformer vs. Traditional Neural Networks

3.3 How to Estimate the GPU Memory Needed to Train a Large Model

Formula GPU_mem = params * dtype + (embed_size * batch_size + ratio * (params + embed_size * batch_size)) where:

• params: the number of model parameters
• dtype: the data type precision (FP32, FP16, etc.)
• embed_size: the embedding size of the input sequence
• batch_size: the batch size
• ratio: the extra overhead ratio (between 1.2 and 2)

Below are memory requirement estimates for some typical models:

3.4 Differences Between Data Mining, Data Modeling, and Large Models

Summary:

Data mining emphasizes exploration; data modeling emphasizes prediction; large models combine the strengths of both, able to both explore knowledge and perform prediction and reasoning. The three can be effectively combined to bring out their respective advantages.