1. Definition There are two completely different phases in LLM inference, and PD separation is The compute-intensive Prefill phase, LLM processes all users’ input and computes the corresponding KV Cache The memory-intensive Decode phase, sequentially producing tokens one by one, computing only a single token per memory access 2. Metrics 2.

1. What Is MLOps MLOps is short for Machine Learning Operations, and it describes the standardization and engineering of the entire lifecycle around model development. MLOps includes the following key steps: Data management: storing, accessing, cleaning, and transforming data Model development: algorithm development and model construction Model training and tuning: training models on data, adjusting hyperparameters to optimize the model, and fine-tuning models Model evaluation: testing a model’s accuracy, generalization ability, and performance metrics Model deployment: deploying the model to the target environment and converting the model Model monitoring: monitoring the model’s behavior, performance degradation, data drift, and robustness Another similar concept is ModelOps.

1. Past Attempts In the post Without Tags, No One Can Recall You, I argued that in a homogeneous group the individual becomes background noise, while tags become effective information. In the image below, it is hard for us to remember any particular person, yet very easy to find someone through the A, B, C, D, E tags.

1. What a Programming Paradigm Is A programming paradigm is a class of typical programming conventions. On one hand, it provides engineers with a way to model entities, linking the physical world to code; on the other hand, it provides engineers with a way of thinking about code and programs.