1. Data Processing Architecture It is mainly divided into four layers: Processing capability layer: Spark on Kubernetes provides streaming data processing capability Data management layer: Iceberg provides dataset access operations such as ACID and tables Storage layer: Hive MetaStore manages Iceberg table metadata, PostgreSQL serves as the storage backend for

1. What GDS (GPUDirectStorage) Is GDS lets an RDMA NIC access GPU memory directly, which helps increase the IO bandwidth for GPU applications reading and writing files, reduces IO latency, and lowers their CPU load. Once the client has GDS enabled, files are opened in O_DIRECT mode and the client no longer caches file data.

A token is a unit tightly bound to data. It can be used to measure how much corpus is needed to train a model, and also to measure the input and output length at inference time. 1. What a token is A token can be a whole word, a subword,

1. About FLOPs FLOPs (Floating Point Operations Per Second) refers to the number of floating-point operations performed per second. Specifically: A single floating-point addition, such as a + b, counts as one floating-point operation. A single floating-point multiplication, such as a * b, also counts as one floating-point operation. Other basic floating-point operations, such as division and square root, can also be counted as one floating-point operation.

1. Definition There are two completely different phases in LLM inference, and PD separation is The compute-intensive Prefill phase, LLM processes all users’ input and computes the corresponding KV Cache The memory-intensive Decode phase, sequentially producing tokens one by one, computing only a single token per memory access 2. Metrics 2.