1. Parameter Server Architecture In the Parameter Server architecture, the nodes in the cluster are divided into two categories: parameter server nodes (Parameter Server) and worker server nodes (Worker). 1.1 Parameter Server The Parameter Server is used to store the model’s parameters. Each parameter server node is responsible for managing and updating one part of the model’s parameters, while each worker node only processes the subset of data corresponding to it.