Data Processing
Processing Data on Kubernetes with Iceberg and Spark
· β˜• 7 min read
1. Data Processing Architecture It is mainly divided into four layers: Processing capability layer: Spark on Kubernetes provides streaming data processing capability Data management layer: Iceberg provides dataset access operations such as ACID and tables Storage layer: Hive MetaStore manages Iceberg table metadata, PostgreSQL serves as the storage backend for

Reading and Writing Excel with Python
· β˜• 2 min read
Python and Excel are both commonly used for data processing, so passing data back and forth and handling computations between them is unavoidable. This article mainly introduces the Python-Excel family of libraries, and how to use the xlrd and xlwt libraries. 1. Common Libraries xlwings, openpyxl, pandas, win32com, xlsxwriter, DataNitro,