https://www.chenshaowen.com/en/tags/data-storage/Recent content in Data Storage on Shaowen Chen's WebsiteHugo -- gohugo.ioen©2016 - {year}, All Rights Reserved.Sun, 26 May 2024 00:00:00 +0000weeklyhttps://www.chenshaowen.com/en/blog/data-storage-in-model-development-cycle.htmlSun, 26 May 2024 00:00:00 +0000Sun, 26 May 2024 00:00:00 +0000https://www.chenshaowen.com/en/blog/data-storage-in-model-development-cycle.html1. Data Delivery Based on Object Storage As shown above, model R&D mainly involves three sub-platforms: Data platform The data platform is mainly responsible for data-related management, such as data ingestion and data processing, ultimately producing the data needed for training. The data platform stores raw data in object storage. During processing, it fetches data from object storage, processes it, and stores the result back in object storage.WeChat Official AccountLLMModelJuiceFSData Storage