Posts
Thoughts on Building a Storage System During Model Development
· β˜• 5 min read
This article is compiled from part of an internal sharing session I gave. 1. Core Elements of a Storage System 1.1 Security Credentials for object storage buckets, authorization when using storage PVCs, and control over access sources β€” these are all issues security needs to address. But they are also very easy to overlook, and when something goes wrong it is a big problem.

Some Ideas for a MemoryFS Storage System
· β˜• 4 min read
1. Using Memory as a Storage Medium The image above is the storage pyramid, showing the relationship between the price of a storage medium and its speed. Today, enterprises widely adopt tape and disks as storage media mainly because of their price advantage. Market prices are driven by supply and demand: even for an expensive storage medium, if the production cost is low, the market is large enough, and the yield rate in the production process is high, its market price will come down.

Using Fluid to Access OSS Storage and Performance Testing
· β˜• 6 min read
1. Jindo Directly Accelerating OSS Set the environment variables 1 2 3 4 5 6 7 export ENDPOINT=oss-cn-beijing-internal.aliyuncs.com export BUCKET= export AK= export SK= export NAMESPACE=default export PVC=myoss-jindo export URI=datacenter Create the credentials 1 2 3 4 5 6 7 8 9 10 11 kubectl apply -f - <<EOF apiVersion:

How to Warm Up JuiceFS Data
· β˜• 2 min read
1. About JuiceFS Caching On a host, warmed cache is placed directly on the host. In a cluster, there are two levels of cache: Worker, which provides cache shared at the cluster level Fuse, which provides cache at the current node level only 2. Warming Data with the JuiceFS Client Note that warming at the Fuse layer is only effective for the current node; if you need to warm the whole cluster, you must warm at the Worker layer.

High-Frequency IO Pods Are Not Suited to Setting a Limit
· β˜• 3 min read
1. Symptoms Elasticsearch on Kubernetes was restarting frequently, leaving the service almost unusable. During data import, the Pod’s memory usage kept growing Once the Pod’s memory usage approached the Limit, continued importing triggered an abnormal Pod exit, with the error log ERROR: Elasticsearch exited unexpectedly The Pod’s memory usage did not drop; it stayed near the Limit, and before long the Pod exited abnormally again The Elasticsearch Pod’s memory limit was 64GB, while the JVM memory limit was 32GB.