Storage
Migrating JuiceFS Metadata from Redis to PGSQL
· β˜• 3 min read
1. Why Migrate Metadata Storage from Redis to PGSQL PGSQL is cheaper Redis stores metadata in memory, while PGSQL stores metadata on disk, so the cost difference is obvious. PGSQL performance is tunable Different PGSQL services offer different performance IOPS. If you do not have a continuously high performance requirement, PGSQL is a good choice.

Thoughts on Building a Storage System During Model Development
· β˜• 5 min read
This article is compiled from part of an internal sharing session I gave. 1. Core Elements of a Storage System 1.1 Security Credentials for object storage buckets, authorization when using storage PVCs, and control over access sources β€” these are all issues security needs to address. But they are also very easy to overlook, and when something goes wrong it is a big problem.

Some Ideas for a MemoryFS Storage System
· β˜• 4 min read
1. Using Memory as a Storage Medium The image above is the storage pyramid, showing the relationship between the price of a storage medium and its speed. Today, enterprises widely adopt tape and disks as storage media mainly because of their price advantage. Market prices are driven by supply and demand: even for an expensive storage medium, if the production cost is low, the market is large enough, and the yield rate in the production process is high, its market price will come down.

How to Warm Up JuiceFS Data
· β˜• 2 min read
1. About JuiceFS Caching On a host, warmed cache is placed directly on the host. In a cluster, there are two levels of cache: Worker, which provides cache shared at the cluster level Fuse, which provides cache at the current node level only 2. Warming Data with the JuiceFS Client Note that warming at the Fuse layer is only effective for the current node; if you need to warm the whole cluster, you must warm at the Worker layer.

Deploying Elasticsearch on Memory Storage - 100M+ Records, Full-Text Search in 100ms
· β˜• 11 min read
1. Mount a Memory Storage Directory on the Host Create a directory to mount 1 mkdir /mnt/memory_storage Mount the tmpfs filesystem 1 mount -t tmpfs -o size=800G tmpfs /mnt/memory_storage Storage is allocated on demand, so using 100G of storage only consumes 100G of memory. The host node has 2T of memory, and here 800G is allocated to store Elasticsearch data.