Redis stores metadata in memory, while PGSQL stores metadata on disk, so the cost difference is obvious.
- PGSQL performance is tunable
Different PGSQL services offer different performance IOPS. If you do not have a continuously high performance requirement, PGSQL is a good choice.
- PGSQL has a higher storage ceiling
Estimating by metadata size should be more accurate, but the formula I prefer is: size of metadata * 2000 = size of storage. In other words, 200TB of data requires a 100GB Redis instance. Yet the largest master-replica Redis instance our usual cloud vendors offer is only 96GB, which cannot meet the storage requirement.
One pitfall of using a Redis cluster is that JuiceFS only uses a single Redis cluster node to store metadata, and the resources of the other nodes go unused.
A PGSQL instance, by contrast, can provide TB-level storage, and a single JuiceFS instance can hold PB-level data.
- Configure environment variables
1
2
3
4
| export REDIS_PASSWORD=
#ip:port/database
export REDIS_ENDPOINT=
export META_SERVER=redis://:$REDIS_PASSWORD@$REDIS_ENDPOINT
|
1
2
3
4
| juicefs dump $META_SERVER meta-redis.json
Dumped entries: 25334296/25752607 [============================================================>-] 17909.7/s ETA: 23s
juicefs[3287398] <INFO>: Dump metadata into meta-redis.json succeed [dump.go:107]
|
Wait for the export to finish.
- Configure environment variables
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
| export ACCESS_KEY=
export SECRET_KEY=
export BUCKET=
export ENDPOINT=ks3-cn-beijing-internal.ksyun.com
export BUCKET_ENPOINT=$BUCKET.$ENDPOINT
export PROVIDER=ks3
export NAMESPACE=
export PVC_NAME=
export NODE_SELECTOR_KEY=
export NODE_SELECTOR_VALUE=
export JUICEFS_IMAGE=juicedata/juicefs-fuse
export DEMO_IMAGE=shaowenchen/demo:ubuntu
export POSTGRES_USER=admin
export POSTGRES_PASSWORD=
export POSTGRES_IP=
export POSTGRES_PORT=5432
export POSTGRES_DB=
export META_SERVER=postgres://$POSTGRES_USER:$POSTGRES_PASSWORD@$POSTGRES_IP:$POSTGRES_PORT/$POSTGRES_DB?sslmode=disable
|
1
| psql -U $POSTGRES_USER -h $POSTGRES_IP -p $POSTGRES_PORT -d postgres -c "CREATE DATABASE $POSTGRES_DB;"
|
1
2
3
4
| juicefs load $META_SERVER meta-redis.json
juicefs[3315492] <WARNING>: Secret key was removed; please correct it with `config` command [load.go:138]
juicefs[3315492] <INFO>: Load metadata from meta-redis.json succeed [load.go:143]
|
- Configure the bucket secret key
1
| juicefs config --secret-key $SECRET_KEY $META_SERVER
|
For the process of configuring JuiceFS in a cluster, see Creating a PVC Backed by JuiceFS in Kubernetes
Finally I tested it with the default parameters, and the speed looked acceptable, within a tolerable range:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
| juicefs bench .
+------------------+------------------+---------------+
| ITEM | VALUE | COST |
+------------------+------------------+---------------+
| Write big file | 408.04 MiB/s | 2.51 s/file |
| Read big file | 232.10 MiB/s | 4.41 s/file |
| Write small file | 11.0 files/s | 91.25 ms/file |
| Read small file | 21.9 files/s | 45.61 ms/file |
| Stat file | 144.2 files/s | 6.94 ms/file |
| FUSE operation | 17912 operations | 1.48 ms/op |
| Update meta | 1228 operations | 9.88 ms/op |
| Put object | 356 operations | 114.92 ms/op |
| Get object | 356 operations | 55.30 ms/op |
| Delete object | 0 operations | 0.00 ms/op |
| Write into cache | 0 operations | 0.00 ms/op |
| Read from cache | 0 operations | 0.00 ms/op |
+------------------+------------------+---------------+
|
