datamini/qdrant-br repository overview

⁠Qdrant Backup and Restore(qdrant-br)

This Docker image is used for the periodic backup of the Qdrant database and supports restoring data from these backups. It supports storing backup data on AWS S3 or Alibaba Cloud OSS.

⁠Preparation

Before you start, make sure you have:

• The URL and access credentials for the Qdrant service.
• Access keys and bucket information for AWS S3 or Alibaba Cloud OSS.

⁠Using the Image

⁠Environment Variables

• STORAGE_SERVICE: The type of storage service, either S3 or OSS.
• STORAGE_REGION: The region of the storage service, for example, us-east-1 or cn-shanghai.
• QDRANT_URL: The URL of the Qdrant service.
• ACCESS_KEY: Access key ID for the storage service.
• SECRET_KEY: Secret access key for the storage service.
• BUCKET_NAME: The name of the storage bucket.
• URI_PREFIX: The prefix for the backup files stored in the bucket, the default is qdrant_backups.
• BACKUP_CRON_SCHEDULE: The Cron expression for scheduling backups, the default is to execute at midnight every day (0 0 * * *).

⁠Starting the Container

1. Build the Docker image:

   docker build -t qdrant-br.py .
   

   Copy

   or pull it from the Docker Hub:

   docker pull datamini/qdrant-br.py
   

   Copy

2. Run the container:

   Using AWS S3:

   docker run -d \
   -e QDRANT_URL=http://your-qdrant-url:6333 \
   -e QDRANT_KEY=your-qdrant-key \
   -e STORAGE_SERVICE=S3 \
   -e STORAGE_REGION=us-east-1 \
   -e ACCESS_KEY=your-access-key \
   -e SECRET_KEY=your-secret-key \
   -e BUCKET_NAME=your-bucket-name \
   -e BACKUP_CRON_SCHEDULE="0 0 * * *" \
   qdrant-br
   

   Copy

   Using Alibaba Cloud OSS:

   docker run -d \
   -e QDRANT_URL=http://your-qdrant-url:6333 \
   -e QDRANT_KEY=your-qdrant-key \
   -e STORAGE_SERVICE=OSS \
   -e STORAGE_REGION=cn-hangzhou \
   -e ACCESS_KEY=your-access-key \
   -e SECRET_KEY=your-secret-key \
   -e BUCKET_NAME=your-bucket-name \
   -e BACKUP_CRON_SCHEDULE="0 0 * * *" \
   qdrant-br
   

   Copy

⁠Using the CLI Tool

Enter the container, you can use the CLI tool (qdrant-br) to perform manual backups, restores, or list backup files:

docker exec -it [container ID or name] /bin/bash

Then:

• Perform a backup: qdrant-br backup
• List backups: qdrant-br list --days 3
• Restore from a backup: qdrant-br restore [snapshot_name] [collection_name]
• Check configuration: qdrant-br check

⁠Verification

Upon container startup, it will automatically execute a check command to verify the storage service credentials. If the check fails, the container will not start.

⁠Considerations

Make sure your storage service credentials are correct, and the Qdrant service URL is accessible. Incorrect configuration will cause the container to fail to start or the backup process to not work properly.

⁠Qdrant Backup and Restore（qdrant-br）

这个Docker镜像用于定时备份Qdrant数据库，并支持从这些备份中恢复数据。它支持将备份数据存储在AWS S3或阿里云OSS。

⁠准备

在开始之前，请确保你已经拥有：

• Qdrant服务的URL和访问凭证。
• AWS S3或阿里云OSS的访问密钥和存储桶信息。

⁠镜像使用

⁠环境变量

• STORAGE_SERVICE: 存储服务类型，S3 或 OSS。
• STORAGE_REGION: 存储服务的区域，例如 us-east-1或cn-shanghai。
• ACCESS_KEY: 存储服务的访问密钥ID。
• SECRET_KEY: 存储服务的访问密钥。
• BUCKET_NAME: 存储桶的名称。
• URI_PREFIX: 存储在存储桶中的备份文件前缀，默认为 qdrant_backups。
• BACKUP_CRON_SCHEDULE: 定时备份的Cron表达式，默认为每天午夜执行（0 0 * * *）。
• QDRANT_URL: Qdrant服务的URL。
• QDRANT_KEY: Qdrant服务的访问密钥。

⁠启动容器

1. 构建Docker镜像：

   docker build -t qdrant-br .
   

   Copy

   或者从Docker Hub拉取：

   docker pull datamini/qdrant-br
   

   Copy

2. 运行容器：

   使用AWS S3:

   docker run -d \
   -e QDRANT_URL=http://your-qdrant-url:6333 \
   -e QDRANT_KEY=your-qdrant-key \
   -e STORAGE_SERVICE=S3 \
   -e STORAGE_REGION=us-east-1 \
   -e ACCESS_KEY=your-access-key \
   -e SECRET_KEY=your-secret-key \
   -e BUCKET_NAME=your-bucket-name \
   -e BACKUP_CRON_SCHEDULE="0 0 * * *" \
   qdrant-br
   

   Copy

   使用阿里云OSS:

   docker run -d \
   -e QDRANT_URL=http://your-qdrant-url:6333 \
   -e QDRANT_KEY=your-qdrant-key \
   -e STORAGE_SERVICE=OSS \
   -e STORAGE_REGION=cn-hangzhou \
   -e ACCESS_KEY=your-access-key \
   -e SECRET_KEY=your-secret-key \
   -e BUCKET_NAME=your-bucket-name \
   -e BACKUP_CRON_SCHEDULE="0 0 * * *" \
   qdrant-br
   

   Copy

⁠使用CLI工具

进入容器内部，你可以使用CLI工具(qdrant-br)执行手动备份、恢复或列出备份文件：

docker exec -it [容器ID或名称] /bin/bash

然后：

• 执行备份：qdrant-br backup
• 列出备份：qdrant-br list --days 3
• 恢复备份：qdrant-br restore [backup_name] [collection_name]
• 检查配置：qdrant-br check

⁠验证

在容器启动时，它会自动执行一个check命令来验证存储服务的凭证。如果检查失败，容器将不会启动。

⁠注意事项

确保你的存储服务凭证是正确的，并且Qdrant服务URL是可访问的。错误的配置会导致容器启动失败或备份过程无法正常工作。