Skip to content

Always downloading after mount the model path #491

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
2 of 4 tasks
kennard520 opened this issue Feb 10, 2025 · 1 comment
Open
2 of 4 tasks

Always downloading after mount the model path #491

kennard520 opened this issue Feb 10, 2025 · 1 comment

Comments

@kennard520
Copy link

System Info

Information

  • Docker
  • The CLI directly

Tasks

  • An officially supported command
  • My own modifications

Reproduction

  1. I have downloaded the model in the path /data/models/bge-reranker-v2-m3
  2. excute the commond
    docker run -d \ --name text-embedding-service \ --restart always \ --gpus all \ -p 8082:80 \ -v /data/models/bge-m3:/data/ \ -v /data/models/.cache:/root/.cache \ -e MODEL_ID=BAAI/bge-m3 \ -e TOKENIZATION_WORKERS=10 \ -e MAX_BATCH_REQUESTS=10 \ -e MAX_CLIENT_BATCH_SIZE=500 \ -e HOSTNAME=0.0.0.0 \ ghcr.io/huggingface/text-embeddings-inference:1.5
  3. It will download the model every time. finally use the mount path

Expected behavior

If I download the model and mounted, there is no need to download it again.

@pocman
Copy link

pocman commented Feb 26, 2025

According to the Air gapped deployment section in the readme, you should probably use the path as model id.
Maybe something like MODEL_ID=/data/models/bge-m3 ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants