Issues Running Model Worker for Gradio Server

[nofreewill42]

(convllava) nofreewill@nofreewill:~/Documentos/convllava/conv-llava$ python llava/serve/model_worker.py
2024-05-27 16:17:58 | INFO | model_worker | args: Namespace(host='localhost', port=21002, worker_address='http://localhost:21002', controller_address='http://localhost:21001', model_path='facebook/opt-350m', model_base=None, model_name=None, device='cuda', multi_modal=False, limit_model_concurrency=5, stream_interval=1, no_register=False, load_8bit=False, load_4bit=False)
2024-05-27 16:17:58 | INFO | model_worker | Loading the model opt-350m on worker 190dfb ...
2024-05-27 16:17:58 | ERROR | stderr | /home/nofreewill/miniconda3/envs/convllava/lib/python3.11/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: resume_download is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use force_download=True.
2024-05-27 16:17:58 | ERROR | stderr | warnings.warn(
tokenizer_config.json: 0%| | 0.00/685 [00:00<?, ?B/s]
tokenizer_config.json: 100%|██████████████████████████████████████████████████████████████████████████████████| 685/685 [00:00<00:00, 1.51MB/s]
2024-05-27 16:17:59 | ERROR | stderr |
config.json: 0%| | 0.00/644 [00:00<?, ?B/s]
config.json: 100%|████████████████████████████████████████████████████████████████████████████████████████████| 644/644 [00:00<00:00, 1.41MB/s]
2024-05-27 16:17:59 | ERROR | stderr |
vocab.json: 0%| | 0.00/899k [00:00<?, ?B/s]
vocab.json: 100%|███████████████████████████████████████████████████████████████████████████████████████████| 899k/899k [00:00<00:00, 2.07MB/s]
vocab.json: 100%|███████████████████████████████████████████████████████████████████████████████████████████| 899k/899k [00:00<00:00, 2.05MB/s]
2024-05-27 16:18:00 | ERROR | stderr |
merges.txt: 0%| | 0.00/456k [00:00<?, ?B/s]
merges.txt: 100%|███████████████████████████████████████████████████████████████████████████████████████████| 456k/456k [00:00<00:00, 1.42MB/s]
merges.txt: 100%|███████████████████████████████████████████████████████████████████████████████████████████| 456k/456k [00:00<00:00, 1.41MB/s]
2024-05-27 16:18:01 | ERROR | stderr |
special_tokens_map.json: 0%| | 0.00/441 [00:00<?, ?B/s]
special_tokens_map.json: 100%|████████████████████████████████████████████████████████████████████████████████| 441/441 [00:00<00:00, 4.61MB/s]
2024-05-27 16:18:01 | ERROR | stderr |
You are using a model of type opt to instantiate a model of type llava_llama. This is not supported for all configurations of models and can yield errors.
pytorch_model.bin: 0%| | 0.00/663M [00:00<?, ?B/s]
pytorch_model.bin: 2%|█▎ | 10.5M/663M [00:00<00:22, 29.4MB/s]
pytorch_model.bin: 3%|██▋ | 21.0M/663M [00:00<00:21, 30.3MB/s]
pytorch_model.bin: 5%|███▉ | 31.5M/663M [00:01<00:20, 30.5MB/s]
pytorch_model.bin: 6%|█████▎ | 41.9M/663M [00:01<00:20, 30.7MB/s]
pytorch_model.bin: 8%|██████▌ | 52.4M/663M [00:01<00:19, 30.7MB/s]
pytorch_model.bin: 9%|███████▉ | 62.9M/663M [00:02<00:19, 30.7MB/s]
pytorch_model.bin: 11%|█████████▏ | 73.4M/663M [00:02<00:19, 30.6MB/s]
...
█████████████████████████████████████████████████████████████████████████████████ | 640M/663M [00:21<00:00, 31.2MB/s]
pytorch_model.bin: 98%|██████████████████████████████████████████████████████████████████████████████████▍ | 650M/663M [00:21<00:00, 31.2MB/s]
pytorch_model.bin: 100%|███████████████████████████████████████████████████████████████████████████████████▊| 661M/663M [00:21<00:00, 31.2MB/s]
pytorch_model.bin: 100%|████████████████████████████████████████████████████████████████████████████████████| 663M/663M [00:21<00:00, 30.3MB/s]
2024-05-27 16:18:24 | ERROR | stderr |
2024-05-27 16:18:24 | ERROR | stderr | /home/nofreewill/miniconda3/envs/convllava/lib/python3.11/site-packages/torch/_utils.py:831: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
2024-05-27 16:18:24 | ERROR | stderr | return self.fget.get(instance, owner)()

Some weights of LlavaLlamaForCausalLM were not initialized from the model checkpoint at facebook/opt-350m and are newly initialized: ['embed_tokens.weight', 'layers.0.input_layernorm.weight', ...]

[John-Ge]

Hello, I am curious about why the model checkpoint is at facebook/opt-350m. Maybe you could check the pretrained weight in the config.json. We would recently release the code for inference. Thank you!