关于Glove词向量复现的问题

[allenlittlestar]

Traceback (most recent call last):
File "main.py", line 73, in
main(args)
File "main.py", line 59, in main
model = get_model(num_classes, args)
File "/media/omnisky/data/wr/M3TR-master/models/init.py", line 12, in get_model
model = model_dict[args.model_name](vit, res101, num_classes)
File "/media/omnisky/data/wr/M3TR-master/models/M3TR.py", line 55, in init
self.sem_embedding = self.get_word_embedding(self.num_classes).detach()
File "/media/omnisky/data/wr/M3TR-master/models/M3TR.py", line 81, in get_word_embedding
loaded = torch.load(embedding_path)
File "/home/omnisky/anaconda2/envs/faster/lib/python3.7/site-packages/torch/serialization.py", line 595, in load
return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
File "/home/omnisky/anaconda2/envs/faster/lib/python3.7/site-packages/torch/serialization.py", line 766, in _legacy_load
if magic_number != MAGIC_NUMBER:
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

您好，我使用配好的环境能够成功复现您初始代码，但是将M3TR.py中#embedding_path = './Bert1/voc_embeddings.pt' 和 #embedding_path = './Bert1/coco_embeddings.pt'分别改为embedding_path = './Glove/voc_glove_word2vec.pkl'及embedding_path = './Glove/coco_glove_word2vec.pkl'之后，报上面的错误，请问可以指导一下如何修改吗？

[JiaweiZhao-git]

Bert的词向量维度是768，Glove的词向量维度是300，因此需要修改 M3TR.py中的self.embedding_dim，同时在网络中增加一层fc将维度从300转换到768。

[allenlittlestar]

首先感谢您的回复。
我修改M3TR.py中的self.embedding_dim=300，然后仿照class M3TR中def init(self, vit, model, num_classes):中的self.fc_transform = nn.Linear(196, self.embedding_dim)增加了一个全连接层 self.fc_transform1 = nn.Linear(self.embedding_dim, 768)，并将该层写入def get_semantic_token(self, x):中，在原有的x = self.fc_transform(mask)下加入了x = self.fc_transform1(x)。

但是还是报一开始的错误，请问我修改的有什么问题吗？如果方便的话，能不能请您上传一版Glove版本的文件，非常感谢。

[allenlittlestar]

Bert的词向量维度是768，Glove的词向量维度是300，因此需要修改 M3TR.py中的self.embedding_dim，同时在网络中增加一层fc将维度从300转换到768。

您好。我将Glove的文件读取修改为字典格式解决了上述问题，但报以下错误，应该跟您所说的在网络中增加一层fc将维度从300转换到768有关，请问您能不能详细说一下，这个FC应该写在哪个文件的哪个地方呢？可以直接写成 self.fc_transform1 = nn.Linear(self.embedding_dim, 768)这种格式吗？写好的语句还需要在别的地方调用吗？

Traceback (most recent call last):
File "main.py", line 73, in
main(args)
File "main.py", line 66, in main
trainer.train()
File "/media/omnisky/data/wr/M3TR-master/trainer.py", line 152, in train
self.run_iteration(self.train_loader, is_train=True)
File "/media/omnisky/data/wr/M3TR-master/trainer.py", line 204, in run_iteration
outputs = self.on_forward(inputs, targets, is_train=is_train)
File "/media/omnisky/data/wr/M3TR-master/trainer.py", line 118, in on_forward
outputs1, outputs2, out3 = self.model(inputs)
File "/home/omnisky/anaconda2/envs/faster/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/omnisky/anaconda2/envs/faster/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 159, in forward
return self.module(*inputs[0], **kwargs[0])
File "/home/omnisky/anaconda2/envs/faster/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/media/omnisky/data/wr/M3TR-master/models/M3TR.py", line 126, in forward
vit_tmp, x_vit = self.vit_features.forward_features(x, sem_token, self.sem_embedding.cuda())
File "/media/omnisky/data/wr/M3TR-master/models/vit_utils/multi_modal_transformer.py", line 404, in forward_features
x = torch.cat((x, semantic_tokens, embedding), dim=1)
RuntimeError: Sizes of tensors must match except in dimension 2. Got 300 and 768 (The offending index is 0)