Skip to content

Commit ab5ca92

Browse files
ai-edge-botcopybara-github
authored andcommitted
Modify tensor names of the image encoder to load weights from checkpoint
PiperOrigin-RevId: 773373003
1 parent d84b34c commit ab5ca92

File tree

1 file changed

+11
-10
lines changed

1 file changed

+11
-10
lines changed

ai_edge_torch/generative/examples/gemma3/image_encoder.py

Lines changed: 11 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -24,26 +24,27 @@
2424

2525

2626
TENSOR_NAMES = loading_utils.ModelLoader.TensorNames(
27-
ff_up_proj="vision_tower.vision_model.encoder.layers.{}.mlp.fc1",
28-
ff_down_proj="vision_tower.vision_model.encoder.layers.{}.mlp.fc2",
27+
ff_up_proj="siglip_vision_model.encoder_blocks.{}.mlp.fc1",
28+
ff_down_proj="siglip_vision_model.encoder_blocks.{}.mlp.fc2",
2929
attn_query_proj=(
30-
"vision_tower.vision_model.encoder.layers.{}.self_attn.q_proj"
30+
"siglip_vision_model.encoder_blocks.{}.self_attn.q_proj"
3131
),
3232
attn_key_proj=(
33-
"vision_tower.vision_model.encoder.layers.{}.self_attn.k_proj"
33+
"siglip_vision_model.encoder_blocks.{}.self_attn.k_proj"
3434
),
3535
attn_value_proj=(
36-
"vision_tower.vision_model.encoder.layers.{}.self_attn.v_proj"
36+
"siglip_vision_model.encoder_blocks.{}.self_attn.v_proj"
3737
),
3838
attn_output_proj=(
39-
"vision_tower.vision_model.encoder.layers.{}.self_attn.out_proj"
39+
"siglip_vision_model.encoder_blocks.{}.self_attn.o_proj"
4040
),
41-
pre_attn_norm="vision_tower.vision_model.encoder.layers.{}.layer_norm1",
42-
embedding="vision_tower.vision_model.embeddings.patch_embedding",
41+
pre_attn_norm="siglip_vision_model.encoder_blocks.{}.layer_norm1",
42+
pre_ff_norm="siglip_vision_model.encoder_blocks.{}.layer_norm2",
43+
embedding="siglip_vision_model.patch_embedding",
4344
embedding_position=(
44-
"vision_tower.vision_model.embeddings.position_embedding.weight"
45+
"siglip_vision_model.position_embedding.weight"
4546
),
46-
final_norm="vision_tower.vision_model.post_layernorm",
47+
final_norm="siglip_vision_model.final_norm",
4748
)
4849

4950

0 commit comments

Comments
 (0)