update README and fix bugs

1. Updated import paths for MoGeModel to reflect new structure.
2. Added instructions for manually downloading and setting up submodules in README.
3. Included VGGT submodule path in pipelines.py for improved functionality.

Author: skygoo2000

README.md

@@ -33,9 +33,17 @@ pip install git+https://github.com/asomoza/image_gen_aux.git
 
 3. Make sure the submodule and requirements are installed:
     ```
+    mkdir -p submodules
     git submodule update --init --recursive
     pip install -r requirements.txt
     ```
+    If the submodules are not installed, you need to manually download them and move them to `submodules/`. Run the following commands to install the submodules:
+    ```
+    # MoGe
+    git clone https://github.com/microsoft/MoGe.git submodules/MoGe
+    # VGGT
+    git clone https://github.com/facebookresearch/vggt.git submodules/vggt
+    ```
 
 4. Manually download these checkpoints:
    - SpatialTracker checkpoint: [Google Drive](https://drive.google.com/drive/folders/1UtzUJLPhJdUg2XvemXXz1oe6KUQKVjsZ) and move it to `checkpoints/`.
@@ -116,7 +124,7 @@ python demo.py \
     --output_dir <output_dir> \ # output directory
     --input_path <input_path> \ # the reference image or video path
     --camera_motion <camera_motion> \ # the camera motion type, see examples below
-    --tracking_method <tracking_method> \ # the tracking method (moge, spatracker). For image input, 'moge' is necessary.
+    --tracking_method <tracking_method> \ # the tracking method (moge, spatracker, cotracker). For image input, 'moge' is necessary.
     --gpu <gpu_id> \ # the gpu id
 ```
 
@@ -345,6 +353,7 @@ This project builds upon several excellent open source projects:
 
 * [MoGe](https://github.com/microsoft/MoGe) - Microsoft's monocular geometry estimation model that helps achieve more accurate 3D reconstruction.
 
+* [vggt](https://github.com/facebookresearch/vggt) - Facebook's video generation model that provides the foundational architecture for this project.
 We thank the authors and contributors of these projects for their valuable contributions to the open source community!
 
 ## Citation

demo.py

@@ -18,7 +18,7 @@
 from diffusers.utils import load_image, load_video
 
 from models.pipelines import DiffusionAsShaderPipeline, FirstFrameRepainter, CameraMotionGenerator, ObjectMotionGenerator
-from submodules.MoGe.moge.model import MoGeModel
+from submodules.MoGe.moge.model.v1 import MoGeModel
 from submodules.vggt.vggt.utils.pose_enc import pose_encoding_to_extri_intri
 from submodules.vggt.vggt.models.vggt import VGGT
 

models/pipelines.py

@@ -6,6 +6,7 @@
 project_root = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
 try:
     sys.path.append(os.path.join(project_root, "submodules/MoGe"))
+    sys.path.append(os.path.join(project_root, "submodules/vggt"))
     os.environ["TOKENIZERS_PARALLELISM"] = "false"
 except:
     print("Warning: MoGe not found, motion transfer will not be applied")
@@ -21,7 +22,7 @@
 from models.spatracker.utils.visualizer import Visualizer
 from models.cogvideox_tracking import CogVideoXImageToVideoPipelineTracking
 
-from submodules.MoGe.moge.model import MoGeModel
+from submodules.MoGe.moge.model.v1 import MoGeModel
 
 from image_gen_aux import DepthPreprocessor
 from moviepy.editor import ImageSequenceClip