Add a flag --skip-cached to skip already-downloaded models and collections #461

khaledgabr77 · 2025-04-25T19:04:18Z

🎉 New feature

Closes #150

Summary

This PR introduces a --skip-cached flag option to gz-fuel-tools10 (available through the gz fuel download command).
When the flag is set, any models that already exist in the local cache are skipped instead of downloaded again, saving time.

Both individual model downloads and collection downloads now respect this flag.

Behaviour before this PR

Every time running of gz fuel download re-download of all models, even if they were already present locally.

Behaviour after this PR

With --skip-cached, the CLI first checks the local cache and only downloads the models that are missing.

Test it

Start a large collection download and interrupt it Press Ctrl+C partway through

gz fuel download -j 4  -u "https://fuel.gazebosim.org/1.0/openrobotics/collections/Gazebo classic model database"

Resume with the new flag – already-downloaded models are skipped

gz fuel download -j 4 --skip-cached -u "https://fuel.gazebosim.org/1.0/openrobotics/collections/Gazebo classic model database"

You can repeat the same steps with an individual model URL too.

gz fuel download -u https://fuel.gazebosim.org/1.0/OpenRobotics/models/Ambulance -v 4

gz fuel download -u --skip-cached https://fuel.gazebosim.org/1.0/OpenRobotics/models/Ambulance -v 4

Checklist

Signed all commits for DCO
Added tests
Added example and/or tutorial
Updated documentation (as needed)
Updated migration guide (as needed)
Consider updating Python bindings (if the library has them)
codecheck passed (See contributing)
All tests passed (See test coverage)
While waiting for a review on your PR, please help review another open pull request to support the maintainers

Note to maintainers: Remember to use Squash-Merge and edit the commit message to match the pull request summary while retaining Signed-off-by messages.

Signed-off-by: khaledgabr77 <[email protected]>

…rlds Signed-off-by: khaledgabr77 <[email protected]>

ahcorde

You have introduced many changes which are not related with this PR, do you mind to restore them

ahcorde · 2025-04-29T21:30:33Z

src/cmd/cmdfuel.rb.in

@@ -228,7 +229,8 @@ class Cmd
      'onlymodels' => '0',
      'onlyworlds' => '0',
      'defaults' => false,
-      'console' => false
+      'console' => false,
+      'skip_cached' => 'false'


Suggested change

'skip_cached' => 'false'

'skip_cached' => false

ahcorde · 2025-04-29T21:30:40Z

src/cmd/cmdfuel.rb.in

@@ -302,6 +304,9 @@ class Cmd
      opts.on('--console', 'Output to console') do
        options['console'] = true
      end
+      opts.on('-s', '--skip-cached', 'Skip downloading resources already in cache') do
+        options['skip_cached'] = 'true'


Suggested change

options['skip_cached'] = 'true'

options['skip_cached'] = true

ahcorde · 2025-04-29T21:31:12Z

src/cmd/cmdfuel.rb.in

@@ -434,9 +439,9 @@ class Cmd
          exit(-1)
        end
      when 'download'
-        Importer.extern 'int downloadUrl(const char *, const  char *, const char *, const char *, unsigned int)'
+        Importer.extern 'int downloadUrl(const char *, const  char *, const char *, const char *, unsigned int, const char *)'


Suggested change

Importer.extern 'int downloadUrl(const char *, const char *, const char *, const char *, unsigned int, const char *)'

Importer.extern 'int downloadUrl(const char *, const char *, const char *, const char *, unsigned int, bool)'

Hi @ahcorde
I followed the changes you suggested and updated both gz.cc and gz.hh to accept a bool parameter. However, when I tried to compile and run the code, I encountered the following issue:

Library error: Problem running [download]() from /root/gz_ws/install/lib/libgz-fuel_tools10.so.10.0.1

I need to determine whether this issue came from the Ruby script or the C++ implementation. Previously, when reading the file, I noticed that char was used to represent boolean values, so I followed the same approach. However, now I plan to recheck the full code to understand exactly where the issue is coming from.

khaledgabr77 · 2025-04-30T06:00:23Z

You have introduced many changes which are not related with this PR, do you mind to restore them

You're right — I forgot to mention that the changes are related to running ament_uncrustify --reformat. After I finished working on each C++ file, I applied the reformatting using that command.
Do you still want me to revert the changes?

ahcorde

You have introduced many changes which are not related with this PR, do you mind to restore them

You're right — I forgot to mention that the changes are related to running ament_uncrustify --reformat. After I finished working on each C++ file, I applied the reformatting using that command. Do you still want me to revert the changes?

we are not using uncrustify in gazebo. Yes, restore the changes

khaledgabr77 · 2025-04-30T12:11:03Z

You have introduced many changes which are not related with this PR, do you mind to restore them

You're right — I forgot to mention that the changes are related to running ament_uncrustify --reformat. After I finished working on each C++ file, I applied the reformatting using that command. Do you still want me to revert the changes?

we are not using uncrustify in gazebo. Yes, restore the changes

I'm sorry for that, i restore changes now.

Signed-off-by: khaledgabr77 <[email protected]>

khaledgabr77 · 2025-04-30T14:20:46Z

I’ve restored the changes and will review the updates you requested later tomorrow.

ahcorde

some linters are failing

  /github/workspace/src/gz.cc:468:  Lines should be <= 80 characters long  [whitespace/line_length] [2]
  /github/workspace/src/gz.cc:515:  If/else bodies with multiple statements require braces  [readability/braces] [4]
  /github/workspace/src/gz.cc:516:  Lines should be <= 80 characters long  [whitespace/line_length] [2]
  /github/workspace/src/gz.cc:559:  If/else bodies with multiple statements require braces  [readability/braces] [4]
  /github/workspace/src/gz.cc:560:  Lines should be <= 80 characters long  [whitespace/line_length] [2]
  /github/workspace/src/gz.cc:658:  Lines should be <= 80 characters long  [whitespace/line_length] [2]
  /github/workspace/src/gz.hh:65:  Lines should be <= 80 characters long  [whitespace/line_length] [2]

khaledgabr77 added 2 commits April 25, 2025 18:28

add --skip-cached to cli

4569fba

Signed-off-by: khaledgabr77 <[email protected]>

add --skip-cached support for individual and collection models and wo…

c64e975

…rlds Signed-off-by: khaledgabr77 <[email protected]>

khaledgabr77 requested a review from nkoenig as a code owner April 25, 2025 19:04

azeey added this to Core development Apr 25, 2025

github-actions bot added 🏛️ ionic Gazebo Ionic 🪵 jetty Gazebo Jetty labels Apr 25, 2025

github-project-automation bot moved this to Inbox in Core development Apr 25, 2025

khaledgabr77 changed the title ~~Feature/skip cached~~ Add a flag --skip-cached to skip already-downloaded models and collections Apr 25, 2025

khaledgabr77 mentioned this pull request Apr 25, 2025

Add option to skip cached models when downloading from command line #150

Open

ahcorde self-requested a review April 28, 2025 22:24

ahcorde requested changes Apr 29, 2025

View reviewed changes

github-project-automation bot moved this from Inbox to In review in Core development Apr 29, 2025

ahcorde requested changes Apr 30, 2025

View reviewed changes

khaledgabr77 force-pushed the feature/skip-cached branch from 0add1b3 to 053fe4c Compare April 30, 2025 14:00

update the Bash Completion Script

f3860b9

Signed-off-by: khaledgabr77 <[email protected]>

khaledgabr77 force-pushed the feature/skip-cached branch from 053fe4c to f3860b9 Compare April 30, 2025 14:03

ahcorde requested changes Apr 30, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add a flag --skip-cached to skip already-downloaded models and collections #461

Add a flag --skip-cached to skip already-downloaded models and collections #461

Uh oh!

khaledgabr77 commented Apr 25, 2025

Uh oh!

ahcorde left a comment

Uh oh!

ahcorde Apr 29, 2025

Uh oh!

ahcorde Apr 29, 2025

Uh oh!

ahcorde Apr 29, 2025

Uh oh!

khaledgabr77 May 4, 2025

Uh oh!

khaledgabr77 commented Apr 30, 2025

Uh oh!

ahcorde left a comment

Uh oh!

khaledgabr77 commented Apr 30, 2025

Uh oh!

khaledgabr77 commented Apr 30, 2025

Uh oh!

ahcorde left a comment

Uh oh!

Uh oh!

	options['skip_cached'] = 'true'
	options['skip_cached'] = true

	Importer.extern 'int downloadUrl(const char , const char , const char , const char , unsigned int, const char *)'
	Importer.extern 'int downloadUrl(const char , const char , const char , const char , unsigned int, bool)'

Add a flag --skip-cached to skip already-downloaded models and collections #461

Are you sure you want to change the base?

Add a flag --skip-cached to skip already-downloaded models and collections #461

Uh oh!

Conversation

khaledgabr77 commented Apr 25, 2025

🎉 New feature

Summary

Behaviour before this PR

Behaviour after this PR

Test it

Checklist

Uh oh!

ahcorde left a comment

Choose a reason for hiding this comment

Uh oh!

ahcorde Apr 29, 2025

Choose a reason for hiding this comment

Uh oh!

ahcorde Apr 29, 2025

Choose a reason for hiding this comment

Uh oh!

ahcorde Apr 29, 2025

Choose a reason for hiding this comment

Uh oh!

khaledgabr77 May 4, 2025

Choose a reason for hiding this comment

Uh oh!

khaledgabr77 commented Apr 30, 2025

Uh oh!

ahcorde left a comment

Choose a reason for hiding this comment

Uh oh!

khaledgabr77 commented Apr 30, 2025

Uh oh!

khaledgabr77 commented Apr 30, 2025

Uh oh!

ahcorde left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!