Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

building error after cloning #412

Open
IroniumStudios opened this issue Jan 19, 2025 · 12 comments
Open

building error after cloning #412

IroniumStudios opened this issue Jan 19, 2025 · 12 comments

Comments

@IroniumStudios
Copy link

IroniumStudios commented Jan 19, 2025

every time i build the project i get this

C:\Windows\System32>cd D:\ungoogled-chromium-windows


C:\Windows\System32>D:

D:\ungoogled-chromium-windows>py build.py
INFO: Setting up cloning environment
INFO: Cloning chromium source: 131.0.6778.264
remote: Total 0 (delta 0), reused 0 (delta 0)
HEAD is now at 2d05e31515 Roll Chrome Mac Arm PGO Profile
INFO: Cloning depot_tools
remote: Total 0 (delta 0), reused 0 (delta 0)
From https://chromium.googlesource.com/chromium/tools/depot_tools

  • branch 20b9bdcace7ed561d6a75728c85373503473cb6b -> FETCH_HEAD
    HEAD is now at 20b9bdc Roll recipe dependencies (trivial).
    Removing git.bat
    error: patch failed: gclient.py:126
    error: gclient.py: patch does not apply
    error: patch failed: gclient_scm.py:952
    error: gclient_scm.py: patch does not apply
    error: patch failed: gsutil.py:25
    error: gsutil.py: patch does not apply
    Traceback (most recent call last):
    File "D:\ungoogled-chromium-windows\ungoogled-chromium\utils\clone.py", line 333, in
    main()
    File "D:\ungoogled-chromium-windows\ungoogled-chromium\utils\clone.py", line 329, in main
    clone(args)
    File "D:\ungoogled-chromium-windows\ungoogled-chromium\utils\clone.py", line 112, in clone
    run(['git', 'apply'],
    File "C:\Users\Damon\AppData\Local\Programs\Python\Python310\lib\subprocess.py", line 524, in run
    raise CalledProcessError(retcode, process.args,
    subprocess.CalledProcessError: Command '['git', 'apply']' returned non-zero exit status 1.
    Traceback (most recent call last):
    File "D:\ungoogled-chromium-windows\build.py", line 305, in
    main()
    File "D:\ungoogled-chromium-windows\build.py", line 178, in main
    subprocess.run([sys.executable, str(Path('ungoogled-chromium', 'utils', 'clone.py')), '-o', 'build\src', '-p', 'win32' if args.x86 else 'win64'], check=True)
    File "C:\Users\Damon\AppData\Local\Programs\Python\Python310\lib\subprocess.py", line 524, in run
    raise CalledProcessError(retcode, process.args,
    subprocess.CalledProcessError: Command '['C:\Users\Damon\AppData\Local\Programs\Python\Python310\python.exe', 'ungoogled-chromium\utils\clone.py', '-o', 'build\src', '-p', 'win64']' returned non-zero exit status 1.

D:\ungoogled-chromium-windows>


Note: i also used the developer command prompt for vs 2022

i do not know how to fix it at all, i have git, python 3.10 and i have the pip command that was needed i installed all deps so i dont see the problem, could someone please help

@IroniumStudios
Copy link
Author

@teeminus @iskunk or anyone any idea downloaded multible time's

@PF4Public
Copy link

INFO: Cloning chromium source: 131.0.6778.264

You're cloning older version, that's why patches do not apply.

@IroniumStudios
Copy link
Author

INFO: Cloning chromium source: 131.0.6778.264

You're cloning older version, that's why patches do not apply.

But how, i cloned the latest repo then followed the instructions

@PF4Public
Copy link

Try following the building instructions from the README: https://github.com/ungoogled-software/ungoogled-chromium-windows/blob/master/README.md#building-1

@IroniumStudios
Copy link
Author

Try following the building instructions from the README: https://github.com/ungoogled-software/ungoogled-chromium-windows/blob/master/README.md#building-1

i already have

@ansoni-san
Copy link

Try making sure that the submodule at ./ungoogled-chromium also matches the platform repo (this repo).

@IroniumStudios
Copy link
Author

Try making sure that the submodule at ./ungoogled-chromium also matches the platform repo (this repo).

using the command git clone --recurse-submodules https://github.com/ungoogled-software/ungoogled-chromium-windows.git already clones the needed synced submodule

@tangalbert919
Copy link
Contributor

Did you use "git submodule update"? Submodules tend to not update when pulling the main repository.

@b3x206
Copy link

b3x206 commented Mar 1, 2025

I got the same error and attempted to fix it but failed.

I ensured that i had the submodules during git checkout --recurse-submodules 133.0.6943.141-1 with a git submodule update

I ran git submodule update (to ensure that the submodules were indeed there) and checked the git log of the cloned "chromium", submodules and the "ungoogled-chromium-windows" to ensure that the versions match:

Last commit of main repo that i got after cloning and checkout
PS C:\Users\BXPC\Desktop\ungoogled-chromium-windows> git log
commit 257e224e5829e0652449b7a3a22fa067573ff8cd (HEAD, tag: 133.0.6943.141-1.1, origin/master, origin/HEAD, master)
Merge: 1b0ad05 43087db
Author: teeminus <[email protected]>
Date:   Sat Mar 1 07:54:46 2025 +0100

    Merge pull request #425 from teeminus/update
...
Last commit of `./ungoogled-chromium` submodule
PS C:\Users\BXPC\Desktop\ungoogled-chromium-windows\ungoogled-chromium> git log
commit 1602c44d38468e729826e8c1b1522042da159959 (HEAD, tag: 133.0.6943.141-1, origin/master, origin/HEAD, master)
Author: Blaise <[email protected]>
Date:   Tue Feb 25 13:13:20 2025 -0600

    Update to Chromium 133.0.6943.141
...
Last commit of Main chromium repo
PS C:\Users\BXPC\Desktop\ungoogled-chromium-windows\build\src> git log
commit 2a5d6da0d6165d7b107502095a937fe7704fcef6 (HEAD, tag: 133.0.6943.141)
Author: chromium-autoroll <[email protected]>
Date:   Mon Feb 24 13:51:30 2025 -0800

    Roll Chrome Mac Arm PGO Profile
...

The version tags seemingly match and the initially cloned repo from build.py clones with INFO: Cloning chromium source: 133.0.6943.141. So I decided to check the root problem.

I first attempted to change these lines of python file named clone.py, adding a debugging output and adding extra args to attempt to make the git apply apply regardless of not matching:

diff --git a/utils/clone.py b/utils/clone.py
index 8d711b18..eae337e9 100755
--- a/utils/clone.py
+++ b/utils/clone.py
@@ -107,8 +107,12 @@ def clone(args): # pylint: disable=too-many-branches, too-many-locals, too-many-
     run(['git', 'clean', '-ffdx'], cwd=dtpath, check=True)
     if iswin:
         (dtpath / 'git.bat').write_text('git')
+
+    # Output for debugging
+    get_logger().info(f"CMD apply patch :\n    git apply {Path(__file__).with_name('depot_tools.patch')}\n    On CWD : {dtpath}")
+
     # Apply changes to gclient
-    run(['git', 'apply'],
+    run(['git', 'apply', '--reject', '--whitespace=fix'],
         input=Path(__file__).with_name('depot_tools.patch').read_text().replace(
             'UC_OUT', str(args.output)).replace('UC_STAGING',
                                                 str(ucstaging)).replace('GSUVER', gsuver),

From here, i saw that it rejects the depot_tools.patch patch (the git error was cryptic so i had to figure it out like this) because the downloaded chromium repo's gclient.py and gclient_scm.py is different in some ways to the .patch file or the files were updated(? the line numbers seemingly don't match).

I attempted to recreate the diff file by applying the changes manually to the changed files on a seperate blank git repo:

The custom `depot_tools.patch`
# Changes to gclient that:
#   move dotfiles into the staging directory
#   skip cipd binary downloads
#   skip gcs downloads unless its an allowed sysroot
#   replace 'src' in checkout paths with the output directory
#   add flag to specify an allowed sysroot
#   ensure shallow fetches
#   utilize a newer version of gsutil to support later versions of python
--- a/gclient.py
+++ b/gclient.py
@@ -126,8 +126,8 @@ DEPOT_TOOLS_DIR = os.path.dirname(os.pat
 # one, e.g. if a spec explicitly says `cache_dir = None`.)
 UNSET_CACHE_DIR = object()
 
-PREVIOUS_CUSTOM_VARS_FILE = '.gclient_previous_custom_vars'
-PREVIOUS_SYNC_COMMITS_FILE = '.gclient_previous_sync_commits'
+PREVIOUS_CUSTOM_VARS_FILE = r'UC_STAGING'+os.sep+'.gclient_previous_custom_vars'
+PREVIOUS_SYNC_COMMITS_FILE = r'UC_STAGING'+os.sep+'.gclient_previous_sync_commits'
 
 PREVIOUS_SYNC_COMMITS = 'GCLIENT_PREVIOUS_SYNC_COMMITS'
 
@@ -424,6 +424,7 @@ class Dependency(gclient_utils.WorkItem,
                  protocol='https',
                  git_dependencies_state=gclient_eval.DEPS,
                  print_outbuf=False):
+        if name and name[0:3] == "src": name = r"UC_OUT"+name[3:]
         gclient_utils.WorkItem.__init__(self, name)
         DependencySettings.__init__(self, parent, url, managed, custom_deps,
                                     custom_vars, custom_hooks, deps_file,
@@ -769,6 +770,7 @@ class Dependency(gclient_utils.WorkItem, DependencySettings):
 
             condition = dep_value.get('condition')
             dep_type = dep_value.get('dep_type')
+            if dep_type == 'cipd': continue
 
             if not self._get_option('process_all_deps', False):
                 should_process = should_process and _should_process(condition)
@@ -820,6 +822,13 @@ class Dependency(gclient_utils.WorkItem, DependencySettings):
                     should_process_object = should_process and _should_process(
                         merged_condition)
 
+                    if name != "src/third_party/node/node_modules" and \
+                    (not name.startswith("src/build/linux/") or \
+                    not f"{self._get_option('sysroot', 'None')}-sysroot" in name):
+                        continue
+                    should_process_object = True
+                    merged_condition = 'True'
+
                     gcs_deps.append(
                         GcsDependency(parent=self,
                                       name=name,
@@ -931,6 +940,8 @@ class Dependency(gclient_utils.WorkItem,
 
         self._gn_args_from = local_scope.get('gclient_gn_args_from')
         self._gn_args_file = local_scope.get('gclient_gn_args_file')
+        if self._gn_args_file and self._gn_args_file[0:3] == "src":
+            self._gn_args_file = r"UC_OUT"+self._gn_args_file[3:]
         self._gn_args = local_scope.get('gclient_gn_args', [])
         # It doesn't make sense to set all of these, since setting gn_args_from
         # to another DEPS will make gclient ignore any other local gn_args*
@@ -3967,6 +3978,7 @@ def CMDsync(parser, args):
                       dest='experiments',
                       default=[],
                       help='Which experiments should be enabled.')
+    parser.add_option('--sysroot')
     (options, args) = parser.parse_args(args)
     client = GClient.LoadCurrentConfig(options)
 
--- a/gclient_scm.py
+++ b/gclient_scm.py
@@ -971,8 +971,7 @@ class GitWrapper(SCMWrapper):
         self._SetFetchConfig(options)
 
         # Fetch upstream if we don't already have |revision|.
-        if not scm.GIT.IsValidRevision(
-                self.checkout_path, revision, sha_only=True):
+        if False:
             self._Fetch(options, prune=options.force)
 
             if not scm.GIT.IsValidRevision(
@@ -988,7 +987,7 @@ class GitWrapper(SCMWrapper):
 
         # This is a big hammer, debatable if it should even be here...
         if options.force or options.reset:
-            target = 'HEAD'
+            target = 'FETCH_HEAD'
             if options.upstream and upstream_branch:
                 target = upstream_branch
             self._Scrub(target, options)
@@ -1003,7 +1002,6 @@ class GitWrapper(SCMWrapper):
             # to the checkout step.
             if not (options.force or options.reset):
                 self._CheckClean(revision)
-            self._CheckDetachedHead(revision, options)
 
             if not current_revision:
                 current_revision = self._Capture(
@@ -1687,8 +1685,7 @@ class GitWrapper(SCMWrapper):
             fetch_cmd.append('--no-tags')
         elif quiet:
             fetch_cmd.append('--quiet')
-        if depth:
-            fetch_cmd.append('--depth=' + str(depth))
+        fetch_cmd.append('--depth=1')
         self._Run(fetch_cmd, options, show_header=options.verbose, retry=True)
 
     def _SetFetchConfig(self, options):
--- a/gsutil.py
+++ b/gsutil.py
@@ -25,7 +25,7 @@ DEFAULT_BIN_DIR = os.path.join(THIS_DIR,
 
 IS_WINDOWS = os.name == 'nt'
 
-VERSION = '4.68'
+VERSION = 'GSUVER'
 
 # Google OAuth Context required by gsutil.
 LUCI_AUTH_SCOPES = [

But for some reason, it fails again:

build.py output with this depot_tools.patch
C:\Users\BXPC\Desktop\ungoogled-chromium-windows>python build.py
INFO: Setting up cloning environment
INFO: Cloning chromium source: 133.0.6943.141
remote: Total 0 (delta 0), reused 0 (delta 0)
HEAD is now at 2a5d6da0d6 Roll Chrome Mac Arm PGO Profile
INFO: Cloning depot_tools
remote: Total 0 (delta 0), reused 0 (delta 0)
From https://chromium.googlesource.com/chromium/tools/depot_tools
 * branch            423f1e1914ab4aa7b2bdf804e216d4c097853ba2 -> FETCH_HEAD
HEAD is now at 423f1e1 pylint: support versioned pylintrc files
Removing git.bat
CMD apply patch :
    git apply C:\Users\BXPC\Desktop\ungoogled-chromium-windows\ungoogled-chromium\utils\depot_tools.patch
    On CWD : build\src\uc_staging\depot_tools
error: patch failed: gclient.py:126
error: gclient.py: patch does not apply
error: patch failed: gclient_scm.py:971
error: gclient_scm.py: patch does not apply
error: patch failed: gsutil.py:25
error: gsutil.py: patch does not apply
Traceback (most recent call last):
  File "C:\Users\BXPC\Desktop\ungoogled-chromium-windows\ungoogled-chromium\utils\clone.py", line 354, in <module>
    main()
  File "C:\Users\BXPC\Desktop\ungoogled-chromium-windows\ungoogled-chromium\utils\clone.py", line 350, in main
    clone(args)
  File "C:\Users\BXPC\Desktop\ungoogled-chromium-windows\ungoogled-chromium\utils\clone.py", line 117, in clone
    run(['git', 'apply'],
  File "C:\Users\BXPC\AppData\Local\Programs\Python\Python310\lib\subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['git', 'apply']' returned non-zero exit status 1.
Traceback (most recent call last):
  File "C:\Users\BXPC\Desktop\ungoogled-chromium-windows\build.py", line 310, in <module>
    main()
  File "C:\Users\BXPC\Desktop\ungoogled-chromium-windows\build.py", line 181, in main
    subprocess.run([sys.executable, str(Path('ungoogled-chromium', 'utils', 'clone.py')), '-o', 'build\\src', '-p', 'win32' if args.x86 else 'win64'], check=True)
  File "C:\Users\BXPC\AppData\Local\Programs\Python\Python310\lib\subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['C:\\Users\\BXPC\\AppData\\Local\\Programs\\Python\\Python310\\python.exe', 'ungoogled-chromium\\utils\\clone.py', '-o', 'build\\src', '-p', 'win64']' returned non-zero exit status 1.

C:\Users\BXPC\Desktop\ungoogled-chromium-windows>REM with git apply --reject --whitespace=fix

C:\Users\BXPC\Desktop\ungoogled-chromium-windows>python build.py
INFO: Setting up cloning environment
INFO: Cloning chromium source: 133.0.6943.141
remote: Total 0 (delta 0), reused 0 (delta 0)
HEAD is now at 2a5d6da0d6 Roll Chrome Mac Arm PGO Profile
INFO: Cloning depot_tools
remote: Total 0 (delta 0), reused 0 (delta 0)
From https://chromium.googlesource.com/chromium/tools/depot_tools
 * branch            423f1e1914ab4aa7b2bdf804e216d4c097853ba2 -> FETCH_HEAD
HEAD is now at 423f1e1 pylint: support versioned pylintrc files
Removing git.bat
CMD apply patch :
    git apply C:\Users\BXPC\Desktop\ungoogled-chromium-windows\ungoogled-chromium\utils\depot_tools.patch
    On CWD : build\src\uc_staging\depot_tools
Checking patch gclient.py...
error: while searching for:
# one, e.g. if a spec explicitly says `cache_dir = None`.)?
UNSET_CACHE_DIR = object()?
?
PREVIOUS_CUSTOM_VARS_FILE = '.gclient_previous_custom_vars'?
PREVIOUS_SYNC_COMMITS_FILE = '.gclient_previous_sync_commits'?
?
PREVIOUS_SYNC_COMMITS = 'GCLIENT_PREVIOUS_SYNC_COMMITS'?
?

error: patch failed: gclient.py:126
error: while searching for:
                 protocol='https',?
                 git_dependencies_state=gclient_eval.DEPS,?
                 print_outbuf=False):?
        gclient_utils.WorkItem.__init__(self, name)?
        DependencySettings.__init__(self, parent, url, managed, custom_deps,?
                                    custom_vars, custom_hooks, deps_file,?

error: patch failed: gclient.py:424
error: while searching for:
?
            condition = dep_value.get('condition')?
            dep_type = dep_value.get('dep_type')?
?
            if not self._get_option('process_all_deps', False):?
                should_process = should_process and _should_process(condition)?

error: patch failed: gclient.py:769
error: while searching for:
                    should_process_object = should_process and _should_process(?
                        merged_condition)?
?
                    gcs_deps.append(?
                        GcsDependency(parent=self,?
                                      name=name,?

error: patch failed: gclient.py:820
error: while searching for:
?
        self._gn_args_from = local_scope.get('gclient_gn_args_from')?
        self._gn_args_file = local_scope.get('gclient_gn_args_file')?
        self._gn_args = local_scope.get('gclient_gn_args', [])?
        # It doesn't make sense to set all of these, since setting gn_args_from?
        # to another DEPS will make gclient ignore any other local gn_args*?

error: patch failed: gclient.py:931
error: while searching for:
                      dest='experiments',?
                      default=[],?
                      help='Which experiments should be enabled.')?
    (options, args) = parser.parse_args(args)?
    client = GClient.LoadCurrentConfig(options)?
?

error: patch failed: gclient.py:3967
Checking patch gclient_scm.py...
error: while searching for:
        self._SetFetchConfig(options)?
?
        # Fetch upstream if we don't already have |revision|.?
        if not scm.GIT.IsValidRevision(?
                self.checkout_path, revision, sha_only=True):?
            self._Fetch(options, prune=options.force)?
?
            if not scm.GIT.IsValidRevision(?

error: patch failed: gclient_scm.py:971
error: while searching for:
?
        # This is a big hammer, debatable if it should even be here...?
        if options.force or options.reset:?
            target = 'HEAD'?
            if options.upstream and upstream_branch:?
                target = upstream_branch?
            self._Scrub(target, options)?

error: patch failed: gclient_scm.py:988
error: while searching for:
            # to the checkout step.?
            if not (options.force or options.reset):?
                self._CheckClean(revision)?
            self._CheckDetachedHead(revision, options)?
?
            if not current_revision:?
                current_revision = self._Capture(?

error: patch failed: gclient_scm.py:1003
error: while searching for:
            fetch_cmd.append('--no-tags')?
        elif quiet:?
            fetch_cmd.append('--quiet')?
        if depth:?
            fetch_cmd.append('--depth=' + str(depth))?
        self._Run(fetch_cmd, options, show_header=options.verbose, retry=True)?
?
    def _SetFetchConfig(self, options):?

error: patch failed: gclient_scm.py:1687
Checking patch gsutil.py...
error: while searching for:
?
IS_WINDOWS = os.name == 'nt'?
?
VERSION = '4.68'?
?
# Google OAuth Context required by gsutil.?
LUCI_AUTH_SCOPES = [?

error: patch failed: gsutil.py:25
Applying patch gclient.py with 6 rejects...
Rejected hunk #1.
Rejected hunk #2.
Rejected hunk #3.
Rejected hunk #4.
Rejected hunk #5.
Rejected hunk #6.
Applying patch gclient_scm.py with 4 rejects...
Rejected hunk #1.
Rejected hunk #2.
Rejected hunk #3.
Rejected hunk #4.
Applying patch gsutil.py with 1 reject...
Rejected hunk #1.
Traceback (most recent call last):
  File "C:\Users\BXPC\Desktop\ungoogled-chromium-windows\ungoogled-chromium\utils\clone.py", line 355, in <module>
    main()
  File "C:\Users\BXPC\Desktop\ungoogled-chromium-windows\ungoogled-chromium\utils\clone.py", line 351, in main
    clone(args)
  File "C:\Users\BXPC\Desktop\ungoogled-chromium-windows\ungoogled-chromium\utils\clone.py", line 118, in clone
    run(['git', 'apply', '--reject', '--whitespace=fix'],
  File "C:\Users\BXPC\AppData\Local\Programs\Python\Python310\lib\subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['git', 'apply', '--reject', '--whitespace=fix']' returned non-zero exit status 1.
Traceback (most recent call last):
  File "C:\Users\BXPC\Desktop\ungoogled-chromium-windows\build.py", line 310, in <module>
    main()
  File "C:\Users\BXPC\Desktop\ungoogled-chromium-windows\build.py", line 181, in main
    subprocess.run([sys.executable, str(Path('ungoogled-chromium', 'utils', 'clone.py')), '-o', 'build\\src', '-p', 'win32' if args.x86 else 'win64'], check=True)
  File "C:\Users\BXPC\AppData\Local\Programs\Python\Python310\lib\subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['C:\\Users\\BXPC\\AppData\\Local\\Programs\\Python\\Python310\\python.exe', 'ungoogled-chromium\\utils\\clone.py', '-o', 'build\\src', '-p', 'win64']' returned non-zero exit status 1.

As the troubleshooting guide says, I deleted everything under the build file (as the download succeeds but build.py fails on a git command) but it did not help.

(either with the unmodified depot_tools.patch or with my modified depot_tools.patch)

So the depot_tools.patch is broken in some way in newer version or the cloned depot_tools repo is broken/incompatible? I am unsure though as I couldn't get it to work.

@IroniumStudios
Copy link
Author

IroniumStudios commented Mar 2, 2025

I got the same error and attempted to fix it but failed.

I ensured that i had the submodules during git checkout --recurse-submodules 133.0.6943.141-1 with a git submodule update

I ran git submodule update (to ensure that the submodules were indeed there) and checked the git log of the cloned "chromium", submodules and the "ungoogled-chromium-windows" to ensure that the versions match:

Last commit of main repo that i got after cloning and checkout
PS C:\Users\BXPC\Desktop\ungoogled-chromium-windows> git log
commit 257e224e5829e0652449b7a3a22fa067573ff8cd (HEAD, tag: 133.0.6943.141-1.1, origin/master, origin/HEAD, master)
Merge: 1b0ad05 43087db
Author: teeminus <[email protected]>
Date:   Sat Mar 1 07:54:46 2025 +0100

    Merge pull request #425 from teeminus/update
...
Last commit of `./ungoogled-chromium` submodule
PS C:\Users\BXPC\Desktop\ungoogled-chromium-windows\ungoogled-chromium> git log
commit 1602c44d38468e729826e8c1b1522042da159959 (HEAD, tag: 133.0.6943.141-1, origin/master, origin/HEAD, master)
Author: Blaise <[email protected]>
Date:   Tue Feb 25 13:13:20 2025 -0600

    Update to Chromium 133.0.6943.141
...
Last commit of Main chromium repo
PS C:\Users\BXPC\Desktop\ungoogled-chromium-windows\build\src> git log
commit 2a5d6da0d6165d7b107502095a937fe7704fcef6 (HEAD, tag: 133.0.6943.141)
Author: chromium-autoroll <[email protected]>
Date:   Mon Feb 24 13:51:30 2025 -0800

    Roll Chrome Mac Arm PGO Profile
...

The version tags seemingly match and the initially cloned repo from build.py clones with INFO: Cloning chromium source: 133.0.6943.141. So I decided to check the root problem.

I first attempted to change these lines of python file named clone.py, adding a debugging output and adding extra args to attempt to make the git apply apply regardless of not matching:

diff --git a/utils/clone.py b/utils/clone.py
index 8d711b18..eae337e9 100755
--- a/utils/clone.py
+++ b/utils/clone.py
@@ -107,8 +107,12 @@ def clone(args): # pylint: disable=too-many-branches, too-many-locals, too-many-
     run(['git', 'clean', '-ffdx'], cwd=dtpath, check=True)
     if iswin:
         (dtpath / 'git.bat').write_text('git')
+
+    # Output for debugging
+    get_logger().info(f"CMD apply patch :\n    git apply {Path(__file__).with_name('depot_tools.patch')}\n    On CWD : {dtpath}")
+
     # Apply changes to gclient
-    run(['git', 'apply'],
+    run(['git', 'apply', '--reject', '--whitespace=fix'],
         input=Path(__file__).with_name('depot_tools.patch').read_text().replace(
             'UC_OUT', str(args.output)).replace('UC_STAGING',
                                                 str(ucstaging)).replace('GSUVER', gsuver),

From here, i saw that it rejects the depot_tools.patch patch (the git error was cryptic so i had to figure it out like this) because the downloaded chromium repo's gclient.py and gclient_scm.py is different in some ways to the .patch file or the files were updated(? the line numbers seemingly don't match).

I attempted to recreate the diff file by applying the changes manually to the changed files on a seperate blank git repo:

The custom `depot_tools.patch`
# Changes to gclient that:
#   move dotfiles into the staging directory
#   skip cipd binary downloads
#   skip gcs downloads unless its an allowed sysroot
#   replace 'src' in checkout paths with the output directory
#   add flag to specify an allowed sysroot
#   ensure shallow fetches
#   utilize a newer version of gsutil to support later versions of python
--- a/gclient.py
+++ b/gclient.py
@@ -126,8 +126,8 @@ DEPOT_TOOLS_DIR = os.path.dirname(os.pat
 # one, e.g. if a spec explicitly says `cache_dir = None`.)
 UNSET_CACHE_DIR = object()
 
-PREVIOUS_CUSTOM_VARS_FILE = '.gclient_previous_custom_vars'
-PREVIOUS_SYNC_COMMITS_FILE = '.gclient_previous_sync_commits'
+PREVIOUS_CUSTOM_VARS_FILE = r'UC_STAGING'+os.sep+'.gclient_previous_custom_vars'
+PREVIOUS_SYNC_COMMITS_FILE = r'UC_STAGING'+os.sep+'.gclient_previous_sync_commits'
 
 PREVIOUS_SYNC_COMMITS = 'GCLIENT_PREVIOUS_SYNC_COMMITS'
 
@@ -424,6 +424,7 @@ class Dependency(gclient_utils.WorkItem,
                  protocol='https',
                  git_dependencies_state=gclient_eval.DEPS,
                  print_outbuf=False):
+        if name and name[0:3] == "src": name = r"UC_OUT"+name[3:]
         gclient_utils.WorkItem.__init__(self, name)
         DependencySettings.__init__(self, parent, url, managed, custom_deps,
                                     custom_vars, custom_hooks, deps_file,
@@ -769,6 +770,7 @@ class Dependency(gclient_utils.WorkItem, DependencySettings):
 
             condition = dep_value.get('condition')
             dep_type = dep_value.get('dep_type')
+            if dep_type == 'cipd': continue
 
             if not self._get_option('process_all_deps', False):
                 should_process = should_process and _should_process(condition)
@@ -820,6 +822,13 @@ class Dependency(gclient_utils.WorkItem, DependencySettings):
                     should_process_object = should_process and _should_process(
                         merged_condition)
 
+                    if name != "src/third_party/node/node_modules" and \
+                    (not name.startswith("src/build/linux/") or \
+                    not f"{self._get_option('sysroot', 'None')}-sysroot" in name):
+                        continue
+                    should_process_object = True
+                    merged_condition = 'True'
+
                     gcs_deps.append(
                         GcsDependency(parent=self,
                                       name=name,
@@ -931,6 +940,8 @@ class Dependency(gclient_utils.WorkItem,
 
         self._gn_args_from = local_scope.get('gclient_gn_args_from')
         self._gn_args_file = local_scope.get('gclient_gn_args_file')
+        if self._gn_args_file and self._gn_args_file[0:3] == "src":
+            self._gn_args_file = r"UC_OUT"+self._gn_args_file[3:]
         self._gn_args = local_scope.get('gclient_gn_args', [])
         # It doesn't make sense to set all of these, since setting gn_args_from
         # to another DEPS will make gclient ignore any other local gn_args*
@@ -3967,6 +3978,7 @@ def CMDsync(parser, args):
                       dest='experiments',
                       default=[],
                       help='Which experiments should be enabled.')
+    parser.add_option('--sysroot')
     (options, args) = parser.parse_args(args)
     client = GClient.LoadCurrentConfig(options)
 
--- a/gclient_scm.py
+++ b/gclient_scm.py
@@ -971,8 +971,7 @@ class GitWrapper(SCMWrapper):
         self._SetFetchConfig(options)
 
         # Fetch upstream if we don't already have |revision|.
-        if not scm.GIT.IsValidRevision(
-                self.checkout_path, revision, sha_only=True):
+        if False:
             self._Fetch(options, prune=options.force)
 
             if not scm.GIT.IsValidRevision(
@@ -988,7 +987,7 @@ class GitWrapper(SCMWrapper):
 
         # This is a big hammer, debatable if it should even be here...
         if options.force or options.reset:
-            target = 'HEAD'
+            target = 'FETCH_HEAD'
             if options.upstream and upstream_branch:
                 target = upstream_branch
             self._Scrub(target, options)
@@ -1003,7 +1002,6 @@ class GitWrapper(SCMWrapper):
             # to the checkout step.
             if not (options.force or options.reset):
                 self._CheckClean(revision)
-            self._CheckDetachedHead(revision, options)
 
             if not current_revision:
                 current_revision = self._Capture(
@@ -1687,8 +1685,7 @@ class GitWrapper(SCMWrapper):
             fetch_cmd.append('--no-tags')
         elif quiet:
             fetch_cmd.append('--quiet')
-        if depth:
-            fetch_cmd.append('--depth=' + str(depth))
+        fetch_cmd.append('--depth=1')
         self._Run(fetch_cmd, options, show_header=options.verbose, retry=True)
 
     def _SetFetchConfig(self, options):
--- a/gsutil.py
+++ b/gsutil.py
@@ -25,7 +25,7 @@ DEFAULT_BIN_DIR = os.path.join(THIS_DIR,
 
 IS_WINDOWS = os.name == 'nt'
 
-VERSION = '4.68'
+VERSION = 'GSUVER'
 
 # Google OAuth Context required by gsutil.
 LUCI_AUTH_SCOPES = [

But for some reason, it fails again:

build.py output with this depot_tools.patch
C:\Users\BXPC\Desktop\ungoogled-chromium-windows>python build.py
INFO: Setting up cloning environment
INFO: Cloning chromium source: 133.0.6943.141
remote: Total 0 (delta 0), reused 0 (delta 0)
HEAD is now at 2a5d6da0d6 Roll Chrome Mac Arm PGO Profile
INFO: Cloning depot_tools
remote: Total 0 (delta 0), reused 0 (delta 0)
From https://chromium.googlesource.com/chromium/tools/depot_tools
 * branch            423f1e1914ab4aa7b2bdf804e216d4c097853ba2 -> FETCH_HEAD
HEAD is now at 423f1e1 pylint: support versioned pylintrc files
Removing git.bat
CMD apply patch :
    git apply C:\Users\BXPC\Desktop\ungoogled-chromium-windows\ungoogled-chromium\utils\depot_tools.patch
    On CWD : build\src\uc_staging\depot_tools
error: patch failed: gclient.py:126
error: gclient.py: patch does not apply
error: patch failed: gclient_scm.py:971
error: gclient_scm.py: patch does not apply
error: patch failed: gsutil.py:25
error: gsutil.py: patch does not apply
Traceback (most recent call last):
  File "C:\Users\BXPC\Desktop\ungoogled-chromium-windows\ungoogled-chromium\utils\clone.py", line 354, in <module>
    main()
  File "C:\Users\BXPC\Desktop\ungoogled-chromium-windows\ungoogled-chromium\utils\clone.py", line 350, in main
    clone(args)
  File "C:\Users\BXPC\Desktop\ungoogled-chromium-windows\ungoogled-chromium\utils\clone.py", line 117, in clone
    run(['git', 'apply'],
  File "C:\Users\BXPC\AppData\Local\Programs\Python\Python310\lib\subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['git', 'apply']' returned non-zero exit status 1.
Traceback (most recent call last):
  File "C:\Users\BXPC\Desktop\ungoogled-chromium-windows\build.py", line 310, in <module>
    main()
  File "C:\Users\BXPC\Desktop\ungoogled-chromium-windows\build.py", line 181, in main
    subprocess.run([sys.executable, str(Path('ungoogled-chromium', 'utils', 'clone.py')), '-o', 'build\\src', '-p', 'win32' if args.x86 else 'win64'], check=True)
  File "C:\Users\BXPC\AppData\Local\Programs\Python\Python310\lib\subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['C:\\Users\\BXPC\\AppData\\Local\\Programs\\Python\\Python310\\python.exe', 'ungoogled-chromium\\utils\\clone.py', '-o', 'build\\src', '-p', 'win64']' returned non-zero exit status 1.

C:\Users\BXPC\Desktop\ungoogled-chromium-windows>REM with git apply --reject --whitespace=fix

C:\Users\BXPC\Desktop\ungoogled-chromium-windows>python build.py
INFO: Setting up cloning environment
INFO: Cloning chromium source: 133.0.6943.141
remote: Total 0 (delta 0), reused 0 (delta 0)
HEAD is now at 2a5d6da0d6 Roll Chrome Mac Arm PGO Profile
INFO: Cloning depot_tools
remote: Total 0 (delta 0), reused 0 (delta 0)
From https://chromium.googlesource.com/chromium/tools/depot_tools
 * branch            423f1e1914ab4aa7b2bdf804e216d4c097853ba2 -> FETCH_HEAD
HEAD is now at 423f1e1 pylint: support versioned pylintrc files
Removing git.bat
CMD apply patch :
    git apply C:\Users\BXPC\Desktop\ungoogled-chromium-windows\ungoogled-chromium\utils\depot_tools.patch
    On CWD : build\src\uc_staging\depot_tools
Checking patch gclient.py...
error: while searching for:
# one, e.g. if a spec explicitly says `cache_dir = None`.)?
UNSET_CACHE_DIR = object()?
?
PREVIOUS_CUSTOM_VARS_FILE = '.gclient_previous_custom_vars'?
PREVIOUS_SYNC_COMMITS_FILE = '.gclient_previous_sync_commits'?
?
PREVIOUS_SYNC_COMMITS = 'GCLIENT_PREVIOUS_SYNC_COMMITS'?
?

error: patch failed: gclient.py:126
error: while searching for:
                 protocol='https',?
                 git_dependencies_state=gclient_eval.DEPS,?
                 print_outbuf=False):?
        gclient_utils.WorkItem.__init__(self, name)?
        DependencySettings.__init__(self, parent, url, managed, custom_deps,?
                                    custom_vars, custom_hooks, deps_file,?

error: patch failed: gclient.py:424
error: while searching for:
?
            condition = dep_value.get('condition')?
            dep_type = dep_value.get('dep_type')?
?
            if not self._get_option('process_all_deps', False):?
                should_process = should_process and _should_process(condition)?

error: patch failed: gclient.py:769
error: while searching for:
                    should_process_object = should_process and _should_process(?
                        merged_condition)?
?
                    gcs_deps.append(?
                        GcsDependency(parent=self,?
                                      name=name,?

error: patch failed: gclient.py:820
error: while searching for:
?
        self._gn_args_from = local_scope.get('gclient_gn_args_from')?
        self._gn_args_file = local_scope.get('gclient_gn_args_file')?
        self._gn_args = local_scope.get('gclient_gn_args', [])?
        # It doesn't make sense to set all of these, since setting gn_args_from?
        # to another DEPS will make gclient ignore any other local gn_args*?

error: patch failed: gclient.py:931
error: while searching for:
                      dest='experiments',?
                      default=[],?
                      help='Which experiments should be enabled.')?
    (options, args) = parser.parse_args(args)?
    client = GClient.LoadCurrentConfig(options)?
?

error: patch failed: gclient.py:3967
Checking patch gclient_scm.py...
error: while searching for:
        self._SetFetchConfig(options)?
?
        # Fetch upstream if we don't already have |revision|.?
        if not scm.GIT.IsValidRevision(?
                self.checkout_path, revision, sha_only=True):?
            self._Fetch(options, prune=options.force)?
?
            if not scm.GIT.IsValidRevision(?

error: patch failed: gclient_scm.py:971
error: while searching for:
?
        # This is a big hammer, debatable if it should even be here...?
        if options.force or options.reset:?
            target = 'HEAD'?
            if options.upstream and upstream_branch:?
                target = upstream_branch?
            self._Scrub(target, options)?

error: patch failed: gclient_scm.py:988
error: while searching for:
            # to the checkout step.?
            if not (options.force or options.reset):?
                self._CheckClean(revision)?
            self._CheckDetachedHead(revision, options)?
?
            if not current_revision:?
                current_revision = self._Capture(?

error: patch failed: gclient_scm.py:1003
error: while searching for:
            fetch_cmd.append('--no-tags')?
        elif quiet:?
            fetch_cmd.append('--quiet')?
        if depth:?
            fetch_cmd.append('--depth=' + str(depth))?
        self._Run(fetch_cmd, options, show_header=options.verbose, retry=True)?
?
    def _SetFetchConfig(self, options):?

error: patch failed: gclient_scm.py:1687
Checking patch gsutil.py...
error: while searching for:
?
IS_WINDOWS = os.name == 'nt'?
?
VERSION = '4.68'?
?
# Google OAuth Context required by gsutil.?
LUCI_AUTH_SCOPES = [?

error: patch failed: gsutil.py:25
Applying patch gclient.py with 6 rejects...
Rejected hunk #1.
Rejected hunk #2.
Rejected hunk #3.
Rejected hunk #4.
Rejected hunk #5.
Rejected hunk #6.
Applying patch gclient_scm.py with 4 rejects...
Rejected hunk #1.
Rejected hunk #2.
Rejected hunk #3.
Rejected hunk #4.
Applying patch gsutil.py with 1 reject...
Rejected hunk #1.
Traceback (most recent call last):
  File "C:\Users\BXPC\Desktop\ungoogled-chromium-windows\ungoogled-chromium\utils\clone.py", line 355, in <module>
    main()
  File "C:\Users\BXPC\Desktop\ungoogled-chromium-windows\ungoogled-chromium\utils\clone.py", line 351, in main
    clone(args)
  File "C:\Users\BXPC\Desktop\ungoogled-chromium-windows\ungoogled-chromium\utils\clone.py", line 118, in clone
    run(['git', 'apply', '--reject', '--whitespace=fix'],
  File "C:\Users\BXPC\AppData\Local\Programs\Python\Python310\lib\subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['git', 'apply', '--reject', '--whitespace=fix']' returned non-zero exit status 1.
Traceback (most recent call last):
  File "C:\Users\BXPC\Desktop\ungoogled-chromium-windows\build.py", line 310, in <module>
    main()
  File "C:\Users\BXPC\Desktop\ungoogled-chromium-windows\build.py", line 181, in main
    subprocess.run([sys.executable, str(Path('ungoogled-chromium', 'utils', 'clone.py')), '-o', 'build\\src', '-p', 'win32' if args.x86 else 'win64'], check=True)
  File "C:\Users\BXPC\AppData\Local\Programs\Python\Python310\lib\subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['C:\\Users\\BXPC\\AppData\\Local\\Programs\\Python\\Python310\\python.exe', 'ungoogled-chromium\\utils\\clone.py', '-o', 'build\\src', '-p', 'win64']' returned non-zero exit status 1.

As the troubleshooting guide says, I deleted everything under the build file (as the download succeeds but build.py fails on a git command) but it did not help.

(either with the unmodified depot_tools.patch or with my modified depot_tools.patch)

So the depot_tools.patch is broken in some way in newer version or the cloned depot_tools repo is broken/incompatible? I am unsure though as I couldn't get it to work.

Thank you for the detailed response. I do appreciate that, to me seems like an issue that they need to fix and I don't see how they're not aware of this considering that it's a very big problem and affects how the project builds.

@iskunk
Copy link
Contributor

iskunk commented Mar 2, 2025

So the depot_tools.patch is broken in some way in newer version or the cloned depot_tools repo is broken/incompatible? I am unsure though as I couldn't get it to work.

I'm not seeing that... if I clone the depot_tools repo, switch to commit 423f1e1914ab4aa7b2bdf804e216d4c097853ba2 (as listed in the top-level DEPS file for the current .141 release), and then take the patch from the latest u-c commit that you linked above...

$ patch -p1 --dry-run < /path/to/ungoogled-chromium/utils/depot_tools.patch 
checking file gclient.py
Hunk #6 succeeded at 3977 (offset 32 lines).
checking file gclient_scm.py
Hunk #1 succeeded at 971 (offset 19 lines).
Hunk #2 succeeded at 987 (offset 19 lines).
Hunk #3 succeeded at 1002 (offset 19 lines).
Hunk #4 succeeded at 1685 (offset 19 lines).
checking file gsutil.py

The patch could use a refresh, but there are no conflicts. What is the commit ID of depot_tools that you are working with?

@b3x206
Copy link

b3x206 commented Mar 2, 2025

It states that it's 423f1e1914ab4aa7b2bdf804e216d4c097853ba2:

PS C:\Users\BXPC\Desktop\ungoogled-chromium-windows\build\src> git submodule status --recursive | Select-String "depot_tools"

-423f1e1914ab4aa7b2bdf804e216d4c097853ba2 third_party/depot_tools

I tried to apply the patch, it succeeds (patch.exe is from scoop):

PS C:\Users\BXPC\Desktop\ungoogled-chromium-windows\build\src\uc_staging\depot_tools> Get-Content C:\Users\BXPC\Desktop\ungoogled-chromium-windows\ungoogled-chromium\utils\depot_tools.patch | patch -p1 --dry-run
(Stripping trailing CRs from patch.)
patching file gclient.py
Hunk #6 succeeded at 3977 (offset 32 lines).
(Stripping trailing CRs from patch.)
patching file gclient_scm.py
Hunk #1 succeeded at 971 (offset 19 lines).
Hunk #2 succeeded at 987 (offset 19 lines).
Hunk #3 succeeded at 1002 (offset 19 lines).
Hunk #4 succeeded at 1685 (offset 19 lines).
(Stripping trailing CRs from patch.)
patching file gsutil.py

Interesting, so the file is not faulty. At this point I didn't know why the git in the python failed.

I tried updating git to latest version (i was using "2.38.0") but didn't help.

Using git directly or feeding stdin with Get-Content -Raw works :

PS C:\Users\BXPC\Desktop\ungoogled-chromium-windows\build\src\uc_staging\depot_tools> git --version
git version 2.48.1.windows.1
PS C:\Users\BXPC\Desktop\ungoogled-chromium-windows\build\src\uc_staging\depot_tools> git apply C:\Users\BXPC\Desktop\ungoogled-chromium-windows\ungoogled-chromium\utils\depot_tools.patch
PS C:\Users\BXPC\Desktop\ungoogled-chromium-windows\build\src\uc_staging\depot_tools> $?
True

(i reset the patched depot_tools files beforehand)

PS C:\Users\BXPC\Desktop\ungoogled-chromium-windows\build\src\uc_staging\depot_tools> Get-Content -Raw C:\Users\BXPC\Desktop\ungoogled-chromium-windows\ungoogled-chromium\utils\depot_tools.patch | git apply
PS C:\Users\BXPC\Desktop\ungoogled-chromium-windows\build\src\uc_staging\depot_tools> $?
True

Using non-raw Get-Content doesn't work (line ending related problem, it appends CRLF)

PS C:\Users\BXPC\Desktop\ungoogled-chromium-windows\build\src\uc_staging\depot_tools> Get-Content C:\Users\BXPC\Desktop\ungoogled-chromium-windows\ungoogled-chromium\utils\depot_tools.patch | git apply
error: patch failed: gclient.py:126
error: gclient.py: patch does not apply
error: patch failed: gclient_scm.py:952
error: gclient_scm.py: patch does not apply
error: patch failed: gsutil.py:25
error: gsutil.py: patch does not apply

VSCode and short xxd inspection confirms that my local copy of depot_tools.patch is LF line ending and file is indeed read as having LF line ending from the python with the patch_file.read_text()

I used a debug file like (patch_file.with_name("debug.patch")).write_bytes(patch_file_data) to check the line endings and it's LF.


But subprocess.run's input argument with a str type and with universal_newline=True seemingly breaks line endings of the file? Because if i change clone.py code to this

diff --git a/utils/clone.py b/utils/clone.py
index 8d711b18..4f95e702 100755
--- a/utils/clone.py
+++ b/utils/clone.py
@@ -107,14 +107,15 @@ def clone(args): # pylint: disable=too-many-branches, too-many-locals, too-many-
     run(['git', 'clean', '-ffdx'], cwd=dtpath, check=True)
     if iswin:
         (dtpath / 'git.bat').write_text('git')
-    # Apply changes to gclient
-    run(['git', 'apply'],
-        input=Path(__file__).with_name('depot_tools.patch').read_text().replace(
+    # Apply changes to gclient, convert patch data to binary
+    patch_file = Path(__file__).with_name('depot_tools.patch')
+    patch_file_data = patch_file.read_text().replace(
             'UC_OUT', str(args.output)).replace('UC_STAGING',
-                                                str(ucstaging)).replace('GSUVER', gsuver),
+                                                str(ucstaging)).replace('GSUVER', gsuver).encode()
+    run(['git', 'apply'],
+        input=patch_file_data,
         cwd=dtpath,
-        check=True,
-        universal_newlines=True)
+        check=True)
 
     # Manualy set up the gsutil directory for newer versions of Python
     get_logger().info('Cloning gsutil')

it makes it work and fixes the git error and the build proceeds beyond this point.

Python version is Python 3.10.8 (tags/v3.10.8:aaaf517, Oct 11 2022, 16:50:30) [MSC v.1933 64 bit (AMD64)] on win32 from python.org if it's any relevant to this issue.


Update

The whole build succeeded after changing clone.py to input the patch file this way (and after downloading 2 versions of Windows SDK, versions 10.0.22621.0 and 10.0.26100.0, I was missing those)

I apologize for the somewhat incorrect suspect, I should have troubleshot the problem better.

Now, would this method of inputting the string to the subprocess cause regressions in other machines or would it not affect anything is the question I have. Thanks for telling me to check the depot_tools.patch and the commit id of the depot_tools

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants