Module: OS::Mac::Keg Private

Extended by:
T::Helpers
Includes:
SystemCommand::Mixin
Included in:
Keg
Defined in:
extend/os/mac/keg.rb,
extend/os/mac/keg_relocate.rb

This module is part of a private API. This module may only be used in the Homebrew/brew repository. Third parties should avoid using this module if possible, as it may be removed or changed without warning.

Defined Under Namespace

Modules: ClassMethods

Constant Summary collapse

VARIABLE_REFERENCE_RX =

This constant is part of a private API. This constant may only be used in the Homebrew/brew repository. Third parties should avoid using this constant if possible, as it may be removed or changed without warning. 

/^@(loader_|executable_|r)path/
FRAMEWORK_RX =

This constant is part of a private API. This constant may only be used in the Homebrew/brew repository. Third parties should avoid using this constant if possible, as it may be removed or changed without warning.

Matches framework references like XXX.framework/Versions/YYY/XXX and XXX.framework/XXX, both with or without a slash-delimited prefix.

%r{(?:^|/)(([^/]+)\.framework/(?:Versions/[^/]+/)?\2)$}

Instance Method Summary collapse

Methods included from SystemCommand::Mixin

#system_command, #system_command!

Instance Method Details

#binary_executable_or_library_files ⇒ Array<MachOShim>

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Returns:



105
 
# File 'extend/os/mac/keg.rb', line 105

def binary_executable_or_library_files = mach_o_files

#change_dylib_id(id, file, write: true) ⇒ Boolean

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Parameters:

Returns:

  • (Boolean) 


43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
 
# File 'extend/os/mac/keg.rb', line 43

def change_dylib_id(id, file, write: true)
  return false if file.dylib_id == id

  require_relocation!
  odebug "Changing dylib ID of #{file}\n  from #{file.dylib_id}\n    to #{id}"
  file.change_dylib_id(id, strict: false, write:)
  true
rescue MachO::MachOError
  onoe <<~EOS
    Failed changing dylib ID of #{file}
      from #{file.dylib_id}
        to #{id}
  EOS
  raise
end

#change_install_name(old, new, file, write: true) ⇒ Boolean

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Parameters:

Returns:

  • (Boolean) 


60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
 
# File 'extend/os/mac/keg.rb', line 60

def change_install_name(old, new, file, write: true)
  return false if old == new

  require_relocation!
  odebug "Changing install name in #{file}\n  from #{old}\n    to #{new}"
  file.change_install_name(old, new, strict: false, write:)
  true
rescue MachO::MachOError
  onoe <<~EOS
    Failed changing install name in #{file}
      from #{old}
        to #{new}
  EOS
  raise
end

#change_rpath(old, new, file, write: true) ⇒ Boolean

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Parameters:

Returns:

  • (Boolean) 


77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
 
# File 'extend/os/mac/keg.rb', line 77

def change_rpath(old, new, file, write: true)
  return false if old == new

  require_relocation!
  odebug "Changing rpath in #{file}\n  from #{old}\n    to #{new}"
  file.change_rpath(old, new, strict: false, write:)
  true
rescue MachO::MachOError
  onoe <<~EOS
    Failed changing rpath in #{file}
      from #{old}
        to #{new}
  EOS
  raise
end

#codesign_patched_binaries(files) ⇒ void

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

This method returns an undefined value.

Parameters:



158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
 
# File 'extend/os/mac/keg.rb', line 158

def codesign_patched_binaries(files)
  return if files.empty?

  # Codesigning shells out on Intel and hashes every page of the file on
  # Apple Silicon, so parallelise it across files.
  queue = Queue.new
  files.each { queue << it }
  queue.close
  Array.new([files.length, ::Hardware::CPU.cores].min) do
    Thread.new do
      while (file = queue.pop)
        # Signing rewrites the file, which may not be user-writable.
        Utils::Path.ensure_writable(file) { codesign_patched_binary(file.to_s) }
      end
    end
  end.each(&:join)
end

#codesign_patched_binary(file) ⇒ void

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

This method returns an undefined value.

Parameters:



108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
 
# File 'extend/os/mac/keg.rb', line 108

def codesign_patched_binary(file)
  unless ::Hardware::CPU.arm?
    # Intel macOS rejects ruby-macho's ad-hoc signatures on larger
    # binaries and does not require unsigned binaries to be signed,
    # so use `codesign` to re-sign only the binaries whose existing
    # signature our modifications have just broken:
    # https://github.com/Homebrew/brew/issues/23418
    result = system_command("codesign", args: ["--verify", file], print_stderr: false)
    return unless result.stderr.match?(/invalid signature/i)

    odebug "Codesigning #{file}"
    return if SystemCommand.quiet_system("codesign", "--sign", "-", "--force",
                                         "--preserve-metadata=entitlements,requirements,flags,runtime",
                                         file)

    # If the codesigning fails, it may be a bug in Apple's codesign utility.
    # A known workaround is to copy the file to another inode, then move it back
    # erasing the previous file. Then sign again.
    Dir::Tmpname.create("workaround") do |tmppath|
      FileUtils.cp file, tmppath
      FileUtils.mv tmppath, file, force: true
    end

    odebug "Codesigning (2nd try) #{file}"
    result = system_command("codesign", args: [
      "--sign", "-", "--force",
      "--preserve-metadata=entitlements,requirements,flags,runtime",
      file
    ], print_stderr: false)
    return if result.success?

    onoe <<~EOS
      Failed applying an ad-hoc signature to #{file}:
      #{result.stderr}
    EOS
    return
  end

  require "macho"

  odebug "Codesigning #{file}"
  MachO.codesign! file
rescue MachO::CodeSigningError => e
  onoe <<~EOS
    Failed applying an ad-hoc signature to #{file}:
    #{e.message}
  EOS
end

#consistent_reproducible_symlink_permissions! ⇒ void

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

This method returns an undefined value.

Needed to make symlink permissions consistent on macOS and Linux for reproducible bottles.



196
197
198
199
200
 
# File 'extend/os/mac/keg.rb', line 196

def consistent_reproducible_symlink_permissions!
  path.find do |file|
    file.lchmod 0777 if file.symlink?
  end
end

#delete_rpath(rpath, file, write: true) ⇒ Boolean

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Parameters:

Returns:

  • (Boolean) 


94
95
96
97
98
99
100
101
102
 
# File 'extend/os/mac/keg.rb', line 94

def delete_rpath(rpath, file, write: true)
  odebug "Deleting rpath #{rpath} in #{file}"
  !file.delete_rpath(rpath, strict: false, write:).nil?
rescue MachO::MachOError
  onoe <<~EOS
    Failed deleting rpath #{rpath} in #{file}
  EOS
  raise
end

#dylib_id_for(file) ⇒ String

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Parameters:

Returns:

Raises:

  • (ArgumentError) 


180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
 
# File 'extend/os/mac/keg_relocate.rb', line 180

def dylib_id_for(file)
  dylib_id = file.dylib_id
  raise ArgumentError, "#{file} has no dylib ID" if dylib_id.nil?

  # Swift dylib IDs should be /usr/lib/swift
  return dylib_id if dylib_id.start_with?("/usr/lib/swift/libswift")

  # Preserve @rpath install names if the formula has specified preserve_rpath
  return dylib_id if dylib_id.start_with?("@rpath") && formula_preserve_rpath?

  # The new dylib ID should have the same basename as the old dylib ID, not
  # the basename of the file itself.
  basename = File.basename(dylib_id)
  relative_dirname = file.dirname.relative_path_from(path)
  (opt_record/relative_dirname/basename).to_s
end

#each_linkage_for(file, linkage_type, resolve_variable_references: false, &block) ⇒ void

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

This method returns an undefined value.

Parameters:

  • file (MachOShim)
  • linkage_type (Symbol)
  • resolve_variable_references (Boolean) (defaults to: false)
  • block (T.proc.params(arg0: String).void) 


173
174
175
176
177
 
# File 'extend/os/mac/keg_relocate.rb', line 173

def each_linkage_for(file, linkage_type, resolve_variable_references: false, &block)
  file.public_send(linkage_type, resolve_variable_references:)
      .grep_v(VARIABLE_REFERENCE_RX)
      .each(&block)
end

#egrep_args ⇒ Array<(String, String)>

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Returns:



290
291
292
293
294
 
# File 'extend/os/mac/keg_relocate.rb', line 290

def egrep_args
  grep_bin = "egrep"
  grep_args = "--files-with-matches"
  [grep_bin, grep_args]
end

#find_dylib(bad_name) ⇒ ::Pathname?

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Parameters:

Returns:



226
227
228
229
230
231
 
# File 'extend/os/mac/keg_relocate.rb', line 226

def find_dylib(bad_name)
  return unless lib.directory?

  suffix = "/#{find_dylib_suffix_from(bad_name)}"
  lib.find { |pn| break pn if pn.to_s.end_with?(suffix) }
end

#find_dylib_suffix_from(bad_name) ⇒ String

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Parameters:

Returns:



221
222
223
 
# File 'extend/os/mac/keg_relocate.rb', line 221

def find_dylib_suffix_from(bad_name)
  bad_name[FRAMEWORK_RX, 1] || File.basename(bad_name)
end

#fix_dynamic_linkage ⇒ void

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

This method returns an undefined value.



79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
 
# File 'extend/os/mac/keg_relocate.rb', line 79

def fix_dynamic_linkage
  fixed_files = []
  mach_o_files.each do |file|
    Utils::Path.ensure_writable(file.to_path) do
      modified = T.let(false, T::Boolean)

      modified = change_dylib_id(dylib_id_for(file), file, write: false) if file.dylib?

      each_linkage_for(file, :dynamically_linked_libraries) do |bad_name|
        # Don't fix absolute paths unless they are rooted in the build directory.
        new_name = if bad_name.start_with?("/") && !rooted_in_build_directory?(bad_name)
          bad_name
        else
          fixed_name(file, bad_name)
        end
        loader_name = loader_name_for(file, new_name)
        if loader_name != bad_name
          modified = change_install_name(bad_name, loader_name, file,
                                         write: false) || modified
        end
      end

      each_linkage_for(file, :rpaths) do |bad_name|
        new_name = opt_name_for(bad_name)
        loader_name = loader_name_for(file, new_name)
        next if loader_name == bad_name

        modified = change_rpath(bad_name, loader_name, file, write: false) || modified
      end

      # Strip duplicate rpaths and rpaths rooted in the build directory.
      # We do this separately from the rpath relocation above to avoid
      # failing to relocate an rpath whose variable duplicate we deleted.
      each_linkage_for(file, :rpaths, resolve_variable_references: true) do |bad_name|
        next if !rooted_in_build_directory?(bad_name) && file.rpaths.count(bad_name) == 1

        modified = delete_rpath(bad_name, file, write: false) || modified
      end

      next unless modified

      # The edits above only mutated the in-memory Mach-O, so persist
      # them in a single write per file.
      file.save_changes
      fixed_files << file
    end
  end

  # Every saved file's signature is now broken, so re-sign each exactly
  # once, parallelised across files.
  codesign_patched_binaries(fixed_files)

  super
end

#fixed_name(file, bad_name) ⇒ String

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

If file is a dylib or bundle itself, look for the dylib named by bad_name relative to the lib directory, so that we can skip the more expensive recursive search if possible.

Parameters:

Returns:



151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
 
# File 'extend/os/mac/keg_relocate.rb', line 151

def fixed_name(file, bad_name)
  if bad_name.start_with? ::Keg::PREFIX_PLACEHOLDER
    bad_name.sub(::Keg::PREFIX_PLACEHOLDER, HOMEBREW_PREFIX)
  elsif bad_name.start_with? ::Keg::CELLAR_PLACEHOLDER
    bad_name.sub(::Keg::CELLAR_PLACEHOLDER, HOMEBREW_CELLAR)
  elsif (file.dylib? || file.mach_o_bundle?) && (file.dirname/bad_name).exist?
    "@loader_path/#{bad_name}"
  elsif file.mach_o_executable? && (lib/bad_name).exist?
    "#{lib}/#{bad_name}"
  elsif file.mach_o_executable? && (libexec/"lib"/bad_name).exist?
    "#{libexec}/lib/#{bad_name}"
  elsif (abs_name = find_dylib(bad_name)) && abs_name.exist?
    abs_name.to_s
  else
    opoo "Could not fix #{bad_name} in #{file}"
    bad_name
  end
end

#formula_preserve_rpath? ⇒ Boolean

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Returns:

  • (Boolean) 


198
199
200
201
202
 
# File 'extend/os/mac/keg_relocate.rb', line 198

def formula_preserve_rpath?
  ::Formula[name].preserve_rpath?
rescue FormulaUnavailableError
  false
end

#loader_name_for(file, target) ⇒ String

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Parameters:

Returns:



135
136
137
138
139
140
141
142
143
144
145
 
# File 'extend/os/mac/keg_relocate.rb', line 135

def loader_name_for(file, target)
  # Use @loader_path-relative install names for other Homebrew-installed binaries.
  if ENV["HOMEBREW_RELOCATABLE_INSTALL_NAMES"] && target.start_with?(HOMEBREW_PREFIX)
    dylib_suffix = find_dylib_suffix_from(target)
    target_dir = ::Pathname.new(target.delete_suffix(dylib_suffix)).cleanpath

    "@loader_path/#{target_dir.relative_path_from(file.dirname)/dylib_suffix}"
  else
    target
  end
end

#mach_o_files ⇒ Array<MachOShim>

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Returns:



234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
 
# File 'extend/os/mac/keg_relocate.rb', line 234

def mach_o_files
  hardlinks = Set.new
  mach_o_files = []
  path.find do |pn|
    next if pn.symlink? || pn.directory?

    pn = MachOPathname.wrap(pn)
    next if !pn.dylib? && !pn.mach_o_bundle? && !pn.mach_o_executable?

    # if we've already processed a file, ignore its hardlinks (which have the same dev ID and inode)
    # this prevents relocations from being performed on a binary more than once
    next unless hardlinks.add? [pn.stat.dev, pn.stat.ino]

    mach_o_files << pn
  end

  mach_o_files
end

#prepare_debug_symbols ⇒ void

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

This method returns an undefined value.



177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
 
# File 'extend/os/mac/keg.rb', line 177

def prepare_debug_symbols
  binary_executable_or_library_files.each do |file|
    file = file.to_s
    odebug "Extracting symbols #{file}"

    result = system_command("dsymutil", args: [file], print_stderr: false)
    next if result.success?

    # If it fails again, error out
    ofail <<~EOS
      Failed to extract symbols from #{file}:
      #{result.stderr}
    EOS
  end
end

#prepare_relocation_to_locations ⇒ ::Keg::Relocation

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Returns:



254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
 
# File 'extend/os/mac/keg_relocate.rb', line 254

def prepare_relocation_to_locations
  relocation = super

  brewed_perl = runtime_dependencies&.any? do |dep|
    dep = T.cast(dep, T::Hash[String, T.untyped])
    dep["full_name"] == "perl" && dep["declared_directly"]
  end
  perl_path = if brewed_perl || name == "perl"
    "#{HOMEBREW_PREFIX}/opt/perl/bin/perl"
  elsif tab.built_on.present? &&
        (preferred_perl_version = tab.built_on&.[]("preferred_perl")&.presence) &&
        preferred_perl_version.match?(/^\d+\.\d+$/) &&
        (perl_path = "/usr/bin/perl#{preferred_perl_version}") &&
        File.exist?(perl_path)
    perl_path
  else
    "/usr/bin/perl#{MacOS.preferred_perl_version}"
  end
  relocation.add_replacement_pair(:perl, ::Keg::PERL_PLACEHOLDER, perl_path)

  if (openjdk = openjdk_dep_name_if_applicable)
    openjdk_path = HOMEBREW_PREFIX/"opt"/openjdk/"libexec/openjdk.jdk/Contents/Home"
    relocation.add_replacement_pair(:java, ::Keg::JAVA_PLACEHOLDER, openjdk_path.to_s)
  end

  relocation
end

#recursive_fgrep_args ⇒ String

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Returns:



283
284
285
286
287
 
# File 'extend/os/mac/keg_relocate.rb', line 283

def recursive_fgrep_args
  # Don't recurse into symlinks; the man page says this is the default, but
  # it's wrong. -O is a BSD-grep-only option.
  "-lrO"
end

#relocate_dynamic_linkage(relocation, with_placeholders: false, files: nil) ⇒ Array<::Pathname>

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Parameters:

Returns:



35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
 
# File 'extend/os/mac/keg_relocate.rb', line 35

def relocate_dynamic_linkage(relocation, with_placeholders: false, files: nil)
  candidates = if files
    # Metadata-driven pour: only the files recorded at bottle time carry
    # placeholdered linkage, so skip the whole-keg walk.
    keg_files(files).map { |file| MachOPathname.wrap(file) }
  else
    mach_o_files
  end

  linkage_files = []
  candidates.each do |file|
    Utils::Path.ensure_writable(file.to_path) do
      modified = T.let(false, T::Boolean)

      if file.dylib? && (dylib_id = file.dylib_id) && (id = relocated_name_for(dylib_id, relocation))
        modified = change_dylib_id(id, file, write: false) || modified
      end

      each_linkage_for(file, :dynamically_linked_libraries) do |old_name|
        new_name = relocated_name_for(old_name, relocation)
        modified = change_install_name(old_name, new_name, file, write: false) || modified if new_name
      end

      each_linkage_for(file, :rpaths) do |old_name|
        new_name = relocated_name_for(old_name, relocation)
        modified = change_rpath(old_name, new_name, file, write: false) || modified if new_name
      end

      next unless modified

      # The edits above only mutated the in-memory Mach-O, so persist
      # them in a single write per file.
      file.save_changes
      linkage_files << file.relative_path_from(path)
    end
  end

  # Every saved file's signature is now broken, so re-sign each exactly
  # once, parallelised across files.
  codesign_patched_binaries(linkage_files.map { path/it })
  linkage_files
end

#relocated_name_for(old_name, relocation) ⇒ String?

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Parameters:

Returns:



205
206
207
208
209
210
211
212
213
214
 
# File 'extend/os/mac/keg_relocate.rb', line 205

def relocated_name_for(old_name, relocation)
  old_prefix, new_prefix = relocation.replacement_pair_for(:prefix)
  old_cellar, new_cellar = relocation.replacement_pair_for(:cellar)

  if old_name.start_with? old_cellar
    old_name.sub(old_cellar, new_cellar)
  elsif old_name.start_with? old_prefix
    old_name.sub(old_prefix, new_prefix)
  end
end