Class: Homebrew::DevCmd::AdvisoryMatch Private

Inherits:
AbstractCommand show all
Defined in:
dev-cmd/advisory-match.rb,
sorbet/rbi/dsl/homebrew/dev_cmd/advisory_match.rbi

This class is part of a private API. This class may only be used in the Homebrew/brew repository. Third parties should avoid using this class if possible, as it may be removed or changed without warning.

Defined Under Namespace

Classes: Args, CountEmitter, DirEmitter, Emitter, JsonEmitter

Instance Method Summary collapse

Methods inherited from AbstractCommand

command, command_name, dev_cmd?, #initialize, parser, ruby_cmd?

Methods included from Utils::Output::Mixin

#issue_reporting_message, #odebug, #odeprecated, #odie, #odisabled, #ofail, #oh1, #oh1_title, #ohai, #ohai_title, #onoe, #opoo, #opoo_once, #opoo_outside_github_actions, #opoo_without_github_actions_annotation, #pretty_cannot_install, #pretty_deprecated, #pretty_disabled, #pretty_duration, #pretty_install_status, #pretty_installed, #pretty_uninstalled, #pretty_unmarked, #pretty_upgradable, #pretty_warning

Constructor Details

This class inherits a constructor from Homebrew::AbstractCommand

Instance Method Details

#args ⇒ Homebrew::DevCmd::AdvisoryMatch::Args

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Returns:



10
 
# File 'sorbet/rbi/dsl/homebrew/dev_cmd/advisory_match.rbi', line 10

def args; end

#build_emitter(formula_names: nil) ⇒ Emitter

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Parameters:

  • formula_names (Array<String>, nil) (defaults to: nil)

Returns:



1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
 
# File 'dev-cmd/advisory-match.rb', line 1200

def build_emitter(formula_names: nil)
  if (dir = args.output)
    DirEmitter.new(dir, verbose: args.verbose?, close_open_ranges: !args.no_history?,
                   reconcile_history: args.reconcile_history?, formula_names:)
  elsif args.json?
    JsonEmitter.new
  else
    CountEmitter.new
  end
end

#each_formula(formula_names: nil) ⇒ T::Enumerator[Formula]

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Parameters:

  • formula_names (Array<String>, nil) (defaults to: nil)

Returns:

Raises:



413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
 
# File 'dev-cmd/advisory-match.rb', line 413

def each_formula(formula_names: nil)
  return args.named.to_resolved_formulae.each if !args.all? && !args.formula_list

  raise UsageError, "`--all` does not take named arguments" if args.named.any?

  tap = CoreTap.instance
  raise TapUnavailableError, tap.name unless tap.installed?

  names = tap.formula_names
  if formula_names
    (formula_names - names).each do |name|
      opoo "#{name}: no longer in homebrew/core; leaving its records unchanged"
    end
    names &= formula_names
  end

  Enumerator.new do |y|
    names.each do |name|
      y << Formulary.factory(name)
    rescue => e
      onoe "Error loading formula '#{name}': #{e}"
    end
  end
end

#emit_index(matcher) ⇒ void

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

This method returns an undefined value.

Parameters:

Raises:



1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
 
# File 'dev-cmd/advisory-match.rb', line 1212

def emit_index(matcher)
  tap = CoreTap.instance
  raise TapUnavailableError, tap.name unless tap.installed?

  index = tap.formula_names.each_with_object({}) do |name, h|
    identity = matcher.identify(Formulary.factory(name))
    next unless identity.identifiable?

    h[name] = {
      git_repo:          identity.git_repo,
      git_tag:           identity.git_tag,
      primary_package:   identity.primary_package&.to_h,
      resource_packages: identity.resource_packages.transform_values(&:to_h),
      distro_packages:   identity.distro_packages,
    }.compact
  rescue => e
    onoe "Error loading formula '#{name}': #{e}"
  end
  puts JSON.pretty_generate(index)
end

#hold_upstream_formula(formula, error, emitter) ⇒ void

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

This method returns an undefined value.

Parameters:



391
392
393
394
 
# File 'dev-cmd/advisory-match.rb', line 391

def hold_upstream_formula(formula, error, emitter)
  emitter.record_upstream_unavailable(formula.name) if emitter.is_a?(DirEmitter)
  opoo "#{formula.name}: upstream unavailable; leaving its records unchanged: #{error.message}"
end

#local_overrides ⇒ Homebrew::Vulns::AdvisoryOverrides?

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Returns:



406
407
408
409
410
 
# File 'dev-cmd/advisory-match.rb', line 406

def local_overrides
  return unless (path = args.overrides)

  Homebrew::Vulns::AdvisoryOverrides.from_file(Pathname(path))
end

#local_repology ⇒ Homebrew::Vulns::Repology?

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

A CI run that has just built the index locally (advisory-database's Ingest) reads it directly instead of fetching the published copy.

Returns:



399
400
401
402
403
 
# File 'dev-cmd/advisory-match.rb', line 399

def local_repology
  return unless (path = args.repology)

  Homebrew::Vulns::Repology.from_file(Pathname(path))
end

#platform_provenance_changed?(matcher, emitter, record, view, family) ⇒ Boolean

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

A platform view must rediscover the record through exactly one hit carrying its stored upstream family with the same matching provenance; anything else means the stored ranges cannot be compared across views.

Parameters:

Returns:

  • (Boolean) 


382
383
384
385
386
387
388
 
# File 'dev-cmd/advisory-match.rb', line 382

def platform_provenance_changed?(matcher, emitter, record, view, family)
  hit = family.first
  return true if !family.one? || hit.nil?
  return true if emitter.range_basis(record) != emitter.range_basis(matcher.to_brew_record(view, hit))

  (record.fetch(:upstream) - hit.identifiers).any?
end

#reconcile_formula(matcher, formula, hits, emitter, latest_macos:) ⇒ void

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

This method returns an undefined value.

Parameters:



271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
 
# File 'dev-cmd/advisory-match.rb', line 271

def reconcile_formula(matcher, formula, hits, emitter, latest_macos:)
  return if matcher.skip_formula?(formula.name)

  resource_records = emitter.reconciliation_resource_records(formula.name).reject do |record|
    if hits.any? { |hit| record.upstream.intersect?(hit.identifiers) }
      true
    elsif matcher.preserve_homebrew_ranges?(formula.name, record.upstream)
      emitter.skip_reconciliation(record.id, [:preserved])
      true
    else
      false
    end
  end
  resource_hits = matcher.resource_advisories_for(resource_records)
  hits = matcher.dedup_by_aliases(hits + resource_hits)
  resource_records.each do |record|
    next if hits.any? { |hit| record.upstream.intersect?(hit.identifiers) }

    emitter.skip_reconciliation(record.id, [:resource_not_rediscovered])
  end
  groups = hits.to_h do |hit|
    ids = matcher.record_ids(formula, hit)
    [ids.fetch(0), ids]
  end
  errors = emitter.prepare_aliases(formula.name, groups)
  if errors.any?
    errors.each { |error| onoe error }
    Homebrew.failed = true
    return
  end

  candidates = hits.to_h { |hit| [matcher.record_id(formula, hit), matcher.to_brew_record(formula, hit)] }
  candidates.select! { |_, record| emitter.reconciliation_record(record) }
  return if candidates.empty?

  outcomes = T.let({}, T::Hash[String, T::Array[Homebrew::Vulns::Match::ReconciledHistory]])
  candidates.each_key { |id| outcomes[id] = [] }
  platforms = T.let([[latest_macos, :arm], [latest_macos, :intel], [:linux, :arm], [:linux, :intel]],
                    T::Array[[Symbol, Symbol]])
  platforms.each_with_index do |(os, arch), index|
    pending = candidates.select do |id, _|
      outcomes.fetch(id).all? { |result| [:range, :never_affected].include?(result.state) }
    end
    break if pending.empty?

    Homebrew::SimulateSystem.with(os:, arch:) do
      # Reload under each platform: resources and primary sources can differ.
      begin
        view = index.zero? ? formula : Formulary.factory(formula.path)
        platform_hits = if index.zero?
          hits
        else
          matcher.dedup_by_aliases(
            matcher.advisories_for(view) + resource_hits,
          )
        end
      rescue Homebrew::Vulns::OSV::Error
        raise
      rescue => e
        pending.each_key do |id|
          outcomes.fetch(id) << Homebrew::Vulns::Match::ReconciledHistory.new(
            state: :unresolved, introduced: nil, fixed: nil, reasons: [:platform_unavailable],
          )
        end
        opoo "#{formula.name} (#{os}/#{arch}): #{e.message}"
        next
      end
      pending.each do |id, record|
        family = platform_hits.select { |hit| hit.identifiers.intersect?(record.fetch(:upstream)) }
        if platform_provenance_changed?(matcher, emitter, record, view, family)
          result = Homebrew::Vulns::Match::ReconciledHistory.new(
            state: :unresolved, introduced: nil, fixed: nil, reasons: [:platform_provenance_changed],
          )
        else
          hit = family.fetch(0)
          result = if resource_records.any? { |stored| stored.upstream.intersect?(hit.identifiers) }
            matcher.reconcile_history(view, hit, require_resource_presence: true)
          else
            matcher.reconcile_history(view, hit)
          end
          emitter.record_history_walk if result.state != :preserved
        end
        outcomes.fetch(id) << result
      end
    end
  end
  candidates.each do |id, record|
    results = outcomes.fetch(id)
    reasons = results.flat_map(&:reasons)
    reasons << :preserved if results.any? { |result| result.state == :preserved }
    decisions = results.map { |result| [result.state, result.introduced, result.fixed] }.uniq
    if reasons.empty? && (results.length != platforms.length || !decisions.one?)
      reasons << :platform_disagreement
    end
    if reasons.any?
      emitter.skip_reconciliation(id, reasons.uniq)
    else
      emitter.reconcile(record, results.fetch(0))
    end
  end
rescue Homebrew::Vulns::OSV::Error => e
  hold_upstream_formula(formula, e, emitter)
end

#report(matcher, formula, hits) ⇒ void

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

This method returns an undefined value.

Parameters:



447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
 
# File 'dev-cmd/advisory-match.rb', line 447

def report(matcher, formula, hits)
  ohai "#{formula.name} #{formula.pkg_version}"
  if hits.empty?
    puts "  No advisories matched."
    return
  end
  hits.sort_by { |h| [-h.vulnerability.severity_level, h.canonical_id] }.each do |hit|
    v = hit.vulnerability
    status, = matcher.range_status(hit, formula_name: formula.name)
    state = case status&.state
    when nil       then "uncomparable"
    when :affected then "AFFECTED#{", upstream fix #{status&.fixed_in}" if status&.fixed_in}"
    when :fixed    then "fixed (upstream #{status&.fixed_in || "?"})"
    else "not applicable"
    end
    summary = v.summary&.slice(0, 60)
    puts "  #{hit.canonical_id} [#{hit.strategy}, #{matcher.confidence_for(hit, status)}] " \
         "#{v.severity_display} #{state}" \
         "#{" (resource: #{hit.resource})" if hit.resource}" \
         "#{" — #{summary}" if summary}"
  end
end

#report_history_load_failures(matcher) ⇒ void

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

This method returns an undefined value.

Parameters:



251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
 
# File 'dev-cmd/advisory-match.rb', line 251

def report_history_load_failures(matcher)
  failures = matcher.history_load_failures
  return if failures.empty?

  revisions = failures.map { |failure| [failure.formula, failure.revision, failure.path] }.uniq.length
  formulae = failures.map(&:formula).uniq.length
  puts "  History loads: #{revisions} failed formula revisions across #{formulae} formulae " \
       "(#{failures.length} platform loads)"
  return unless args.verbose?

  failures.each do |failure|
    puts "    #{failure.formula}: #{failure.revision}:#{failure.path} [#{failure.platform}] " \
         "#{failure.error_class}: #{failure.message}"
  end
end

#run ⇒ void

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

This method returns an undefined value.



71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
 
# File 'dev-cmd/advisory-match.rb', line 71

def run
  if args.formula_list && !args.reconcile_history? && !args.new_history?
    raise UsageError, "`--formula-list` requires `--new-history` or `--reconcile-history`"
  end
  if args.reconcile_history? && !args.overrides
    raise UsageError, "`--reconcile-history` requires an explicit `--overrides` file"
  end
  if args.formula_list && args.named.any?
    raise UsageError, "`--formula-list` does not take named arguments"
  end

  formula_names = if (list = args.formula_list)
    File.readlines(list, chomp: true).reject(&:empty?).uniq
  end
  Formulary.enable_factory_cache!
  Homebrew::API.with_no_api_env do
    latest_macos = MacOSVersion.new((HOMEBREW_MACOS_NEWEST_UNSUPPORTED.to_i - 1).to_s).to_sym
    Homebrew::SimulateSystem.with(os: latest_macos, arch: :arm) do
      overrides = local_overrides
      matcher = Homebrew::Vulns::Match.new(repology:        local_repology,
                                           overrides:,
                                           bulk:            args.all? || args.index? || !formula_names.nil?,
                                           strict_upstream: args.reconcile_history?)
      next emit_index(matcher) if args.index?

      emitter = build_emitter(formula_names:)
      begin
        on_error = if args.reconcile_history?
          lambda do |formula, error|
            hold_upstream_formula(formula, error, emitter)
          end
        end
        matcher.each_advisory_batch(
          each_formula(formula_names:),
          on_error:,
        ) do |formula, hits|
          if args.reconcile_history? && emitter.is_a?(DirEmitter)
            reconcile_formula(matcher, formula, hits, emitter, latest_macos:)
            next
          end

          report(matcher, formula, hits) if text_mode?
          # A below-introduced hit would otherwise look open to OSV
          # consumers; it must not participate in alias maintenance.
          actionable = hits.filter_map do |hit|
            status, = matcher.range_status(hit, formula_name: formula.name)
            [hit, status] if status&.state != :not_applicable
          end
          record_ids_by_canonical = actionable.to_h do |hit, _status|
            ids = matcher.record_ids(formula, hit)
            [ids.fetch(0), ids]
          end
          alias_errors = emitter.prepare_aliases(formula.name, record_ids_by_canonical)
          if alias_errors.any?
            alias_errors.each { |error| onoe error }
            Homebrew.failed = true
            next
          end

          actionable.each do |hit, status|
            record_id = matcher.record_id(formula, hit)
            next if emitter.alias_protected?(record_id)

            reviewed_state = emitter.reviewed_range_state(record_id)
            if reviewed_state && !status && matcher.current_prerelease_boundary?(hit)
              opoo "#{record_id}: prerelease_boundary in current version; leaving reviewed record unchanged"
              next
            end
            initial_introduction = false
            initial_introduction = true if !args.no_history? && status && reviewed_state.nil?
            candidate = matcher.to_brew_record(formula, hit)
            basis_changed = emitter.range_basis_changed?(candidate)
            if basis_changed && !initial_introduction && (args.no_history? || !status&.fixed?)
              emitter.emit(candidate)
              next
            end

            override = overrides&.advisory_override(formula.name, hit.identifiers)
            if initial_introduction && override&.state
              upstream_state = matcher.aggregate_state_at(formula, hit)
              if override.state != upstream_state
                emitter.record_history_unavailable(formula.name)
                opoo "#{record_id}: reviewed state override " \
                     "#{upstream_state.nil? ? "cannot be checked against" : "disagrees with"} " \
                     "upstream history; " \
                     "skipping automatic update. Review its ranges and provenance together."
                next
              end
            end

            has_open_range = reviewed_state == :open
            has_terminal_range = reviewed_state == :terminal
            transition = (status&.fixed? && has_open_range) ||
                         (status&.affected? && has_terminal_range)
            if args.no_history? && transition
              opoo "#{record_id}: reviewed range transition needs history; leaving it unchanged"
              next
            end

            walk_history = !args.no_history? && status&.fixed?
            walk_history &&= basis_changed || emitter.history_required?(record_id) if args.new_history?
            emitter.record_history_walk if walk_history
            first_fixed = matcher.first_fixed_version(formula, hit) if walk_history
            fixed_boundary = T.let(nil, T.nilable(String))
            case first_fixed
            when String
              fixed_boundary = first_fixed
            when nil
              # No history walk was required.
            when :never_affected
              next
            when :history_unavailable
              emitter.record_history_unavailable(formula.name)
              opoo "#{record_id}: formula history is unavailable; skipping automatic update"
              next
            else
              raise TypeError, "unexpected fixed-history result: #{first_fixed.inspect}"
            end

            if fixed_boundary && !emitter.fixed_boundary_valid?(record_id, fixed_boundary)
              onoe "#{record_id}: fixed #{fixed_boundary} does not follow its reviewed range"
              Homebrew.failed = true
              next
            end

            first_introduced = T.let(nil, T.nilable(String))
            if initial_introduction
              emitter.record_history_walk
              introduced = matcher.first_introduced_version(formula, hit, first_fixed: fixed_boundary)
              case introduced
              when String
                first_introduced = introduced
              when :history_unavailable
                emitter.record_history_unavailable(formula.name)
                opoo "#{record_id}: affected introduction cannot be established; skipping automatic update"
                next
              else
                raise TypeError, "unexpected introduction-history result: #{introduced.inspect}"
              end
            end
            if status&.affected? && has_terminal_range
              emitter.record_history_walk
              reintroduced = matcher.first_reintroduced_version(formula, hit)
              if reintroduced == :history_unavailable
                emitter.record_history_unavailable(formula.name)
                opoo "#{record_id}: reintroduction history is unavailable; skipping automatic update"
                next
              end
              unless reintroduced.is_a?(String)
                onoe "#{record_id}: could not find a prior non-affected version for its reviewed fixed range"
                Homebrew.failed = true
                next
              end
              unless emitter.reintroduction_boundary_valid?(record_id, reintroduced)
                onoe "#{record_id}: reintroduction #{reintroduced} does not follow its reviewed range"
                Homebrew.failed = true
                next
              end
              first_introduced = reintroduced
            end

            candidate = matcher.to_brew_record(formula, hit, first_fixed: fixed_boundary, first_introduced:)
            # A metadata override does not replace the upstream constraints
            # used by the history walk. Correct its reviewed ranges and
            # provenance together instead of automatically certifying them.
            revalidated = !fixed_boundary.nil? && !override&.fixed_in_overridden
            emitter.emit(candidate, revalidated:, initial_introduction:)
          end
        end
      rescue Homebrew::Vulns::OSV::Error => e
        onoe "OSV query failed: #{e.message}"
        Homebrew.failed = true
      end
      emitter.finish
      report_history_load_failures(matcher) if args.reconcile_history?
    end
  end
end

#text_mode? ⇒ Boolean

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Returns:

  • (Boolean) 


439
440
441
 
# File 'dev-cmd/advisory-match.rb', line 439

def text_mode?
  !args.json? && args.output.nil?
end