Class: Homebrew::DevCmd::Benchmark Private

Inherits:
AbstractCommand show all
Includes:
SystemCommand::Mixin
Defined in:
dev-cmd/benchmark.rb,
sorbet/rbi/dsl/homebrew/dev_cmd/benchmark.rbi

This class is part of a private API. This class may only be used in the Homebrew/brew repository. Third parties should avoid using this class if possible, as it may be removed or changed without warning.

Defined Under Namespace

Classes: Args, Sample, Scenario

Constant Summary collapse

PHASES =

This constant is part of a private API. This constant may only be used in the Homebrew/brew repository. Third parties should avoid using this constant if possible, as it may be removed or changed without warning. 

%w[
  startup cli_parse command_load api_metadata_load formula_inflation dependency_resolution
  download_enqueue curl_headers curl_body checksum pour link
].freeze
BENCHMARK_ENV =

This constant is part of a private API. This constant may only be used in the Homebrew/brew repository. Third parties should avoid using this constant if possible, as it may be removed or changed without warning. 

T.let({
  "HOMEBREW_NO_ANALYTICS"                  => "1",
  "HOMEBREW_NO_AUTO_UPDATE"                => "1",
  "HOMEBREW_NO_AUTOREMOVE"                 => "1",
  "HOMEBREW_NO_ENV_HINTS"                  => "1",
  "HOMEBREW_NO_INSTALLED_DEPENDENTS_CHECK" => "1",
  "HOMEBREW_NO_INSTALL_CLEANUP"            => "1",
}.freeze, T::Hash[String, String])

Instance Method Summary collapse

Methods included from SystemCommand::Mixin

#system_command, #system_command!

Methods inherited from AbstractCommand

command, command_name, dev_cmd?, #initialize, parser, ruby_cmd?

Methods included from Utils::Output::Mixin

#issue_reporting_message, #odebug, #odeprecated, #odie, #odisabled, #ofail, #oh1, #oh1_title, #ohai, #ohai_title, #onoe, #opoo, #opoo_outside_github_actions, #opoo_without_github_actions_annotation, #pretty_deprecated, #pretty_disabled, #pretty_duration, #pretty_install_status, #pretty_installed, #pretty_uninstalled, #pretty_unmarked, #pretty_upgradable, #pretty_warning

Constructor Details

This class inherits a constructor from Homebrew::AbstractCommand

Instance Method Details

#args ⇒ Homebrew::DevCmd::Benchmark::Args

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Returns:



10
 
# File 'sorbet/rbi/dsl/homebrew/dev_cmd/benchmark.rbi', line 10

def args; end

#run ⇒ void

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

This method returns an undefined value.

Raises:



72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
 
# File 'dev-cmd/benchmark.rb', line 72

def run
  # Resolve `hyperfine` before any workload is prepared: installing it later
  # would refill the API metadata a cold workload has just removed.
  hyperfine = which("hyperfine", ENV.fetch("HOMEBREW_PATH")) || (HOMEBREW_PREFIX/"bin/hyperfine")
  unless hyperfine.executable?
    ohai "Installing hyperfine..."
    safe_system BENCHMARK_ENV, HOMEBREW_BREW_FILE, "install", "--formula", "hyperfine"
    # An already-installed but unlinked `hyperfine` makes the install a no-op.
    raise "`#{hyperfine}` is missing: try `brew link hyperfine`." unless hyperfine.executable?
  end
  exec(hyperfine.to_s, *args.named) if args.exec?

  runs = args.runs || "3"
  raise UsageError, "`--runs` must be a positive integer." unless runs.match?(/\A[1-9]\d*\z/)

  formulae = args.named
  install_formula = formulae.fetch(0)
  installed = system_command(
    HOMEBREW_BREW_FILE,
    args:         ["list", "--versions", "--formula", *formulae],
    env:          BENCHMARK_ENV,
    print_stderr: false,
  ).stdout.lines(chomp: true)
  raise UsageError, "Formulae must be uninstalled: #{installed.join(", ")}" if installed.any?

  brew!("install", "--only-dependencies", "--formula", install_formula)

  scenarios = [
    *%w[metadata_cold archive_cold archive_warm fully_warm].map do |workload|
      Scenario.new(operation: "install", workload:, formulae: formulae.first(1))
    end,
    *%w[metadata_cold archive_cold archive_warm]
      .product([1, 10, 50, 100].select { |batch_size| batch_size <= formulae.length })
      .map do |workload, batch_size|
        Scenario.new(operation: "fetch", workload:, formulae: formulae.first(batch_size))
      end,
  ]
  samples = T.let([], T::Array[Sample])

  begin
    Dir.mktmpdir("brew-benchmark") do |temporary_directory|
      runs.to_i.times do |iteration|
        # Machines drift by tens of milliseconds over a run, so rotate rather
        # than repeat the order: every workload is then measured from several
        # positions instead of always the same one.
        scenarios.rotate(iteration).each do |scenario|
          ohai "#{scenario.workload.tr("_", " ")} #{scenario.operation}",
               "run #{iteration + 1}, #{scenario.formulae.length} formulae"
          prepare(scenario, install_formula)
          samples << measure(hyperfine, scenario, iteration + 1, samples.length + 1,
                             Pathname(temporary_directory))
        end
      end
    end
  ensure
    remove_formula(install_formula)
  end

  output = Pathname("benchmark/results.json").expand_path
  output.dirname.mkpath
  output.write("#{JSON.pretty_generate({
    "schema_version" => 1,
    "generated_at"   => Time.now.utc.iso8601,
    "brew_version"   => HOMEBREW_VERSION,
    "runs"           => runs.to_i,
    "formulae"       => formulae,
    "phases"         => PHASES,
    "samples"        => samples.map(&:serialize),
  })}\n")
  summarise(samples)
  ohai "Full results written to #{output}"
end