Module: Homebrew::Style Private

Extended by:
SystemCommand::Mixin, Utils::Output::Mixin
Defined in:
style.rb

Overview

This module is part of a private API. This module may only be used in the Homebrew/brew repository. Third parties should avoid using this module if possible, as it may be removed or changed without warning.

Helper module for running RuboCop.

Defined Under Namespace

Classes: Offense, Offenses

Constant Summary collapse

RUBOCOP =

This constant is part of a private API. This constant may only be used in the Homebrew/brew repository. Third parties should avoid using this constant if possible, as it may be removed or changed without warning. 

T.let((HOMEBREW_LIBRARY_PATH/"utils/rubocop.rb").freeze, Pathname)

Class Method Summary collapse

Methods included from Utils::Output::Mixin

issue_reporting_message, odebug, odeprecated, odie, odisabled, ofail, oh1, oh1_title, ohai, ohai_title, onoe, opoo, opoo_outside_github_actions, opoo_without_github_actions_annotation, pretty_deprecated, pretty_disabled, pretty_duration, pretty_install_status, pretty_installed, pretty_uninstalled, pretty_unmarked, pretty_upgradable, pretty_warning

Methods included from SystemCommand::Mixin

system_command, system_command!

Class Method Details

.actionlint ⇒ Pathname

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Returns:



516
517
518
519
520
521
522
 
# File 'style.rb', line 516

def self.actionlint
  require "formula"
  T.cast(Formula["actionlint"].ensure_installed!(latest:       true,
                                                 reason:       "GitHub Actions checks",
                                                 executable:   "actionlint",
                                                 version_args: ["-version"]), Pathname)
end

.check_style_and_print(files, **options) ⇒ Boolean

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Checks style for a list of files, printing simple RuboCop output. Returns true if violations were found, false otherwise.

Parameters:

Returns:

  • (Boolean) 


20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
 
# File 'style.rb', line 20

def self.check_style_and_print(files, **options)
  success = check_style_impl(files, :print, **options)

  if GitHub::Actions.env_set? && !success
    check_style_json(files, **options).each do |path, offenses|
      offenses.each do |o|
        line = o.location.line
        column = o.location.line

        annotation = GitHub::Actions::Annotation.new(:error, o.message, file: path, line:, column:)
        puts annotation if annotation.relevant?
      end
    end
  end

  T.cast(success, T::Boolean)
end

.check_style_impl(files, output_type, fix: false, todo: false, except_cops: nil, only_cops: nil, display_cop_names: false, reset_cache: false, debug: false, verbose: false) ⇒ Offenses, Boolean

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Parameters:

  • files (Array<Pathname>)
  • output_type (Symbol)
  • fix (Boolean) (defaults to: false)
  • todo (Boolean) (defaults to: false)
  • except_cops (Array<String>, nil) (defaults to: nil)
  • only_cops (Array<String>, nil) (defaults to: nil)
  • display_cop_names (Boolean) (defaults to: false)
  • reset_cache (Boolean) (defaults to: false)
  • debug (Boolean) (defaults to: false)
  • verbose (Boolean) (defaults to: false)

Returns:

Raises:

  • (ArgumentError) 


59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
 
# File 'style.rb', line 59

def self.check_style_impl(files, output_type,
                          fix: false,
                          todo: false,
                          except_cops: nil, only_cops: nil,
                          display_cop_names: false,
                          reset_cache: false,
                          debug: false, verbose: false)
  raise ArgumentError, "Invalid output type: #{output_type.inspect}" if [:print, :json].exclude?(output_type)

  ruby_files = T.let([], T::Array[Pathname])
  shell_files = T.let([], T::Array[Pathname])
  actionlint_files = T.let([], T::Array[Pathname])
  Array(files).map { Pathname(it) }
              .each do |path|
    case path.extname
    when ".rb"
      ruby_files << path
    when ".sh"
      shell_files << path
    when ".yml"
      actionlint_files << path if path.realpath.to_s.include?("/.github/workflows/")
    else
      ruby_files << path
      shell_files += if [HOMEBREW_PREFIX, HOMEBREW_REPOSITORY].include?(path)
        shell_scripts
      else
        path.glob("**/*.sh")
            .reject { |file_path| file_path.to_s.include?("/vendor/") || file_path.directory? }
      end
      actionlint_files += (path/".github/workflows").glob("*.y{,a}ml")
    end
  end

  rubocop_needed = files.blank? || ruby_files.any?
  shell_needed = files.blank? || shell_files.any?

  actionlint_files = github_workflow_files if files.blank? && actionlint_files.blank?
  has_actionlint_workflow = actionlint_files.any? do |path|
    path.to_s.end_with?("/.github/workflows/actionlint.yml")
  end
  odebug "actionlint workflow detected. Skipping actionlint checks." if has_actionlint_workflow
  actionlint_needed = files.blank? || (!has_actionlint_workflow && actionlint_files.any?)

  # Resolve the linter executables (installing them if necessary) before
  # spawning threads so those threads cannot race to install formulae.
  shellcheck_path = (shellcheck if shell_needed || actionlint_needed)
  shfmt_path = (shfmt_executable if shell_needed)
  actionlint_path = (actionlint if actionlint_needed)

  shellcheck_out = StringIO.new
  shellcheck_err = StringIO.new
  shfmt_out = StringIO.new
  shfmt_err = StringIO.new
  actionlint_out = StringIO.new
  actionlint_err = StringIO.new

  # Run the shell and GitHub Actions checks on background threads with
  # buffered output while RuboCop runs on the main thread.
  shell_thread = Thread.new do
    shellcheck_result = if shell_needed
      run_shellcheck(shell_files, output_type, fix:, shellcheck_path:,
                     out: shellcheck_out, err: shellcheck_err)
    elsif output_type == :json
      []
    else
      true
    end
    # `shellcheck --fix` and `shfmt --write` may touch the same files so
    # they must not run concurrently with each other.
    shfmt_result = !shell_needed || run_shfmt!(shell_files, fix:, shfmt_path:,
                                               out: shfmt_out, err: shfmt_err)
    [shellcheck_result, shfmt_result]
  end
  actionlint_thread = Thread.new do
    !actionlint_needed ||
      run_actionlint!(actionlint_files, actionlint_path:, shellcheck_path:,
                      out: actionlint_out, err: actionlint_err)
  end

  rubocop_result = if rubocop_needed
    run_rubocop(ruby_files, output_type,
                fix:,
                todo:,
                except_cops:, only_cops:,
                display_cop_names:,
                reset_cache:,
                debug:, verbose:)
  elsif output_type == :json
    []
  else
    true
  end

  shellcheck_result, shfmt_result = shell_thread.value
  actionlint_result = actionlint_thread.value

  [
    [shellcheck_out, shellcheck_err],
    [shfmt_out, shfmt_err],
    [actionlint_out, actionlint_err],
  ].each do |out, err|
    $stdout.print out.string
    $stderr.print err.string
  end

  if output_type == :json
    Offenses.new(
      T.cast(rubocop_result, T::Array[T::Hash[String, T.untyped]]) +
      T.cast(shellcheck_result, T::Array[T::Hash[String, T.untyped]]),
    )
  else
    rubocop_result && !!shellcheck_result && shfmt_result && actionlint_result
  end
end

.check_style_json(files, **options) ⇒ Offenses

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Checks style for a list of files, returning results as an Offenses object parsed from its JSON output.

Parameters:

Returns:



41
42
43
 
# File 'style.rb', line 41

def self.check_style_json(files, **options)
  T.cast(check_style_impl(files, :json, **options), Offenses)
end

.github_workflow_files ⇒ Array<Pathname>

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Returns:



490
491
492
 
# File 'style.rb', line 490

def self.github_workflow_files
  HOMEBREW_REPOSITORY.glob(".github/workflows/*.yml")
end

.json_result!(result) ⇒ T.untyped

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Parameters:

Returns:

  • (T.untyped) 


459
460
461
462
463
464
465
466
 
# File 'style.rb', line 459

def self.json_result!(result)
  # An exit status of 1 just means violations were found; other numbers mean
  # execution errors.
  # JSON needs to be at least 2 characters.
  result.assert_success! if !(0..1).cover?(result.status.exitstatus) || result.stdout.length < 2

  JSON.parse(result.stdout)
end

.run_actionlint!(files, actionlint_path: nil, shellcheck_path: nil, out: $stdout, err: $stderr) ⇒ Boolean

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Parameters:

  • files (Array<Pathname>)
  • actionlint_path (Pathname, nil) (defaults to: nil)
  • shellcheck_path (Pathname, nil) (defaults to: nil)
  • out (IO, StringIO) (defaults to: $stdout)
  • err (IO, StringIO) (defaults to: $stderr)

Returns:

  • (Boolean) 


427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
 
# File 'style.rb', line 427

def self.run_actionlint!(files, actionlint_path: nil, shellcheck_path: nil, out: $stdout, err: $stderr)
  actionlint_path ||= actionlint
  shellcheck_path ||= shellcheck
  files = github_workflow_files if files.blank?

  tap_configs = files.filter_map do |f|
    tap = Tap.from_path(f)
    next unless tap

    tap_config = tap.path/".github/actionlint.yaml"
    tap_config if tap_config.exist?
  end.uniq

  config_file = if tap_configs.one?
    tap_configs.fetch(0)
  else
    HOMEBREW_REPOSITORY/".github/actionlint.yaml"
  end

  # the ignore is to avoid false positives in e.g. actions, homebrew-test-bot
  args = ["-shellcheck", shellcheck_path,
          "-config-file", config_file,
          "-ignore", "image: string; options: string",
          "-ignore", "label .* is unknown"]
  args << "-color" if Tty.color?
  result = system_command actionlint_path, args: [*args, *files], print_stderr: false
  out.print result.stdout
  err.print result.stderr
  result.success?
end

.run_rubocop(files, output_type, fix: false, todo: false, except_cops: nil, only_cops: nil, display_cop_names: false, reset_cache: false, debug: false, verbose: false) ⇒ Boolean, Array<Hash{String => T.untyped}>

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Parameters:

  • files (Array<Pathname>)
  • output_type (Symbol)
  • fix (Boolean) (defaults to: false)
  • todo (Boolean) (defaults to: false)
  • except_cops (Array<String>, nil) (defaults to: nil)
  • only_cops (Array<String>, nil) (defaults to: nil)
  • display_cop_names (Boolean) (defaults to: false)
  • reset_cache (Boolean) (defaults to: false)
  • debug (Boolean) (defaults to: false)
  • verbose (Boolean) (defaults to: false)

Returns:



190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
 
# File 'style.rb', line 190

def self.run_rubocop(files, output_type,
                     fix: false, todo: false, except_cops: nil, only_cops: nil, display_cop_names: false,
                     reset_cache: false,
                     debug: false, verbose: false)
  require "warnings"

  Warnings.ignore :parser_syntax do
    require "rubocop"
  end

  require "rubocops/all"

  args = %w[
    --force-exclusion
  ]
  args << "--autocorrect-all" if fix
  args << "--disable-uncorrectable" if todo

  args += ["--extra-details"] if verbose

  if except_cops
    except_cops.map! { |cop| RuboCop::Cop::Registry.global.qualified_cop_name(cop.to_s, "") }
    cops_to_exclude = except_cops.select do |cop|
      RuboCop::Cop::Registry.global.names.include?(cop) ||
        RuboCop::Cop::Registry.global.departments.include?(cop.to_sym)
    end

    args << "--except" << cops_to_exclude.join(",") unless cops_to_exclude.empty?
  elsif only_cops
    only_cops.map! { |cop| RuboCop::Cop::Registry.global.qualified_cop_name(cop.to_s, "") }
    cops_to_include = only_cops.select do |cop|
      RuboCop::Cop::Registry.global.names.include?(cop) ||
        RuboCop::Cop::Registry.global.departments.include?(cop.to_sym)
    end

    odie "RuboCops #{only_cops.join(",")} were not found" if cops_to_include.empty?

    args << "--only" << cops_to_include.join(",")
  end

  files.map!(&:expand_path)
  base_dir = Dir.pwd
  if files.blank? || files == [HOMEBREW_REPOSITORY]
    files = [HOMEBREW_LIBRARY_PATH]
    base_dir = HOMEBREW_LIBRARY_PATH
  elsif files.any? { |f| f.to_s.start_with?(HOMEBREW_REPOSITORY/"docs") || (f.basename.to_s == "docs") }
    args << "--config" << (HOMEBREW_REPOSITORY/"docs/docs_rubocop_style.yml")
  elsif files.any? { |f| f.to_s.start_with? HOMEBREW_LIBRARY_PATH }
    base_dir = HOMEBREW_LIBRARY_PATH
  else
    args << "--config" << (HOMEBREW_LIBRARY/".rubocop.yml")
    base_dir = HOMEBREW_LIBRARY if files.any? { |f| f.to_s.start_with? HOMEBREW_LIBRARY }
  end

  HOMEBREW_CACHE.mkpath
  cache_dir = HOMEBREW_CACHE.realpath/"style"
  cache_env = if (!cache_dir.exist? && cache_dir.parent.writable?) || cache_dir.writable?
    args << "--parallel"

    FileUtils.rm_rf cache_dir if reset_cache

    { "XDG_CACHE_HOME" => cache_dir.to_s }
  else
    args << "--cache" << "false"

    {}
  end

  args += files

  ruby_args = HOMEBREW_RUBY_EXEC_ARGS.dup
  case output_type
  when :print
    args << "--debug" if debug

    # Don't show the default formatter's progress dots
    # on CI or if only checking a single file.
    args << "--format" << "clang" if ENV["CI"] || files.one? { |f| !f.directory? }

    args << "--color" if Tty.color?

    system cache_env, *ruby_args, "--", RUBOCOP, *args, chdir: base_dir
    $CHILD_STATUS.success?
  when :json
    result = system_command ruby_args.shift,
                            args:  [*ruby_args, "--", RUBOCOP, "--format", "json", *args],
                            env:   cache_env,
                            chdir: base_dir
    json = json_result!(result)
    json["files"].each do |file|
      file["path"] = File.absolute_path(file["path"], base_dir)
    end
  end
end

.run_shellcheck(files, output_type, fix: false, shellcheck_path: nil, out: $stdout, err: $stderr) ⇒ Boolean, ...

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Parameters:

  • files (Array<Pathname>)
  • output_type (Symbol)
  • fix (Boolean) (defaults to: false)
  • shellcheck_path (Pathname, nil) (defaults to: nil)
  • out (IO, StringIO) (defaults to: $stdout)
  • err (IO, StringIO) (defaults to: $stderr)

Returns:



295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
 
# File 'style.rb', line 295

def self.run_shellcheck(files, output_type, fix: false, shellcheck_path: nil, out: $stdout, err: $stderr)
  shellcheck_path ||= shellcheck
  files = shell_scripts if files.blank?

  files = files.map(&:realpath) # use absolute file paths

  args = [
    "--shell=bash",
    "--enable=all",
    "--external-sources",
    "--source-path=#{HOMEBREW_LIBRARY}",
  ]

  if fix
    # patch options:
    #   -g 0 (--get=0)       : suppress environment variable `PATCH_GET`
    #   -f   (--force)       : we know what we are doing, force apply patches
    #   -d / (--directory=/) : change to root directory, since we use absolute file paths
    #   -p0  (--strip=0)     : do not strip path prefixes, since we are at root directory
    # NOTE: We use short flags for compatibility.
    patch_command = %w[patch -g 0 -f -d / -p0]
    patches = shellcheck_chunks(shellcheck_path, files, ["--format=diff", *args]).map(&:stdout).join
    Utils.safe_popen_write(*patch_command) { |p| p.write(patches) } if patches.present?
  end

  case output_type
  when :print
    print_args = ["--format=tty", *args]
    print_args << "--color=always" if Tty.color?
    results = shellcheck_chunks(shellcheck_path, files, print_args)
    results.each do |result|
      out.print result.stdout
      err.print result.stderr
    end
    results.all?(&:success?)
  when :json
    results = shellcheck_chunks(shellcheck_path, files, ["--format=json", *args])
    json = results.flat_map { |result| json_result!(result) }

    # Convert to same format as RuboCop offenses.
    severity_hash = { "style" => "refactor", "info" => "convention" }
    json.group_by { |v| v["file"] }
        .map do |k, v|
      {
        "path"     => k,
        "offenses" => v.map do |o|
          o.delete("file")

          o["cop_name"] = "SC#{o.delete("code")}"

          level = o.delete("level")
          o["severity"] = severity_hash.fetch(level, level)

          line = o.delete("line")
          column = o.delete("column")

          o["corrected"] = false
          o["correctable"] = o.delete("fix").present?

          o["location"] = {
            "start_line"   => line,
            "start_column" => column,
            "last_line"    => o.delete("endLine"),
            "last_column"  => o.delete("endColumn"),
            "line"         => line,
            "column"       => column,
          }

          o
        end,
      }
    end
  end
end

.run_shfmt!(files, fix: false, shfmt_path: nil, out: $stdout, err: $stderr) ⇒ Boolean

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Parameters:

  • files (Array<Pathname>)
  • fix (Boolean) (defaults to: false)
  • shfmt_path (Pathname, nil) (defaults to: nil)
  • out (IO, StringIO) (defaults to: $stdout)
  • err (IO, StringIO) (defaults to: $stderr)

Returns:

  • (Boolean) 


399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
 
# File 'style.rb', line 399

def self.run_shfmt!(files, fix: false, shfmt_path: nil, out: $stdout, err: $stderr)
  shfmt_path ||= shfmt_executable
  files = shell_scripts if files.blank?
  # Do not format completions and Dockerfile
  files.delete(HOMEBREW_REPOSITORY/"completions/bash/brew")
  files.delete(HOMEBREW_REPOSITORY/"Dockerfile")

  args = ["--language-dialect", "bash", "--indent", "2", "--case-indent", "--", *files]
  args.unshift("--write") if fix # need to add before "--"

  result = system_command shfmt,
                          args:,
                          env:          { "HOMEBREW_SHFMT" => shfmt_path.to_s },
                          print_stderr: false
  out.print result.stdout
  err.print result.stderr
  result.success?
end

.shell_scripts ⇒ Array<Pathname>

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Returns:



469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
 
# File 'style.rb', line 469

def self.shell_scripts
  [
    HOMEBREW_ORIGINAL_BREW_FILE.realpath,
    HOMEBREW_REPOSITORY/"completions/bash/brew",
    HOMEBREW_REPOSITORY/"Dockerfile",
    *HOMEBREW_REPOSITORY.glob(".devcontainer/**/*.sh"),
    *HOMEBREW_REPOSITORY.glob(".github/scripts/*.sh"),
    *HOMEBREW_REPOSITORY.glob("package/scripts/*"),
    *HOMEBREW_LIBRARY.glob("Homebrew/**/*.sh").reject { |path| path.to_s.include?("/vendor/") },
    *HOMEBREW_LIBRARY.glob("Homebrew/shims/**/*").map(&:realpath).uniq
                     .reject(&:directory?)
                     .reject { |path| path.basename.to_s == "cc" }
                     .select do |path|
                       %r{^#! ?/bin/(?:ba)?sh( |$)}.match?(path.read(13))
                     end,
    *HOMEBREW_LIBRARY.glob("Homebrew/{dev-,}cmd/*.sh"),
    *HOMEBREW_LIBRARY.glob("Homebrew/{cask/,}utils/*.sh"),
  ]
end

.shellcheck ⇒ Pathname

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Returns:



495
496
497
498
499
500
 
# File 'style.rb', line 495

def self.shellcheck
  require "formula"
  T.cast(Formula["shellcheck"].ensure_installed!(latest:     true,
                                                 reason:     "shell style checks",
                                                 executable: "shellcheck"), Pathname)
end

.shfmt ⇒ Pathname

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Returns:



503
504
505
 
# File 'style.rb', line 503

def self.shfmt
  HOMEBREW_LIBRARY/"Homebrew/utils/shfmt.sh"
end