Class: Utils::Analytics::Repository Private
- Defined in:
- utils/analytics/repository.rb
Overview
This class is part of a private API. This class may only be used in the Homebrew/brew repository. Third parties should avoid using this class if possible, as it may be removed or changed without warning.
Repository analytics derived from local Git objects.
Constant Summary collapse
- SPLIT_COMMITS =
This constant is part of a private API. This constant may only be used in the Homebrew/brew repository. Third parties should avoid using this constant if possible, as it may be removed or changed without warning.
Inherited commits refer to legacy-homebrew PR numbers before these split commits.
T.let({ "Homebrew/brew" => "c5132daf630", "Homebrew/homebrew-core" => "c491e31616b4", }.freeze, T::Hash[String, String])
- FIRST_LEGACY_PULL_REQUEST =
This constant is part of a private API. This constant may only be used in the Homebrew/brew repository. Third parties should avoid using this constant if possible, as it may be removed or changed without warning.
Legacy issue #2326 (1 September 2010) is the first surviving numbered PR.
2326
Instance Attribute Summary collapse
-
#author_identities ⇒ Hash{String => Array<String>}
readonly
private
Mailmapped email identities by UTC year, hex-encoded for non-UTF-8 history.
Instance Method Summary collapse
-
#catalogue_count(revision, formula_repository:) ⇒ Integer
private
Count package files in a historical tree without following aliases.
- #git(*args) ⇒ String private
- #initialize(path, name) ⇒ void constructor private
-
#pull_request_numbers(message) ⇒ Array<String>
private
Extract PR identities from merge subjects and closing references in this repository.
- #rows ⇒ Array<Hash{String => String, Integer, nil}> private
Constructor Details
#initialize(path, name) ⇒ void
This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.
24 25 26 27 28 |
# File 'utils/analytics/repository.rb', line 24 def initialize(path, name) @path = path @name = name @author_identities = T.let({}, T::Hash[String, T::Array[String]]) end |
Instance Attribute Details
#author_identities ⇒ Hash{String => Array<String>} (readonly)
This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.
Mailmapped email identities by UTC year, hex-encoded for non-UTF-8 history.
21 22 23 |
# File 'utils/analytics/repository.rb', line 21 def @author_identities end |
Instance Method Details
#catalogue_count(revision, formula_repository:) ⇒ Integer
This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.
Count package files in a historical tree without following aliases.
196 197 198 199 200 201 202 203 204 205 206 207 |
# File 'utils/analytics/repository.rb', line 196 def catalogue_count(revision, formula_repository:) git("ls-tree", "-rz", revision).split("\0").count do |entry| , path = entry.split("\t", 2) next false unless &.match?(/\A100(?:644|755) blob /) if formula_repository path&.match?(%r{\A(?:Library/)?Formula/(?:[^/]+/)?[^/]+\.rb\z}) else path&.match?(%r{\ACasks?/(?:[^/]+/)*[^/]+\.rb\z}) end end end |
#git(*args) ⇒ String
This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.
249 250 251 |
# File 'utils/analytics/repository.rb', line 249 def git(*args) Utils.safe_popen_read("git", "-C", @path.to_s, *args).strip end |
#pull_request_numbers(message) ⇒ Array<String>
This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.
Extract PR identities from merge subjects and closing references in this repository.
211 212 213 214 215 216 217 218 219 220 221 222 223 224 |
# File 'utils/analytics/repository.rb', line 211 def pull_request_numbers() subject = .lines.first.to_s.chomp numbers = [subject[/\AMerge pull request #(\d+)\b/, 1], subject[/\(#(\d+)\)\z/, 1]].compact .scan(/^Closes (.+?)\.?$/i).flatten.each do |reference| number = reference[/\A#(\d+)\z/, 1] || reference[/\A#{Regexp.escape(@name)}#(\d+)\z/i, 1] || reference[%r{\Ahttps://github\.com/#{Regexp.escape(@name)}/pull/(\d+)\z}i, 1] numbers << number if number end if @name == "Homebrew/legacy-homebrew" numbers.reject! { |number| number.to_i < FIRST_LEGACY_PULL_REQUEST } end numbers.uniq end |
#rows ⇒ Array<Hash{String => String, Integer, nil}>
This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.
31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 |
# File 'utils/analytics/repository.rb', line 31 def rows raise "Analytics require complete Git history for #{@name}" if git("rev-parse", "--is-shallow-repository") == "true" head = git("rev-parse", "HEAD") mailmap = git("ls-tree", "HEAD", "--", ".mailmap") cask_repository = @name == "Homebrew/homebrew-cask" formula_repository = ["Homebrew/homebrew-core", "Homebrew/legacy-homebrew"].include?(@name) release_repository = ["Homebrew/brew", "Homebrew/legacy-homebrew"].include?(@name) = if release_repository git("for-each-ref", "--merged=HEAD", "--format=%(refname:strip=2)%09%(*committerdate:unix)%09%(committerdate:unix)", "refs/tags") else "" end cache = HOMEBREW_CACHE/"analytics/git/#{@name}.json" data = cache.exist? ? JSON.parse(cache.read) : {} generator = Digest::SHA256.file(__FILE__).hexdigest if data["head"] && data.values_at("mailmap", "generator") == [mailmap, generator] begin git("merge-base", "--is-ancestor", data.fetch("head"), "HEAD") rescue ErrorDuringExecution data = {} end else data = {} end year = Time.now.utc.year if data.values_at("head", "year", "tags") == [head, year, ] @author_identities = data.fetch("authors") return data.fetch("rows") end = data.fetch("authors", {}) range = data["head"] ? "#{data.fetch("head")}..HEAD" : "HEAD" git("log", "--no-merges", "--format=%ct%x09%aE", range).each_line do |line| , email = line.chomp.split("\t", 2) next if .nil? || email.nil? commit_year = () # Historical author addresses can contain non-UTF-8 bytes. ([commit_year] ||= []) << email.unpack1("H*") end .transform_values!(&:uniq) @author_identities = first_years = .keys.sort.each_with_object({}) do |period, first| .fetch(period).each { |identity| first[identity] ||= period } end.values.tally pull_requests = data.fetch("pull_requests", {}) excluded = [] if (split = SPLIT_COMMITS[@name]) && system("git", "-C", @path.to_s, "cat-file", "-e", "#{split}^{commit}", out: File::NULL, err: File::NULL) excluded << "^#{split}^" end git("log", "--first-parent", "--reverse", "--format=%ct%x00%B%x00", range, *excluded) .split("\0").each_slice(2) do |, | next if .nil? pull_request_numbers().each do |number| pull_requests[number] ||= () end end merged_years = pull_requests.values.tally changes = data.fetch("changes", {}) if cask_repository period = "" tokens = git("log", "--first-parent", "--reverse", "--raw", "--root", "--no-abbrev", "--diff-merges=first-parent", "--find-renames=100%", "--format=%x00year%x00%ct", "-z", range).split("\0") while (token = tokens.shift) if token == "year" period = (tokens.shift) next end next unless token.lstrip.start_with?(":") old_mode, new_mode, _, _, status = token.delete_prefix("\n").delete_prefix(":").split old_path = tokens.shift.to_s new_path = status&.start_with?("R") ? tokens.shift.to_s : old_path before = cask_entry?(old_mode, old_path) after = cask_entry?(new_mode, new_path) next if before == after metric = after ? "added" : "removed" period_changes = (changes[period] ||= {}) period_changes[metric] = period_changes.fetch(metric, 0) + 1 end end revisions = data.fetch("year_end_revisions", {}) missing_years = (2009...year).reject { |y| revisions.key?(y.to_s) } if missing_years.any? timeline = git("log", "--first-parent", "--format=%ct%x09%H", "HEAD").lines.map do |line| , revision = line.chomp.split("\t", 2) [.to_i, revision] end missing_years.each do |y| revisions[y.to_s] = timeline.find { |, _| < Time.utc(y + 1).to_i }&.last end end release_years = .lines.filter_map do |line| tag, annotated_date, commit_date = line.chomp.split("\t", -1) next unless tag&.match?(/\Av?\d+(?:\.\d+){1,2}\z/) (annotated_date.presence || commit_date) end.tally catalogues = data.fetch("catalogues", {}) maintainers = data.fetch("maintainers", {}) rows = ["current", *(2009..year).map(&:to_s).reverse].map do |period| current = period == "current" revision = (current || period == year.to_s) ? head : revisions[period] catalogue = if @name == "Homebrew/brew" || revision.nil? 0 else catalogues[revision] ||= catalogue_count(revision, formula_repository:) end if @name == "Homebrew/brew" && revision && !maintainers.key?(revision) readme = git("ls-tree", "--name-only", revision).lines(chomp: true).find do |file| file.match?(/\AREADME(?:\.md)?\z/) end identities = if readme git("show", "#{revision}:#{readme}").split(/\n\s*\n/).select do |paragraph| paragraph.match?(/\b(?:maintainers?|Project Leader|Technical Steering Committee)\b/i) && !paragraph.match?(/\bformer\b/i) end.join("\n").scan(%r{\]\(https?://github\.com/([\w-]+)/?\)}).flatten.map(&:downcase).uniq end maintainers[revision] = identities&.presence&.size end contributors = (current ? .values.flatten : .fetch(period, [])).uniq.size { "period" => period, "revision" => revision, "contributors" => contributors, "new_contributors" => current ? contributors : first_years.fetch(period, 0), "maintainers" => maintainers[revision], "pull_requests_merged" => current ? pull_requests.size : merged_years.fetch(period, 0), "added" => if cask_repository change_count(changes, period, "added") end, "removed" => if cask_repository change_count(changes, period, "removed") end, "formulae" => formula_repository ? catalogue : nil, "casks" => cask_repository ? catalogue : nil, "releases" => if release_repository if current release_years.values.sum else release_years.fetch(period, 0) end end, } end cache.dirname.mkpath cache.atomic_write(JSON.generate({ head:, year:, mailmap:, tags:, authors:, generator:, year_end_revisions: revisions, catalogues:, maintainers:, changes:, pull_requests:, rows: })) rows end |