Module: Utils::Curl::Responses Private

Extended by:
T::Helpers
Included in:
Utils::Curl, Utils::Curl
Defined in:
utils/curl/responses.rb

Overview

This module is part of a private API. This module may only be used in the Homebrew/brew repository. Third parties should avoid using this module if possible, as it may be removed or changed without warning.

Parse HTTP responses and resolve redirect locations from curl output.

Instance Method Summary collapse

Instance Method Details

#curl_response_follow_redirections(responses, base_url) ⇒ String

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Returns the final URL by following location headers in cURL responses.

Parameters:

  • responses (Array<Hash{Symbol => T.untyped}>) —

    An array of hashes containing response status information and headers from #parse_curl_response.

  • base_url (String) —

    The URL to use as a base.

Returns:

  • (String) —

    The final absolute URL after redirections.



119
120
121
122
123
124
125
126
127
128
129
130
 
# File 'utils/curl/responses.rb', line 119

def curl_response_follow_redirections(responses, base_url)
  responses.each do |response|
    next if response[:headers].blank?

    location = Array(response[:headers]["location"]).last
    next if location.blank?

    base_url = URI.join(base_url, location).to_s
  end

  base_url
end

#curl_response_last_location(responses, absolutize: false, base_url: nil) ⇒ String?

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Returns the URL from the last location header found in cURL responses, if any.

Parameters:

  • responses (Array<Hash{Symbol => T.untyped}>) —

    An array of hashes containing response status information and headers from #parse_curl_response.

  • absolutize (Boolean) (defaults to: false) —

    Whether to make the location URL absolute.

  • base_url (String, nil) (defaults to: nil) —

    The URL to use as a base for making the location URL absolute.

Returns:

  • (String, nil) —

    The URL from the last-occurring location header in the responses or nil (if no location headers found).



94
95
96
97
98
99
100
101
102
103
104
105
106
 
# File 'utils/curl/responses.rb', line 94

def curl_response_last_location(responses, absolutize: false, base_url: nil)
  responses.reverse_each do |response|
    next if response[:headers].blank?

    location = Array(response[:headers]["location"]).last
    next if location.blank?

    absolute_url = URI.join(base_url, location).to_s if absolutize && base_url.present?
    return absolute_url || location
  end

  nil
end

#http_status_ok?(status) ⇒ Boolean

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Parameters:

Returns:

  • (Boolean) 


38
39
40
41
42
 
# File 'utils/curl/responses.rb', line 38

def http_status_ok?(status)
  return false if status.nil?

  (100..299).cover?(status.to_i)
end

#parse_curl_output(output, max_iterations: 25) ⇒ Hash{Symbol => T.untyped}

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Separates the output text from curl into an array of HTTP responses and the final response body (i.e. content). Response hashes contain the :status_code, :status_text and :headers.

Parameters:

  • output (String) —

    The output text from curl containing HTTP responses, body content, or both.

  • max_iterations (Integer) (defaults to: 25) —

    The maximum number of iterations for the while loop that parses HTTP response text. This should correspond to the maximum number of requests in the output. If curl's --max-redirs option is used, max_iterations should be max-redirs + 1, to account for any final response after the redirections.

Returns:

  • (Hash{Symbol => T.untyped}) —

    A hash containing an array of response hashes and the body content, if found.



57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
 
# File 'utils/curl/responses.rb', line 57

def parse_curl_output(output, max_iterations: 25)
  responses = []

  iterations = 0
  output = output.lstrip
  while output.match?(%r{\AHTTP/[\d.]+ \d+}) && output.include?(HTTP_RESPONSE_BODY_SEPARATOR)
    iterations += 1
    raise "Too many redirects (max = #{max_iterations})" if iterations > max_iterations

    response_text, _, output = output.partition(HTTP_RESPONSE_BODY_SEPARATOR)
    output = output.lstrip
    next if response_text.blank?

    response_text.chomp!
    response = parse_curl_response(response_text)
    responses << response if response.present?
  end

  { responses:, body: output }
end

#parse_curl_response(response_text) ⇒ Hash{Symbol => T.untyped}

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Parses HTTP response text from curl output into a hash containing the information from the status line (status code and, optionally, descriptive text) and headers.

Parameters:

  • response_text (String) —

    The text of a curl response, consisting of a status line followed by header lines.

Returns:

  • (Hash{Symbol => T.untyped}) —

    A hash containing the response status information and headers (as a hash with header names as keys).



140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
 
# File 'utils/curl/responses.rb', line 140

def parse_curl_response(response_text)
  response = {}
  return response unless (match = response_text.match(HTTP_STATUS_LINE_REGEX))

  # Parse the status line and remove it
  response[:status_code] = match["code"]
  response[:status_text] = match["text"] if match["text"].present?
  response_text = response_text.sub(%r{^HTTP/.* \d+.*$\s*}, "")

  # Create a hash from the header lines
  response[:headers] = {}
  response_text.split("\r\n").each do |line|
    header_name, header_value = line.split(/:\s*/, 2)
    next if header_name.blank? || header_value.nil?

    header_name = header_name.strip.downcase
    header_value.strip!

    case response[:headers][header_name]
    when String
      response[:headers][header_name] = [response[:headers][header_name], header_value]
    when Array
      response[:headers][header_name].push(header_value)
    else
      response[:headers][header_name] = header_value
    end

    response[:headers][header_name]
  end

  response
end

#url_protected_by_cloudflare?(response) ⇒ Boolean

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Check if a URL is protected by CloudFlare (e.g. badlion.net and jaxx.io).

Parameters:

  • response (Hash{Symbol => T.untyped}) —

    A response hash from #parse_curl_response.

Returns:

  • (Boolean) —

    Whether a response contains headers indicating that the URL is protected by Cloudflare.



17
18
19
20
21
22
 
# File 'utils/curl/responses.rb', line 17

def url_protected_by_cloudflare?(response)
  return false if response[:headers].blank?
  return false unless [403, 503].include?(response[:status_code].to_i)

  [*response[:headers]["server"]].any? { |server| server.match?(/^cloudflare/i) }
end

#url_protected_by_incapsula?(response) ⇒ Boolean

This method is part of a private API. This method may only be used in the Homebrew/brew repository. Third parties should avoid using this method if possible, as it may be removed or changed without warning.

Check if a URL is protected by Incapsula (e.g. corsair.com).

Parameters:

  • response (Hash{Symbol => T.untyped}) —

    A response hash from #parse_curl_response.

Returns:

  • (Boolean) —

    Whether a response contains headers indicating that the URL is protected by Incapsula.



29
30
31
32
33
34
35
 
# File 'utils/curl/responses.rb', line 29

def url_protected_by_incapsula?(response)
  return false if response[:headers].blank?
  return false if response[:status_code].to_i != 403

  set_cookie_header = Array(response[:headers]["set-cookie"])
  set_cookie_header.compact.any? { |cookie| cookie.match?(/^(visid_incap|incap_ses)_/i) }
end