Get Page (Web Mining)
SynopsisGets a page via HTTP.
This operator sends a GET request via HTTP. The returned page is output as a document.
The output port.
- urlThe URL from which should be read. Range:
- random_user_agentChoose a user agent randomly from a set of 7000 user agents Range:
- user_agentThe user agent property. Range:
- connection_timeoutThe timeout (in ms) for the connection. Range:
- read_timeoutThe timeout (in ms) for reading from the URL. Range:
- follow_redirectsSpecifies, whether redirects should be followed. Range:
- accept_cookiesSpecifies, whether cookies should be accepted. Range:
- cookie_scopeSpecifies the scope of the cookies used Range:
- request_methodSpecifies the request method. Range:
- query_parametersThe query parameters as key/value pairs. Range:
- request_propertiesWith this parameter you can define all properties that are sent with the HTTP request to match the needs of your webservice. Range:
- override_encodingNormally, the encoding of the retrieved page is determined automatically. In some rare cases this does not work well or the server provides a wrong encoding string. In this case, you can enable this option to override the automatically detected encoding. Range:
- encodingThe encoding used for reading or writing files. Range:
- keep_sensitive_headersKeep "Authorization" and "Cookie" header during a redirect to a different domain or subdomain. Range: