Skip to content

Version 4.2.8
Compare
Choose a tag to compare
@strohne strohne released this 24 Mar 09:30
· 435 commits to master since this release
v4.2.8
0c4b4cc

Note: At the moment, Facepager is under heavy reconstruction, a bunch of features is under development. Keep an eye on the status log. If you encounter any bugs or black cats, update to the latest version and report in the issues section. After updating to a new version, reinstalling default API definitions from GitHub may be necessary: start Facepager, wait for the message in the status log, restart Facepager.

Latest changes:

  • Webscraping features in the Generic Module: Set the response format to "text" and you will find the HTML source code of downloaded pages in the text property. Then, you can use CSS selectors, XPath and regular expression to extract data. See the wiki for a very brief explanation.
  • Preview in the Extract data dialog. This will greatly help you with webscraping. Type keys such as text|xpath://a and you will directly see the HTML content of all a-elements. Clicking Apply creates new nodes. Or you can devlop your keys here and enter them into the column setup or even in the placeholders for further fetching actions.
  • Renaming of keys in the column setup or when extracting data. Prefix your key with newname=, for example links=text|xpath://@href will save all links contained in the text property under the new key links.
  • Resume canceled data collection, even with pagination. See the tooltip of the Resume collection checkbox.
  • Option to stop pagination based on data from a request (e.g. stop if the value "hasnextpage" is empty)
  • Login using cookies: Authorization=header; Name=Cookie; Click settings button next to login button, choose "Cookie", add URL of website, then click Login. After logging into the website, the cookies are transfered to the access token field. Close the login window.
  • Detect rate limit in Generic module (status 429)
  • Timestamp modifier |timestamp in keys converts timestamp to date & time.
  • API key support in YouTube module
  • Improved error logging (request errors won't stop the whole process, error nodes are created instead)
  • Separate all connections. Each request gets its own session now.
  • Support empty keys for extracting the object ID (e.g. to get IDs of Twitter followers or friends).
  • Pro tip: You can build a pipeline by creating multiple presets in the same category and then apply the category.
  • Bug fix: bring tool windows to front on OSX (preset window, api viewer, extract data dialog)
Assets 4
Loading