Skip to content
This repository has been archived by the owner on Nov 27, 2019. It is now read-only.

ScraperWiki.select or ScraperWiki.save_sqlite first? #24

Open
GildedHonour opened this issue Sep 10, 2014 · 1 comment
Open

ScraperWiki.select or ScraperWiki.save_sqlite first? #24

GildedHonour opened this issue Sep 10, 2014 · 1 comment

Comments

@GildedHonour
Copy link

If I do this

  items.each do |item|
    if ScraperWiki.select("* from swdata where ....).empty? 
       ScraperWiki.save_sqlite(["key123"], item)
     else
       puts "Skipping " + item["key123"]
     end
 end

and the db (or rather table) doesn't exist then I get an exception.

/Users/alex/.rvm/gems/ruby-2.1.2/gems/sqlite_magic-0.0.3/lib/sqlite_magic.rb:49:in `rescue in execute': no such table: swdata (SqliteMagic::NoSuchTable)
  from /Users/alex/.rvm/gems/ruby-2.1.2/gems/sqlite_magic-0.0.3/lib/sqlite_magic.rb:42:in `execute'
  from /Users/alex/.rvm/gems/ruby-2.1.2/gems/scraperwiki-3.0.2/lib/scraperwiki.rb:186:in `select

How can I handle that?

@henare
Copy link
Contributor

henare commented Sep 21, 2015

PlanningAlerts does this in a very hacky way with most of our scrapers:

if (ScraperWiki.select("* from swdata where ....").empty? rescue true)
  ScraperWiki.save_sqlite(["key123"], item)
else
  puts "Skipping " + item["key123"]
end

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants