-
Notifications
You must be signed in to change notification settings - Fork 1
/
Changelog
70 lines (51 loc) · 2.25 KB
/
Changelog
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
07/14/2011:
* Added a sample Test for Database
* Added scala test-1.1.jar
How to compile and run test:
-------------------------------------------
scalac -cp scalatest-1.1.jar DatabaseTestSuite.scala Database.scala
scala -cp scalatest-1.1.jar:. org.scalatest.tools.Runner -p . -o -s DatabaseTestSuite
07/14/2011:
* Added simple database library.
Usage;
Welcome to Scala version 2.9.0.1 (OpenJDK Server VM, Java 1.6.0_22).
Type in expressions to have them evaluated.
Type :help for more information.
scala> import Database._
import Database._
scala> val db1 = DatabaseFactory.get("http://deneme.com")
db1: Database.Database = Database.Database@15651df
scala> val db2 = DatabaseFactory.get("http://asd.com")
db2: Database.Database = Database.Database@1080f38
scala> val db3 = DatabaseFactory.get("http://deneme.com")
db3: Database.Database = Database.Database@15651df
scala> DatabaseFactory.databases
http://deneme.com
http://asd.com
scala> db1.add("/page-1233.html")
scala> db1.add("http://deneme.com/page-not-found.html")
scala> db1.print_entries
Domain : http://deneme.com
Entry :Map(url -> /page-not-found.html, visited -> false, crawled -> false, hash -> -588173248)
Entry :Map(url -> /page-1233.html, visited -> false, crawled -> false, hash -> 105208169)
scala> db1.remove("http://deneme.com/page-not-found.html")
res4: Boolean = true
scala> db1.print_entries
Domain : http://deneme.com
Entry :Map(url -> /page-not-found.html, visited -> false, crawled -> false, hash -> -588173248)
Entry :Map(url -> /page-1233.html, visited -> false, crawled -> false, hash -> 105208169)
07/13/2011:
* Added jsoup library.
How to compile and run jsoup based crawler:
-------------------------------------------
scalac -cp ./jsoup-1.6.1.jar Crawler.scala
scala -cp ./jsoup-1.6.1.jar:. Crawler http://www.pandora.com.tr/urun/hayvan-alfabesi/236371
Useful resources:
-----------------
jsoup Cookbook : http://jsoup.org/cookbook/
jsoup API : http://jsoup.org/apidocs/
07/12/2011:
* Added tagsoup parsing usage.
after installing tagsoup java library, following commands supposed to work in linux;
scalac -classpath /usr/share/java/tagsoup.jar parser.scala
scala -classpath "/usr/share/java/tagsoup.jar:/usr/share/java/scala-library.jar:." deneme