Skip to content

Testspace: Guide: Add a new foreign scraper

Eric Delaporte edited this page Aug 2, 2023 · 1 revision

Add a new foreign scraper

Implement ForeignScrapperer interface at pkg/dia/scraper/foreign-scrapers/Scrapper.go file

type ForeignScrapperer interface {
	UpdateQuotation() error
	GetQuoteChannel() chan *models.ForeignQuotation
}

Add the scraper implementation a MyForeignScraper.go Golang file to pkg/dia/scraper/foreign-scrapers/ folder and add the next implementations:

func (scraper *MyForeignScraper) Pool() chan dia.Pool {}
func (scraper *MyForeignScraper) Done() chan bool {}
  1. Add a constructor for the scraper:
func NewMyForeignScraper(exchange dia.Exchange) *MyForeignScraper {}
  1. Add MyForeignScraper case at scraper file cmd/foreignscraper/foreign.go:
switch *scraperType {
	case "MyForeignScraper":
		log.Println("Start scraping data")
		sc = scrapers.NewMyForeignScraper(ds)
}

Foreign Command

go mod tidy -go=1.16 && go mod tidy -go=1.17 && go install && foreignscraper -foreignType=ForeignType

Foreign Test

go mod tidy -go=1.16 && go mod tidy -go=1.17 && go install && foreignscraper -foreignType=FOREIGN_TYPE