Create a huge collection of all tamil books metadata

[tshrinivasan]

There are many websites with tamil books details

• https://www.projectmadurai.org/
• https://www.panuval.com
• tamilvu.org
• https://www.tamildigitallibrary.in/
• freetamilebooks.com
• https://www.amazon.in/s?bbn=10837926031&rh=n%3A10837926031%2Cp_n_feature_three_browse-bin%3A10837941031&dc=&language=ta

etc

list all the websites that have all the books details.

scrap them all the data and publish.

create a portal with all the book's metadata.

Once the data is collected, we can publish as a website with Omeka S https://omeka.org/s/ or Islandora https://www.islandora.ca/

[Natkeeran]

Some of the current datasets can be found here:

https://github.com/KaniyamFoundation/tamilbooks_metadata/tree/main/data

[kjanani27]

https://archive.org/
www.storytel.com
https://uvesalibrary.org/
www.goodreads.com
https://kannadhasanpathippagham.com
https://www.alliancebook.com
https://tamilvanan.com/

[amotbeli]

https://www.noolulagam.com/
https://dialforbooks.in/
https://www.annacentenarylibrary.org/pages/view/library-catalogue

[HariharanUmapathi]

I'm Scrapping text from https://www.projectmadurai.org/
git hub link comming soon....

Edit 1:
Github : https://github.com/HariharanUmapathi/programmerlife/tree/kaniyam-book-list-scraper/Python/book-list-scrapper

Free free to create github issues regarding the code

[rajkannan1978]

I am scraping book details from https://www.panuval.com
Soon will create git hub link.

[amotbeli]

I'm scraping data from the Anna Centenary Library catalogue.

GitHub link is here.

[rajkannan1978]

I am scraping data from Panuval book store.
Posted my code to git.
GitHub link is https://github.com/rajkannan1978/web-scraping.git
Please let me know if any bugs there.
I welcome any suggestions to improve the code.

[rajkannan1978]

Hello,
Posting some of my python projects are here.
Web Scraping https://github.com/rajkannan1978/web-scraping.git
Grocery https://github.com/rajkannan1978/grocery.git
Number Guess Game https://github.com/rajkannan1978/number_guess_game.git

Thanks.

[rajkannan1978]

Hi,
Got 18708 books from panuval.com
Web Scraping https://github.com/rajkannan1978/web-scraping.git

[amotbeli]

Got 15845 books from the Anna Centenary Library catalogue.

See here.

[rajkannan1978]

Super. I visited the website. Also learned from your code. It is neat and clean. Thanks.

…

On Thu, Sep 12, 2024 at 7:15 AM amotbeli ***@***.***> wrote: Got 15845 books from the Anna Centenary Library catalogue. See here. <https://github.com/amotbeli/acl_data/blob/main/acl_data.json> — Reply to this email directly, view it on GitHub <#228 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BKZNVCPFE2TGLSQWIZJRIW3ZWDW3FAVCNFSM6AAAAABM4G32EWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNBVGA4DSMZSGA> . You are receiving this because you commented.Message ID: ***@***.***>

[amotbeli]

Thank you, rajkannan1978!

[tshrinivasan]

We need a central place to store and display all the books metadata.

Explored Omeka https://omeka.org/ and Islandora https://www.islandora.ca
Installed both as https://omeka.kaniyam.cloudns.nz/ and https://islandora.kaniyam.cloudns.nz/

Omeka is missing features of customizing themes and appearance. custom fields is tough. can not store the files on a remote server. can not translate easily. less plugins.

Islandora has more features, we can customise the display, it has bilingual capacity. As it is a drupal based project, all the drupal's powers comes along, with tons of drupal plugins.

Here is the doc for lightweight islandora installation.
https://github.com/digitalutsc/islandora_lite_docs/wiki/7.-Installation

Thanks to @Natkeeran for the guidance on the setup.

Hence, going with islandora. seeing this video to learn the basics - https://www.youtube.com/watch?v=dfc7WUGAmow

Will explore on how to add data .

[kamalaak]

I’ve scraped 50,000+ book details from the www.noolulagam.com
Here’s the repository: https://github.com/kamalaak/noolulagam_books_scraping.git

[tshrinivasan]

Wonderful. thanks. Please scrap the cover images also.

[kamalaak]

I scraped 50,000+ books with their images. The images are stored in a separate folder, and the CSV file includes the paths to those images.

Here’s the link to the images folder and the repository with the CSV:

CSV: https://github.com/kamalaak/books_scraping/blob/main/with_books.csv

Images: https://drive.google.com/file/d/19z8WCYSoNIxo1nOVzHJtEttFpKDHkeET/view?usp=drivesdk

[kamalaak]

I scraped details of 20,000+ books, including images, from https://dialforbooks.in. However, some books are missing images. Here are the CSV and image links.

https://github.com/kamalaak/books_scraping/blob/main/cleaned_file.csv
https://drive.google.com/drive/folders/17wB8tsKXU6tYPmEebn5sXwXnsbco_PaO?usp=drive_link