Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add scrape of ETF top/major holdings #1872

Open
wants to merge 5 commits into
base: dev
Choose a base branch
from

Conversation

dotJoel
Copy link

@dotJoel dotJoel commented Feb 27, 2024

This PR is a a completion of the work started in #1571

To address the issue with returning price information when checking the holdings of a non-fund ticker, it does the following:

  1. check for the presence of a redirect away from /holdings to identify when they are n/a.
  2. return empty list in that case

@ValueRaider
Copy link
Collaborator

ValueRaider commented Mar 2, 2024

Can it be converted to a DataFrame? More usable. Use get_earnings_dates for inspiration.

  • symbol as index
  • convert % Assets to actual number

Documentation - update README.

Add a unit test, in case someone in future breaks your code.

@dotJoel
Copy link
Author

dotJoel commented Mar 3, 2024

It seems that this data is no longer a table on that page?
Top holdings that were a table now seem to be divs.
I swear this worked last week...

I'd rather help with the quoteSummary than parse more html.

@ValueRaider
Copy link
Collaborator

Does the webpage fetch from API? Firefox -> F11 -> Network -> refresh

@dotJoel
Copy link
Author

dotJoel commented Mar 3, 2024

Not directly.
There is a JS script block that has the data that comes from the API.
Seems to be the same API endpoint that the Fund data PR is using.

@ValueRaider
Copy link
Collaborator

btw your commit history is a mess, I can't merge that.

@ValueRaider
Copy link
Collaborator

I've fixed formatting but only tested on IVV. Can people try more ETFs?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants