Go API for Finviz
Home Page: https://github.com/d3an/finviz/wiki
License: MIT License
Hey, I'm getting this error lately, when calling symbol LABU:getQts, err = client.GetQuotes(symbols).
It's not caught by err. Is it possible to fix it? 🙏
Add functions and a CLI subcommand to support the Groups page on FinViz.
Hey there, thanks for the great api.
I'd like to get e.g. a particular's ticker Current Ratio. Based on the screener example on your home page, If I do e.g.
client := quote.New(nil)
stocks, err := client.GetQuotes([]string{"AAPL", "TSLA"})
all parameters/columns yield no information, please see the pic:
Am I doing sth wrong? Any suggestion appreciated 👍
Check out Bubble Tea framework. Might be nicer than Cobra for creating user-friendly CLIs.
Dataframes are currently returned in the format they were created in. That means that a column that should be in percent is actually represented in string format (i.e. "+35.79%" vs. 0.3579) Create cleaning functions that "type casts" data into the proper format for further analysis.
Add breadth-first iteration to the Bulk Tickers views (510 & 520) to preserve order. The tickers in these views are sorted in a horizontal fashion, but are structured (HTML) vertically. i.e. You have an array of td elements (columns) and within each element is an array of tr elements (tickers). Implement a method to output a dataframe of tickers (and data) in the correct order specified.
The async approach to screener/quote downloads may result in a temporary/permanent service block. Switch to a strict synchronous approach to better avoid Cloudflare roadblocks.
Finviz/Cloudflare may have blocked some of the user agents generated for requests. Update the user agent library for more modern options. Blocked user agents result in HTTP status code 403 and Cloudflare "error code: 1010".
Consider conducting research on which user agents have been compromised (1423 options).
Add go-vcr package. This package is likely based on the pyvcr package for Python. Its incredibly useful for testing, permitting the avoidance of writing custom patch and mock objects.
Currently, the maximum number of securities returned in a screen is 20 (free version). Write a function that allows for multiple pages of screen results to be downloaded. Consider running a recursive operation on the existence of the next button at the bottom of the page.
Allow the default 20 number of rows to be modified by users. Finviz Elite allows this number to be varied up to 100 rows. If it's too much of a hassle, don't bother.
Fix the module name to be github.com/d3an/finviz
Recently, I am getting code error 429 often. I have not got it for last 8 months with the same code.
Has FinViz put some limit on requests? Has anyone getting the same error? Thanks..
ScreenInputReader function that would consume a JSON object (marshalled/unmarshalled?) or maybe a byte array and extract a ScreenInput object from the input.screener.go into a better designed approach. Use interfaces where necessary.This is good functionality to add and may help with writing Python bindings later on.
GetQuoteData into GetFullQuoteData and GetBasicQuoteData. The difference is that GetFullQuoteData also returns the special sections (Analyst Recommendations, News, Insider Trading, Description, Chart) in its DataFrame.GetInsiderTrading, GetNews, GetAnalystRecommendations, GetDescription, GetChart.I am downloading data monthly for +12k symbols. All of a sudden, I am getting this panic which points to the library script, i.e. quote.go.
Could this be fixed internally please?
Evaluate whether certain components should be abstracted, i.e., Client, the backoff procedure, and/or the general request procedure.
Using gopy, figure out if it's possible to export this library as a Python package, especially as to whether or not PyPI will support it.
There are two primary FinViz packages for Python, finviz and finvizfinance. Both are decently maintained, but still have a lot of room for improvement. If this package can support all features included in the forementioned packages, improve on outstanding issues, and include even more features (AND documentation), Python developers may begin to use this package instead.
The code for filters.go is over 3000 lines. There's a lot of repetition, which can probably be refactored.
There has to be a way to refactor the implementation to be simpler. Although the lack of generics prevents a generalization of types, there has to be a better object-oriented (or composition) way to reduce the complexity of the codebase. Figure it out and clean up the code.
The FilterValueLookup covers all 70 filter types and their corresponding values. The dictionary needs to be fixed up so that each value perfectly mirrors that provided on the site. Once this step is completed, shortcuts should be added as well. Consider the Market Cap filter. A possible shortcut for Mega ($200bln and more) could just be mega.
As a bonus task, consider adding support for approximate string matching. Take a look at the Levenshtein distance metric and see how others have implemented this technique in their projects.
"responsively" -> "responsibly"
Fix the url generation bug in which two question marks appear in the final result instead of one. This doesn't affect functionality, but may stand out among FinViz's other requests.
Every good package has unit tests. Write a few, get the coverage stamp on the README, and maybe find a couple bugs along the way. Having good test coverage is a great way to make sure new code won't break the system on a refactoring or new feature.
The testing package is probably all that's needed, but it's probably a good idea to patch the requests and use mocks where necessary. Take a look at a few of the other go testing packages and see if they're necessary.
filter_test.go
screener_test.go
views_test.go
EDIT: Added the coverage badge. Currently using the testing, govcr, and testify packages to write unit tests.
Create a subcommand that accepts a FinViz url as input and returns a DataFrame (or exports to a file) the resulting scrape of that url. If the FinViz app detailed in the url is not supported, print an informative error message.
Examples:
$ finviz urlscraper https://finviz.com/quote.ashx?t=AAPL
OR
$ finviz usr https://finviz.com/screener.ashx?v=111&f=exch_nasd&ft=4
OR (unsupported app)
$ finviz usr https://finviz.com/groups.ashx?g=industry&sg=consumerdefensive&v=210&o=name
[ERROR] Groups app not supported.FinViz offers several views for viewing screen results. Each view is labelled with a unique ID.
The following views exist:
Add descriptions to filters (or in some lookup) so that CLI users can get better understanding on different filter types.
Coveralls' badge should be good enough. Add it with goveralls.
Add support for the charts view (210) along with the different types of graphs and tick frequencies.
Add properties to filters, i.e. whether or not they support multiple or custom values (custom values is looking ahead to elite support)
Add functionality to support the output of screener results to a csv or json. Also support the creation of external file results for csv or json.
Restructure the existing codebase to support more inherited methods/classes. This extends into supporting other FinViz apps such as News.
Create a partially abstract base class View and interface ViewInterface:
type View struct {}
type ViewInterface interface {
GenerateURL() string
Scrape(document *goquery.Document) ([][]string, error)
}Each app should have a derived object that inherits the predefined interface methods, such as for News:
type NewsView struct {
finviz.View
}Going deeper, an app should derive its actual views from the app view base class, ex:
type SourceNewsView struct {
NewsView
}This restructuring should help greatly with view URL generation as derived objects can call parent methods, thus generating the URL in a step by step basis: ('https://finviz.com' OR 'https://elite.finviz.com') + '/news.ashx' + '?v=2'.
Additionally, this restructuring will help clean up the file structure of the codebase and make it more presentable and less daunting on GitHub.
FinViz also has quote pages for its tickers (quote.ashx). Given a list of tickers, create a function that completes a download and scrape using goroutines for concurrency and efficiency. Add a CLI subcommand for this as well.
$ finviz quote aapl,goog,amzn,tslaEarnings Calendar scrape is producing an empty DataFrame. Fix it.
FinViz has a News page. Add support for it. Create a new subcommand for the CLI to facilitate this as well.
There are only two views. So, two scraping functions are required (probably).
Other CLI libraries, specifically argparse and click for Python, support arguments that must be one of a few given options. Figure out if Cobra supports this, and if it does then implement it. If Cobra does not support it, submit a PR to Cobra to fix it.
Line 160 in c781b63
I haven't come across this, yet, so it caught my eye. After an initial search, I see that it's not recommended due to the memory size implications. Can you explain why a pointer to a channel is used? (Still learning Go btw)
The Gota library has a print() method for dataframes, but the method is private. As such, each call to print a dataframe results in a shortened screen printing. Create a custom PrintDataFrame function to override these actions and allow the dataframe (regardless of size) to print to screen.
Ex.
[
{
"A": {
"Avg Volume": "1.47M",
"Change": "0.00%",
"No.": 1,
"Perf Half": "28.54%",
"Perf Month": "6.04%",
"Perf Quart": "13.47%",
"Perf Week": "4.12%",
"Perf YTD": "21.23%",
"Perf Year": "45.95%",
"Price": 103.42,
"Recom": "2.40",
"Rel Volume": 0.97,
"Ticker": "A",
"Volatility M": "1.80%",
"Volatility W": "1.82%",
"Volume": 0
},
"AA": {
"Avg Volume": "7.33M",
"Change": "0.00%",
"No.": 2,
"Perf Half": "23.38%",
"Perf Month": "7.32%",
"Perf Quart": "26.03%",
"Perf Week": "-2.65%",
"Perf YTD": "-31.80%",
"Perf Year": "-19.75%",
"Price": 14.67,
"Recom": "2.70",
"Rel Volume": 0.77,
"Ticker": "AA",
"Volatility M": "4.72%",
"Volatility W": "4.12%",
"Volume": 0
}
}
]instead of
[
{
"Avg Volume": "1.47M",
"Change": "0.00%",
"No.": 1,
"Perf Half": "28.54%",
"Perf Month": "6.04%",
"Perf Quart": "13.47%",
"Perf Week": "4.12%",
"Perf YTD": "21.23%",
"Perf Year": "45.95%",
"Price": 103.42,
"Recom": "2.40",
"Rel Volume": 0.97,
"Ticker": "A",
"Volatility M": "1.80%",
"Volatility W": "1.82%",
"Volume": 0
},
{
"Avg Volume": "7.33M",
"Change": "0.00%",
"No.": 2,
"Perf Half": "23.38%",
"Perf Month": "7.32%",
"Perf Quart": "26.03%",
"Perf Week": "-2.65%",
"Perf YTD": "-31.80%",
"Perf Year": "-19.75%",
"Price": 14.67,
"Recom": "2.70",
"Rel Volume": 0.77,
"Ticker": "AA",
"Volatility M": "4.72%",
"Volatility W": "4.12%",
"Volume": 0
}
]Although this API still works, Mario's Python Finviz API had an issue where users received 403 error codes on requests. The issue was fixed by randomizing/setting the user agent to be a more common one.
Add support for user agent generation to avoid a possible block in the future.
Refactor CleanScreenerDataFrame to support both Screener df's and Quote df's.
Add functions to support the scrape and export of FinViz's weekly economic calendar. Add a new subcommand to the CLI to support this function.
Upgrade the project's Go version from 1.14 to 1.15. Sort out any bugs that may arise.
Create a base command finviz and add subcommands as functionality increases.
Create a screener subcommand to run screens against Finviz.
Create a help sub-subcommand for screener to help with displaying valid options.
Examples:
$ finviz screener -s TopGainers -o -ChangeFromOpen -f Industry:gold,airlines Price:PriceOver1
# Same as running the RunScreen function and printing the dataframe to screen
$ finviz screener help
# Returns general help for all options (basically lists possible values and syntax)
$ finviz screener help signal filter:industry,sector
# Returns general help for signal and filter, along with specific help for industry, sector filtersI am requesting quotes client.GetQuotes([]symbols) for a bunch of tickers and get panic if a ticker is not available in FinViz, e.g. ticker SHAC is not..
error received while scraping quote for 'SHAC': error getting url: 'https://finviz.com/quote.ashx?t=SHAC&ty=c&p=d&b=1', status code: '404'
I could catch the error, remove the ticker and pull again, but that could be much overhead, specially from FinViz point of view. Could this be fixed internally so the program does not panic but throws warning instead that a ticker is not available?
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.