markcheno / go-quote

Yahoo finance/Google finance/Coinbase/Bittrex/Binance/Tiingo historical quote downloader library and cli written in golang

go-quote

A free quote downloader library and cli

Downloads daily historical price quotes from Yahoo and daily/intraday data from various api's. Written in pure Go. No external dependencies. Now downloads crypto coin historical data from various exchanges.

• Update: 02/15/2024 - Major update: updated to Go 1.22, removed bittrex/binance support, fixed nasdaq/tiingo markets

• Update: 11/15/2021 - Removed obsolete markets, converted to go modules

• Update: 7/18/2021 - Removed obsolete Google support

• Update: 6/26/2019 - updated GDAX to Coinbase, added coinbase market

• Update: 4/26/2018 - Added preliminary tiingo CRYPTO support. Use -source=tiingo-crypto -token=<your_tingo_token> You can also set env variable TIINGO_API_TOKEN. To get symbol lists, use market: tiingo-btc, tiingo-eth or tiingo-usd

• Update: 12/21/2017 - Added Amibroker format option (creates csv file with separate date and time). Use -format=ami

• Update: 12/20/2017 - Added Binance exchange support. Use -source=binance

• Update: 12/18/2017 - Added Bittrex exchange support. Use -source=bittrex

• Update: 10/21/2017 - Added Coinbase GDAX exchange support. Use -source=gdax All times are in UTC. Automatically rate limited.

• Update: 7/19/2017 - Added preliminary tiingo support. Use -source=tiingo -token=<your_tingo_token> You can also set env variable TIINGO_API_TOKEN

• Update: 5/24/2017 - Now works with the new Yahoo download format. Beware - Yahoo data quality is now questionable and the free Yahoo quotes are likely to permanently go away in the near future. Use with caution!

Still very much in alpha mode. Expect bugs and API changes. Comments/suggestions/pull requests welcome!

Copyright 2024 Mark Chenoweth

Install CLI utility (quote) with:

go install github.com/markcheno/go-quote/quote@latest

Usage:
  quote -h | -help
  quote -v | -version
  quote <market> [-output=<outputFile>]
  quote [-years=<years>|(-start=<datestr> [-end=<datestr>])] [options] [-infile=<filename>|<symbol> ...]

Options:
  -h -help             show help
  -v -version          show version
  -years=<years>       number of years to download [default=5]
  -start=<datestr>     yyyy[-[mm-[dd]]]
  -end=<datestr>       yyyy[-[mm-[dd]]] [default=today]
  -infile=<filename>   list of symbols to download
  -outfile=<filename>  output filename
  -period=<period>     1m|3m|5m|15m|30m|1h|2h|4h|6h|8h|12h|d|3d|w|m [default=d]
  -source=<source>     yahoo|tiingo|tiingo-crypto|coinbase [default=yahoo]
  -token=<tiingo_tok>  tingo api token [default=TIINGO_API_TOKEN]
  -format=<format>     (csv|json|hs|ami) [default=csv]
  -adjust=<bool>       adjust yahoo prices [default=true]
  -all=<bool>          all in one file (true|false) [default=false]
  -log=<dest>          filename|stdout|stderr|discard [default=stdout]
  -delay=<ms>          delay in milliseconds between quote requests

Note: not all periods work with all sources

Valid markets:
etf,nasdaq,nasdaq100,amex,nyse,megacap,largecap,midcap,smallcap,microcap,nanocap,
telecommunications,health_care,finance,real_estate,consumer_discretionary,
consumer_staples,industrials,basic_materials,energy,utilities
coinbase,tiingo-usd,tiingo-btc,tiingo-eth

CLI Examples

# display usage
quote -help

# downloads 5 years of Yahoo SPY history to spy.csv 
quote spy

# downloads 1 year of bitcoin history to BTC-USD.csv
quote -years=1 -source=coinbase BTC-USD

# downloads 1 year of Yahoo SPY & AAPL history to quotes.csv 
quote -years=1 -all=true -outfile=quotes.csv spy aapl

# downloads full etf symbol list to etf.txt, also works for nasdaq,nasdaq100,nyse,amex
quote etf

# download fresh etf list and 5 years of etf data all in one file
quote etf && quote -all=true -outfile=etf.csv -infile=etf.txt 

Install library

Install the package with:

go get github.com/markcheno/go-quote@latest

Library example

package main

import (
	"fmt"
	"github.com/markcheno/go-quote"
	"github.com/markcheno/go-talib"
)

func main() {
	spy, _ := quote.NewQuoteFromYahoo("spy", "2016-01-01", "2016-04-01", quote.Daily, true)
	fmt.Print(spy.CSV())
	rsi2 := talib.Rsi(spy.Close, 2)
	fmt.Println(rsi2)
}

License

MIT License - see LICENSE for more details

go-quote's Issues

Error: "main.go:353: invalid market"

The quickstart examples does not work:

$ go/bin/quote spy
quote: 2021/11/15 11:31:20 main.go:353: invalid market
$ go/bin/quote -years=1 -all=true -outfile=quotes.csv spy aapl
quote: 2021/11/15 11:31:32 main.go:353: invalid market
$ go/bin/quote -years=1 -source=coinbase BTC-USD
quote: 2021/11/15 11:31:44 main.go:353: invalid market

While installing CLI quote, Bitdefender throws a virus detected error

I tried to install CLI quote running go install github.com/markcheno/go-quote/quote and Bitdefender throws the following error:
"The file C:\Users\XXXXX\AppData\Local\Temp\go-build3256829184\b001\exe\a.out.exe is infected with Gen:Variant.Bulz.368851"
and the install is not allowed to execute since Bitdefender denies access to the file.

QuoteFromCSV ignores last data

NewQuoteFromCSV method ignores last csv data, I guess this is because of -1 in code:

numrows := len(tmp) - 1

Can you add proxy and timeout for API?

like NewQuoteFromYahoo, proxy and timeout parameter is useful when concurrent data downloading. Thank you.

func SetProxy(client *http.Client, proxyUrlString string) error {
	if proxyUrlString == "" {
		return nil
	}
	ui_proxy, err := url.Parse(proxyUrlString)
	if err != nil {
		return err
	}
	transport := http.DefaultTransport.(*http.Transport)
	if client.Transport != nil {
		transport = client.Transport.(*http.Transport)
	}
	transport.Proxy = http.ProxyURL(ui_proxy)
	client.Transport = transport
	return nil
}

func SetTimeout(client *http.Client, to time.Duration) {
	client.Timeout = to
}

Getting segfaults that shouldn't happen (I think)

My code:

package main

import (
	"flag"
	"fmt"
	"os"
	"strings"

	"github.com/markcheno/go-quote"
	"github.com/markcheno/go-talib"
)

var start *string
var end *string
var stock *string

func init() {
	if os.Args[0] == "quote" {
		start = flag.String("start", "", "starting point of your stock value")
		end = flag.String("end", "", "ending point of your stock value")
		stock = flag.String("stock", "", "stock name")
		flag.Parse()
	}
}

func main() {
	spy, err := quote.NewQuoteFromYahoo(strings.ToLower(*stock), *start, *end, quote.Daily, true)
	if err != nil {
		panic(err)
	}
	rsi2 := talib.Rsi(spy.Close, 2)
	fmt.Println(rsi2)
}

My error

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x652a29]

goroutine 1 [running]:
main.main()
        /home/tarekali/code/ETF-sim/main.go:25 +0x39
exit status 2

coinmarketcap data via yahoo finance

Does go-quote support obtaining the coinmarketcap data via Yahoo Finance? https://finance.yahoo.com/cryptocurrencies/

Running quote CLI command on Pi 3 returns word in quotes

Hi, I've downloaded the quote package and installed it on my Pi 3. The OS is "Raspbian GNU/Linux 8 (jessie)" and the go version is go version go1.10.2 linux/arm. I can run other go applications, for instance the "Hello World" app from petergloor. However, when I try to run the quote application by running quote etf or quote spy from the CLI, what I get is 'etf' or 'spy' returned respectively. The quote application seems to be conflicting with the built in quote os apllication. Am I missing something? Let me know, thanks!

README CLI Examples section has a typo

No data from quote.NewQuoteFromGoogle()

I am getting no data when I try to access data from Google. Data from Yahoo are accessed without issue. Here are the codes for each.

Yahoo
stock, err := quote.NewQuoteFromYahoo(tickerstr, todayb, today, quote.Daily, true)

Google
stock, err := quote.NewQuoteFromGoogle(tickerstr, todayb, today, quote.Daily)

My dates

currentTime := time.Now()
previousdays := 60
previousTime := currentTime.AddDate(0, 0, -previousdays)
today := currentTime.Format(layoutISO)
todayb := previousTime.Format(layoutISO)

[Feature suggestions] VWAP & trade count for binance data

Thank you for the great package. I'm new here and wonder if it's possible to add VWAP and trades count for bars for Binance data (possibly also other data)? This is something quite standard and very useful when it comes to bars being minute, daily, etc. Thank you 🙇‍♂️

README.md error: "main.go:353: invalid market

$ go/bin/quote spy
quote: 2021/11/15 11:31:20 main.go:353: invalid market
$ go/bin/quote -years=1 -all=true -outfile=quotes.csv spy aapl
quote: 2021/11/15 11:31:32 main.go:353: invalid market
$ go/bin/quote -years=1 -source=coinbase BTC-USD
quote: 2021/11/15 11:31:44 main.go:353: invalid market

Recommendation: Add adjusted data **besides** orginal data into the output

How it is currently being done:

The output will either render the original close data point OR the adjusted close data point.
With this practice, you're reducing information irrevocably and reducing the meaningfulness of the data you're gathering.

EDIT: After reviewing the code further in detail, I see that the adjusted close and original close are swapped. When adjustQuote is true, original data is being rendered besides the adjusted close, or adjusted data is being rendered besides original close. Bug or Feature?

How it would be advised:

The output should compile (if adjusted data is given in from the source) a list with the original data columns AND adjusted data columns (see e.g. https://www.quandl.com/databases/WIKIP/data). Adjusted open, high, low prices can be rendered by using the ratio between original close and adjusted close, multiplied by the each original data column.

So given a data-source where you could gain maximum information as summary per trading day (incl. ex-dividend and split), the output would look like this:

• date / timestamp
• open
• high
• low
• close
• volume
• ex-dividend
• split
• adj-open
• adj-high
• adj-low
• adj-close
• adj-volume

If you want to stick to your output structure, I'd recommend to either output bare unadjusted data or pure adjusted data.
Otherwise you get a distorted output like the following (e.g. AAPL from yahoo):

datetime,open,high,low,close,volume
2015-04-27 00:00,132.31,133.13,131.15,122.73,96954200.00
2015-04-28 00:00,134.46,134.54,129.57,120.80,118924000.00

The spread between OHL data to AdjClose is almost 10% in this case. This distorts OHLC-based analytics.

Please improve docs.

Why i can't get hourly quotes?

quote.NewQuoteFromYahoo("F", "2020-05-01 10:00", "2020-05-24 23:59", "1h", true)

Reading From CSV files into quote object causes error

When reading a csv file into a quote object there is a slight error which needs to be checked before reading the file.

NewQuoteFromCSV(symbol, csv string) (Quote, error) {

	tmp := strings.Split(csv, "\n")
	numrows := len(tmp)
	q := NewQuote(symbol, numrows-1)

	for row, bar := 1, 0; row < numrows; row, bar = row+1, bar+1 {
		line := strings.Split(tmp[row], ",") # we need to add in a conditional check as such
                if len(line) != 6 {
			break
		}

The reason being is that the way that the "records" are being written a "\n" is automatically appended at the end, this simple check worked and helped me avoid the annoying out of bounds error.

Google functionality failing

The google functionality is failed as the Google Finance API no longer has support from google. I would request that this code be removed or commented out. I know this project is somewhat abandoned so perhaps marking it as so would be helpful as well. Many many thanks for building out the features here.

google and tse

Hi,

Congrats for your software, it is very easy to use!
I'm trying to get the data for a stock listed on the Toronto Stock Exchange from Google.
I tried various combination of way to enter my stock without success.
Usually It is referred with a prefix "TSE:"
When I try TSE:XSP it does not work. I tried to escape the : with \ without success.

See : https://finance.google.ca/finance?q=TSE%3AXSP&sq=TSE%3AXSP&sp=1&ei=HpmAWtibCdaKe_eLv7AI

runtime error:quote: 2022/05/17 10:39:30 quote.go:566: bad data for symbol 'spy'

This example can not be run

gitignore .vscode directory

http client timeout should be set to non-infinity default

Inside quote.go client := &http.Client{} is used but we should have some sane timeout setting there. E.g.: client := &http.Client{Timeout: 10 * time.Second}

Even better, there should probably be a way to inject your own http client.

Create the cli tool using spf13/cobra library

It enables to easily document your cli application, and code is more readable
source: https://github.com/spf13/cobra

README Library example broken

Hello, the example in the readme no longer works for getting started. Thanks!

Port library to go modules