softwarehistorysociety / twitter-stream

A Go wrapper for Twitter's V2 Filtered Stream API

TwitterStream

TwitStream is a Go library for creating streaming rules and streaming tweets with Twitter's v2 Filtered Streaming API. See examples to start adding your own rules and start streaming.

Installation

go get github.com/fallenstedt/twitter-stream

Examples

Starting a stream

Obtain an Access Token using your Twitter Access Key and Secret.

You need an access token to do any streaming. twitterstream provides an easy way to fetch an access token.

	tok, err := twitterstream.NewTokenGenerator().SetApiKeyAndSecret("key", "secret").RequestBearerToken()

	if err != nil {
		panic(err)
	}

Create a streaming api

Create a twitterstream instance with your access token from above.

	api := twitterstream.NewTwitterStream(tok.AccessToken)

Set your unmarshal hook

It is encouraged you set an unmarshal hook for thread-safety. Go's bytes.Buffer is not thread safe. Sharing a bytes.Buffer across multiple goroutines introduces risk of panics when decoding json source. To avoid panics, it's encouraged to unmarshal json in the same goroutine where the bytes.Buffer exists. Use SetUnmarshalHook to set a function that unmarshals json.

By default, twitterstream's unmarshal hook will return []byte if you want to live dangerously.

    api.Stream.SetUnmarshalHook(func(bytes []byte) (interface{}, error) {
        // StreemData is a struct that represents your returned json
    	// This is a quick resource to generate a struct from your json
        // https://mholt.github.io/json-to-go/
        data := StreamData{}
        if err := json.Unmarshal(bytes, &data); err != nil {
            log.Printf("Failed to unmarshal bytes: %v", err)
        }
        return data, err
    })

Start Stream

Start your stream. This is a long-running HTTP GET request. You can get specific data you want by adding query params. Additionally, view an example of query params here, or in the examples

    err := api.Stream.StartStream("")

	if err != nil {
		panic(err)
	}

Consume Messages from the Stream Handle any io.EOF and other errors that arise first, then unmarshal your bytes into your favorite struct. Below is an example with strings

	go func() {
		for message := range api.Stream.GetMessages() {
			if message.Err != nil {
				panic(message.Err)
			}
                        // Will print something like: 
                        //{"data":{"id":"1356479201000","text":"Look at this cat picture"},"matching_rules":[{"id":12345,"tag":"cat tweets with images"}]}
			fmt.Println(string(message.Data))
		}
	}()

	time.Sleep(time.Second * 30)
	api.Stream.StopStream()

Creating, Deleting, and Getting Rules

Obtain an Access Token using your Twitter Access Key and Secret.

You need an access token to do anything. twitterstream provides an easy way to fetch an access token.

	tok, err := twitterstream.NewTokenGenerator().SetApiKeyAndSecret("key", "secret").RequestBearerToken()

	if err != nil {
		panic(err)
	}

Create a streaming api

Create a twitterstream instance with your access token from above.

	api := twitterstream.NewTwitterStream(tok.AccessToken)

Get Rules

Use the Rules struct to access different Rules endpoints as defined in Twitter's API Reference

    res, err := api.Rules.GetRules()

	if err != nil {
		panic(err)
	}

	if res.Errors != nil && len(res.Errors) > 0 {
		//https://developer.twitter.com/en/support/twitter-api/error-troubleshooting
		panic(fmt.Sprintf("Received an error from twiiter: %v", res.Errors))
	}

	fmt.Println(res.Data)

Add Rules

	res, err := api.Rules.AddRules(`{
		"add": [
				{"value": "cat has:images", "tag": "cat tweets with images"}
			]
		}`, true) // dryRun is set to true

	if err != nil {
		panic(err)
	}

	if res.Errors != nil && len(res.Errors) > 0 {
		//https://developer.twitter.com/en/support/twitter-api/error-troubleshooting
		panic(fmt.Sprintf("Received an error from twiiter: %v", res.Errors))
	}

Delete Rules

// use api.Rules.GetRules to find the ID number for an existing rule
	res, err := api.Rules.AddRules(`{
		"delete": {
				"ids": ["1234567890"]
			}
		}`, true)

	if err != nil {
		panic(err)
	}

	if res.Errors != nil && len(res.Errors) > 0 {
		//https://developer.twitter.com/en/support/twitter-api/error-troubleshooting
		panic(fmt.Sprintf("Received an error from twiiter: %v", res.Errors))
	}

Contributing

Pull requests are always welcome. Please accompany a pull request with tests.