brendanhay / gogol Goto Github PK
View Code? Open in Web Editor NEWA comprehensive Google Services SDK for Haskell.
License: Other
A comprehensive Google Services SDK for Haskell.
License: Other
Given the following code:
{-# LANGUAGE TemplateHaskell #-}
{-# LANGUAGE OverloadedStrings #-}
module Lib
( fetchSpreadsheetValues
) where
import System.Environment
import System.IO (stdout)
import Network.Google(newLogger, newEnvWith, envLogger, LogLevel(Debug), envScopes)
import Network.Google.Auth.ApplicationDefault(fromJSONCredentials)
import Network.Google.Resource.Sheets.Spreadsheets.Get(spreadsheetsGet, sgAccessToken)
import Network.Google.Sheets(spreadsheetsScope)
import Network.HTTP.Client(newManager, defaultManagerSettings)
import Control.Lens ((&), (.~), (<&>), (?~))
import qualified Data.ByteString.Lazy.Char8 as B
import qualified Data.Text as T
fetchSpreadsheetValues :: IO ()
fetchSpreadsheetValues = do
serviceAccountJson <- getEnv "SERVICE_ACCOUNT_JSON"
let credentials = fromJSONCredentials $ B.pack serviceAccountJson
case credentials of
Left errMessage -> putStrLn errMessage
Right creds -> do
apiKey <- getEnv "API_KEY"
spreadSheetId <- getEnv "SPREADSHEET_ID"
let getRequest = spreadsheetsGet $ T.pack spreadSheetId
let authorizedRequest = sgAccessToken .~ (Just $ T.pack apiKey) $ getRequest
manager <- newManager defaultManagerSettings
lgr <- newLogger Debug stdout
env <- (newEnvWith creds lgr manager) <&> (envScopes .~ spreadsheetsScope)
putStrLn apiKey
I'm getting this output
Failed parsing service_account: Error in $: Unable to parse key contents from "private_key", Failed parsing authorized_user: Error in $: key "refresh_token" not present
Any idea what's going on? The JSON string in the SERVICE_ACCOUNT_JSON
environment variable parses fine in a python shell into the equivalent of the file downloaded from the Google IAM console.
Is there an easy way to get the AdWords API included in this?
I think it is a little different to the other Google APIs so maybe it is not possible to use the same approach.
The jclQuote lens should actually be optional. Currently it is a required text, but it should have been Maybe Text
.
The consequence of the current implementation is that it is impossible to process a JSON payload without getting an error about this field. The error is:
Only CSV imports may specify a quote character
Setting jclQuote
to an empty Text has no effect on the error.
Hi,
I'm currently trying to upload a 2MB image to a bucket, but the upload seems to be corrupted. I'm running the example from examples/gogol-examples
.
> stack ghci gogol-examples
ghci> example "my-bucket" "image.jpg"
...
ghci> :! gsutil ls gs://my-bucket
2035740 2016-03-14T20:59:51Z gs://my-bucket/image.jpg
TOTAL: 1 objects, 2035740 bytes (1.94 MiB)
ghci> :! stat -f "%z" image.jpg
2035466
I discovered that this lib uses AltMedia
for the uploadType
query parameter. It resolves to "media"
as a value for the parameter, but this is wrong for multipart uploads and should be "multipart"
(https://cloud.google.com/storage/docs/json_api/v1/objects/insert).
For a quick fix, I changed the Line 65 to:
instance ToText AltMedia where toText = const "multipart"
and the image is now correctly uploaded.
> stack ghci gogol-examples
ghci> example "my-bucket" "image.jpg"
ghci> :! gsutil ls -rl gs://my-bucket/
2035466 2016-03-14T21:10:40Z gs://my-bucket/image.jpg
TOTAL: 1 objects, 2035466 bytes (1.94 MiB)
It would be great if gogol would support Haxl out of the box. I see that it relies in part on servant, and I see that servant-haxl-client is a thing, but I'm not sure how possible it is to use the two together. It seems to me that both libraries have code that overlaps: the part that issues the actual HTTP request.
gogol is fantastically exciting! (But trying to stumble through OAuth is painful... can the core gogol
lib walk us through this a bit more for newbies?)
Here's my current stumbling block. Currently the docs on the Credentials
data type say that its ok to use "service_account" credentials:
Load the Application Default Credentials from a specific file path. The file can be formatted as either a service_account or an authorized_user.
Ok, so I followed Google's instructions for "Application Default Credentials", and got a json file with a private key in it, which says it is in fact of type "service_account". Specifically, it's of the form:
{
"private_key_id": "...",
"private_key": "...",
"client_email": "[email protected]",
"client_id": "....apps.googleusercontent.com",
"type": "service_account"
}
Then for my Credentials
data type I point it at this file with FromFile "blah.json"
. But when trying to execute newEnv
I get this error:
mytest-exe: InvalidFileError "blah.json" "key \"client_secret\" not present"
Hmm, that looks like it's trying to use the other form of credentials, not the "service_account" type...
CC @craigcitro
Hey, thank you for creating this library! I was wondering if you could wrap the People API, i.e. Google Contacts? The Google Discovery Service link is here:
I'm trying to use gogol-datastore
with the Datastore Emulator, for local testing a debugging. I can't find any definitive information on it, but I assume this emulator just ignores OAuth headers, but I'm unable to test, since I can't figure out a way to run requests while ignoring authentication.
What's the right way to run requests against an emulator, auth-wise?
Hi!
I would really appreciate a new hackage release of all libs, would help using gogol with latest GHC and servant.
Thank you!
Can you generate library for google cloud dataproc api?
> /tmp/stackage-build8$ stack unpack gogol-0.1.0
Unpacked gogol-0.1.0 to /tmp/stackage-build8/gogol-0.1.0/
> /tmp/stackage-build8/gogol-0.1.0$ runghc -clear-package-db -global-package-db -package-db=/var/stackage/work/builds/nightly/pkgdb Setup configure --package-db=clear --package-db=global --package-db=/var/stackage/work/builds/nightly/pkgdb --libdir=/var/stackage/work/builds/nightly/lib --bindir=/var/stackage/work/builds/nightly/bin --datadir=/var/stackage/work/builds/nightly/share --libexecdir=/var/stackage/work/builds/nightly/libexec --sysconfdir=/var/stackage/work/builds/nightly/etc --docdir=/var/stackage/work/builds/nightly/doc/gogol-0.1.0 --htmldir=/var/stackage/work/builds/nightly/doc/gogol-0.1.0 --haddockdir=/var/stackage/work/builds/nightly/doc/gogol-0.1.0 --flags=
Configuring gogol-0.1.0...
> /tmp/stackage-build8/gogol-0.1.0$ runghc -clear-package-db -global-package-db -package-db=/var/stackage/work/builds/nightly/pkgdb Setup build
Building gogol-0.1.0...
Preprocessing library gogol-0.1.0...
[ 1 of 13] Compiling Network.Google.Internal.Multipart ( src/Network/Google/Internal/Multipart.hs, dist/build/Network/Google/Internal/Multipart.o )
src/Network/Google/Internal/Multipart.hs:17:1: warning: [-Wunused-imports]
The qualified import of ‘Data.CaseInsensitive’ is redundant
except perhaps to import instances from ‘Data.CaseInsensitive’
To import instances alone, use: import Data.CaseInsensitive()
[ 2 of 13] Compiling Network.Google.Compute.Metadata ( src/Network/Google/Compute/Metadata.hs, dist/build/Network/Google/Compute/Metadata.o )
src/Network/Google/Compute/Metadata.hs:229:11: error:
Not in scope: ‘Client.checkStatus’
Module ‘Network.HTTP.Conduit’ does not export ‘checkStatus’.
Hi
I'm trying to hit this API endpoint:
https://us.gcr.io/v2/<gcp_project_name>/<my_image_name>/tags/list
(the endpoint that the following CLI command hits: gcloud beta container images list-tags us.gcr.io/<gcp_project_name>/<my_image_name>
)
I realize that gogol
likely does not support this API, but I suspect that I could reuse gogol
's auth mechanism to authenticate with this endpoint.
I don't mind hand-crafting a gogol
request for this endpoint, but would really appreciate an advice or assistance as to how to go about it
Thanks
I'm curious about how the generation works, and I suspect so might be other people. Does it use this?: https://developers.google.com/discovery/
This happens when I run the provided example: https://github.com/brendanhay/gogol/tree/develop/examples/src/Example
I have credentials setup by gcloud
in my ~/.config/gcloud/application_default_credentials.json
as per default.
getting this runtime error (when executing the newEnv line):
TokenRefreshError (Status {statusCode = 400, statusMessage = "Bad Request"}) "Failure refreshing token from accounts.google.com/o/oauth2/v2/auth" Nothing
Observation: logger does not log anything, even though I used Debug/Trace mods and looking at the code (https://github.com/brendanhay/gogol/blob/develop/gogol/src/Network/Google/Internal/Auth.hs#L288-L289) it should be logging things.
I've tried printing the offending request and response and here is what i found:
Request {
host = "accounts.google.com"
port = 443
secure = True
requestHeaders = [("Content-Type","application/x-www-form-urlencoded")]
path = "/o/oauth2/v2/auth"
queryString = ""
method = "POST"
proxy = Nothing
rawBody = False
redirectCount = 10
responseTimeout = ResponseTimeoutDefault
requestVersion = HTTP/1.1
}
Response:
<!DOCTYPE html><html lang=en><meta charset=utf-8><meta name=viewport content=\"initial-scale=1, minimum-scale=1, width=device-width\"><title>Error 400 (OAuth2 Error)!!1</title><style>*{margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{color:#222;text-align:unset;margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px;}* > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}pre{white-space:pre-wrap;}ins{color:#777;text-decoration:none}a img{border:0}@media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}#logo{background:url(//www.google.com/images/branding/googlelogo/1x/googlelogo_color_150x54dp.png) no-repeat;margin-left:-5px}@media only screen and (min-resolution:192dpi){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat 0% 0%/100% 100%;-moz-border-image:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) 0}}@media only screen and (-webkit-min-device-pixel-ratio:2){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat;-webkit-background-size:100% 100%}}#logo{display:inline-block;height:54px;width:150px}</style><div id=\"af-error-container\"><a href=//www.google.com/><span id=logo aria-label=Google></span></a><p><b>400.</b> <ins>That\226\128\153s an error.</ins><p><script src='https://ssl.gstatic.com/accounts/o/3918119028-common_lib.js'></script><style>\n #request_info_header {\n cursor: default;\n outline: none;\n padding-left: 14px;\n padding-top: 10px;\n }\n\n #request_info_items {\n line-height: 18px;\n list-style-type: none;\n margin-top: 8px;\n padding-left: 14px;\n }\n\n .param_entry {\n margin-top: 2px;\n }\n\n .goog-zippy-expanded,\n .goog-zippy-collapsed {\n list-style: none;\n padding: 2px 0 1px 15px;\n position: relative;\n }\n\n .goog-zippy-expanded:before {\n content: url(https://ssl.gstatic.com/ui/v1/zippy/arrow_down.png);\n left: 1px;\n position: absolute;\n top: 7px;\n }\n\n .goog-zippy-collapsed:before {\n content: url(https://ssl.gstatic.com/ui/v1/zippy/arrow_right.png);\n left: 3px;\n position: absolute;\n top: 6px;\n }\n </style><p id=\"errorCode\"><b>Error: invalid_request</b></p><p id=\"errorDescription\">Required parameter is missing: response_type</p><p id=\"errorUri\"><a target=\"_blank\" href=\"http://code.google.com/apis/accounts/docs/OAuth2.html\">Learn more</a></p><div id=\"request_info_header\">Request Details<ul id=\"request_info_items\"><li class=\"param_entry\" id=\"param_entry_0\">grant_type=refresh_token</li><li class=\"param_entry\" id=\"param_entry_1\">client_id=764086051850-6qr4p6gpi6hn506pt8ejuq83di341hur.apps.googleusercontent.com</li><li class=\"param_entry\" id=\"param_entry_2\">client_secret=d-FL95Q19q7MQmFpd7hHD0Ty</li><li class=\"param_entry\" id=\"param_entry_3\">refresh_token=1/1Yrj_4sULbMQca3rqX3BO919jyj2JR1LDGXy9M3Y9Bw</li></ul></div><script type=\"text/javascript\">lso.doZippy(\"request_info_header\", \"request_info_items\");</script><p> <ins>That\226\128\153s all we know.</ins></div>
viewing this response in a browser tells the following story:
Error: invalid_request
Required parameter is missing: response_type
Request Details
grant_type=refresh_token
client_id=764086051850-6qr4p6gpi6hn506pt8ejuq83di341hur.apps.googleusercontent.com
client_secret=<CLEANED>
refresh_token=<CLEANED>
I've got a module in my project that's the single interface to all of my usage of google cloud services. Currently, each individual function is rebuilding the env
each time, which seems pretty wasteful. I would rather the user be able to pass in an env to each function.
The problem comes with the scopes needed for each function:
The problem with 1 is if one of my google functions uses multiple google functions internally, the required type signature does not seem to concatenate lists but forces me to specify them individually, duplicates and all. So if I use foo
which requires '[a, b, c]
and bar
which requires '[b, c, d]
, then fooBar
's constraints won't require (HasScope s '[a, b, c, d] ~ True) => Env s -> ...
but rather (HasScope s '[a, b, c] ~ True, HasScope s '[b, c, d]) => Env s -> ...
, which can get really repetitive the more you use. I am not willing to forgo writing type signatures for toplevel functions.
The problem with 2 is envScopes :: Lens' a (Proxy s)
uses a Lens'
which does not allow type-changing, so I cannot do something like let env' = env & envScopes .~ neededScopes
because that changes s
and that lens will not allow it.
I tried to look at the examples for guidance but because they set up the env and set scopes in one shot, they can't really model the scenario of the user passing in an env to something that demands a certain set of scopes. What should I do?
Could you provide some documentation about the usage of this client lib?
One example that fetches data from a public dataset would be sufficient.
Hey there!
I'm taking gogol for a spin. I'm a long time amazonka user and noticed that theres some departers from the amazonka design. In particular, in amazonka, things like BucketName
are a newtype over text which I really like because it helps prevent you from mixing up something that's semantically a BucketName with one of the most ubiquitous types. I noticed in gogol-storage its just a Text. I'm curious if this was a conscious decision and if so, why?
Thanks for creating gogol! 😄
My goal is to download an object from Storage and write its contents to a file. But I'm stumped by the use of ResumableSource
s.
Calling download
works just fine. It finds the object, yielding a Stream
(aka: ResumableSource (ResourceT IO) ByteString
).
But I don't know how to work with this Stream
. I've tried to use Data.Conduit.($$+-)
to put the contents in a file, to no avail. Nothing at all happens.
Here's my code:
fetchFile key = do
lgr ← newLogger Debug stdout
env ← newEnv <&> (envLogger .~ lgr) . (envScopes .~ storageReadWriteScope)
runResourceT . runGoogle env $ do
-- Fetch file into Stream
stream ← download (objectsGet bucketName key)
-- Attempt to write stream into file. Nothing happens.
return $ stream $$+- sinkFile "./my_file_name"
Could you guide me in the right direction, please?
Thanks!
My understanding is that MPL-2.0
would be the correct choice.
Many Network.Google.Datastore.Value
types -- and types contained within these -- have sensible Data.Aeson.ToJSON/FromJSON
instances, because their purpose is to map to JSON types.
This goes for types like ValueNullValue
(toJSON = const Data.Aeson.Null
), ArrayValue
(toJSON = ^. avValues
) and Entity
:
instance ToJSON Entity where
toJSON e = Object $
fmap toJSON . view epAddtional .
fromMaybe (entityProperties Map.empty) $
(e ^. eProperties)
However, these types also need to be serialized, and sent over the API, hence this library's ToJSON
and FromJSON
instances, which are used for this internal purpose.
I would like to ask if it's possible to contain these instances internally in the library, such that I can create an elegant wrapper around gogol-datastore
which has the right Aeson instances for the various types, thus allowing users to convert seamlessly between eg. a value & vStringValue
and a Data.Aeson.String
.
I have created JSON conversion functions for all the useful Network.Google.Datastore.Value
types, as well as eg. ArrayValue
and ValueNullValue
. However, these implementations really belong as ToJSON
/FromJSON
instances of Network.Google.Datastore.Value
, ArrayValue
and ValueNullValue
, such that toJSON
and fromJSON
knows which lenses to use for conversion, for example vIntegerValue
or vDoubleValue
to convert to/from a JSON Number.
Looks like you may not want the argument at all here:
usersDraftsList
:: Text
-> UsersDraftsList
usersDraftsList pUdlUserId_ =
UsersDraftsList
{ _udlUserId = "me"
, _udlPageToken = Nothing
, _udlMaxResults = 100
}
I'm currently experimenting with the gogol-datastore library, and experienced an bug. Here my example code:
{-# LANGUAGE OverloadedStrings #-}
module Main where
import Network.Google as Google
import Network.Google.Datastore
import Control.Lens
import System.IO (stdout)
import Control.Monad.Trans (liftIO)
main = do
let papegoPartition = partitionId & piProjectId ?~ "my-project-id" & piNamespaceId ?~ "dev"
logger <- Google.newLogger Google.Debug stdout
env <- Google.newEnv <&> (Google.envLogger .~ logger) . Google.allow datastoreScope
let proofId = key
& kPartitionId ?~ papegoPartition
& kPath .~ [pathElement & peKind ?~ "Proof" & peName ?~ "proof_abc"]
proofEntity = entity
& eKey ?~ proofId
putStrLn "Entity:"
print proofEntity
let upsertProof = commitRequest
& crMutations .~ [mutation & mUpsert ?~ proofEntity]
& crMode ?~ NonTransactional
print upsertProof
runResourceT . runGoogle env $ do
resp <- Google.send (projectsCommit upsertProof "my-project-id")
liftIO $ print resp
putStrLn "Done!"
Results into a 404 with this debug output:
[Client Request] {
host = datastore.googleapis.com:443
secure = True
method = POST
timeout = Just 70000000
redirects = 10
path = /v1beta3/projects/my-project-idcommit
query = ?pp=true&alt=json
headers = authorization: Bearer XXX; accept: application/json; content-type: application/json
body = {"mutations":[{"upsert":{"key":{"partitionId":{"namespaceId":"dev","projectId":"my-project-id"},"path":[{"kind":"Proof","name":"proof_abc"}]}}}],"mode":"NON_TRANSACTIONAL"}
}
[Client Response] {
status = 404 Not Found
headers = date: Tue, 26 Jul 2016 13:29:12 GMT; content-type: text/html; charset=UTF-8; server: ESF; content-length: 1616; x-xss-protection: 1; mode=block; x-frame-options: SAMEORIGIN; x-content-type-options: nosniff; alternate-protocol: 443:quic; alt-svc: quic=":443"; ma=2592000; v="36,35,34,33,32,31,30,29,28,27,26,25"
}
The Problem is the path: /v1beta3/projects/my-project-idcommit
which should be /v1beta3/projects/my-project-id:commit
(see REST-API Reference)
Changing
resp <- Google.send (projectsCommit upsertProof "my-project-id")
to
resp <- Google.send (projectsCommit upsertProof "my-project-id:")
solves it as quick workaround.
I've just made my first subscription pull request to PubSub for an existing subscription.
I noticed that my Network.Google.PubSub.Types:PullRequest
field prMaxMessages
which is typed as a Maybe Int32
apparently is required to be a non optional value.
Here's the ServiceError that got returned:
ServiceError (ServiceError' {_serviceId = ServiceId "pubsub:v1", _serviceStatus = Status {statusCode = 400, statusMessage = "Bad Request"}, _serviceHeaders = [("Vary","Origin"),("Vary","X-Origin"),("Vary","Referer"),("Content-Type","application/json; charset=UTF-8"),("Content-Encoding","gzip"),("Date","Sat, 11 Mar 2017 19:06:21 GMT"),("Server","ESF"),("Cache-Control","private"),("X-XSS-Protection","1; mode=block"),("X-Frame-Options","SAMEORIGIN"),("X-Content-Type-Options","nosniff"),("Alt-Svc","quic=\":443\"; ma=2592000; v=\"36,35,34\""),("Transfer-Encoding","chunked")], _serviceBody = Just "{\n \"error\": {\n \"code\": 400,\n \"message\": \"A required argument is missing in the request: (argument=\\\"max_messages\\\").\",\n \"status\": \"INVALID_ARGUMENT\"\n }\n}\n"})
If there's anything I can do to help fix this I'd be happy to submit a PR. Thank you so much for this incredible work @brendanhay. I've said it before but this and amazonka
are a tour de force in Haskell.
The current implementation of the parser/AST reification is just a hack port from the amazonka
generator.
Next step is to rewrite and conform to the JSON Schema specification.
According to the official docs from Google, the URL format should be:
POST https://pubsub.googleapis.com/v1/{topic=projects/*/topics/*}:publish
However, the publish URL being generated is missing the colon between the topic and "publish": /v1/projects/my-project-here/topics/topic-testerpublish
It works fine though if I add a colon myself, but I doubt that's the intended way to use the library:
{-# LANGUAGE OverloadedStrings #-}
import Control.Lens ((&), (.~), (<&>), (?~))
import Data.Text (Text)
import Network.Google
import Network.Google.PubSub
import System.IO (stdout)
import qualified Data.Text as Text
example :: IO PublishResponse
example = do
lgr <- newLogger Debug stdout
env <- newEnv <&> (envLogger .~ lgr) . (envScopes .~ pubSubScope)
runResourceT . runGoogle env $ do
send $ projectsTopicsPublish
(publishRequest & prMessages .~ [pubsubMessage & pmData ?~ "Hi!"])
"projects/my-project-here/topics/topic-tester:"
I understand this is very early stage work and there is a lot to do, but I am really like a hen before an egg while looking at http://hackage.haskell.org/package/gogol-gmail-0.1.0. I have no idea how to start using it. In particular, I don't see how this works with OAuth2 authentication flow...
Thanks for the great work in making this available in Haskell!
Currently the ServerConfig
looks like this:
data ServerConfig = ServerConfig'
{ _scValidNodeVersions :: !(Maybe [Text])
, _scDefaultImageFamily :: !(Maybe Text)
, _scValidImageFamilies :: !(Maybe [Text])
, _scDefaultClusterVersion :: !(Maybe Text)
} deriving (Eq,Show,Data,Typeable,Generic)
Hitting gcloud container get-server-config --log-http
returns a result like this:
{
"defaultClusterVersion": …,
"validNodeVersions": […],
"defaultImageType": …,
"validImageTypes": [...],
"validMasterVersions": [...]
}
Can we get _scValidMasterVersions
in the datatype? Thanks!
Would be nice to have an ability to use Google Places API with this library.
https://developers.google.com/places/
Thanks for this wonderful package. Any plans to support FCM?
Unless I'm missing something, the InstalledApplication
credentials can only handle the initial authorization, and does not allow to store refresh token or use refresh token to authenticate.
The "Installed application" flow should be something like
So currently missing building blocks would be
data Credentials s = FromRefreshToken RefereshToken
| ...
retrieveStoredAuth :: MonadIO m => Store s -> m (Auth s)
And installed app usage would be like
initialiseCreds :: OAuthClient -> Logger -> Manager -> IO (Credentials s)
initialiseCreds client logger manager = do
refreshToken <- readSavedToken
case refreshToken of
Just t -> pure (FromRefreshToken t)
Nothing -> do
T.putStrLn (formUrl ...)
code <- OAuthCode <$> T.getLine
pure (FromClient client code)
main = do
logger <- newLogger
manager <- newManager
creds <- initializeCreds client logger manager
env <- newEnvWith ....
runApp
auth <- retreiveStoreAuth (env^.envStore)
saveRefreshToken auth
On top of that we can build something more like a google python api does - let users store this auth information in a json file before program exists, and try to read it same way as ServiceAccount
or AuthorizedUser
does.
{ "client_id" : "..."
, "client_secret" : "..."
, "access_token" : ".."
, "token_expiration" : "2016-12-01T..."
, "refresh_token" : "..."
}
For all use cases google recommends storing refresh tokens and reusing them, so this may be useful not only for "Installed application" case.
What do you think?
I'm trying to use gogol with GCE. I'm getting this error when connecting:
RetrievalError (StatusCodeException (Status {statusCode = 404, statusMessage = "Not Found"})
... and later on ...
The requested URL <code>/instance/service-accounts/default/token</code> was not found on this server.
...
("X-Request-URL","GET http://metadata.google.internal:80instance/service-accounts/default/token")
It looks like the correct path is this:
$ curl -H "Metadata-Flavor: Google" \
http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token
<< a valid token >>
But it looks like gogol
is missing part of the path. Or I'm doing something dumb...
src/Network/Google/Types.hs:318:34:
Not in scope: ‘foldMap’
Perhaps you meant one of these:
‘CL.foldMap’ (imported from Data.Conduit.List),
‘ifoldMap’ (imported from Control.Lens),
‘CL.foldMapM’ (imported from Data.Conduit.List)
made bound revisions on Hackage
gogol
doesn't offer much context for API errors right now, since it uses checkStatus
to throw ServiceError
on non-success response statuses. (And checkStatus
doesn't have access to the response body.)
Is this intentional, or a planned change in the future? It's hard to work with these APIs without any error context beyond the raw status code.
These libraries (amazonka
etc) are amazing btw 👍😀🔧
Hey!
Love the library. I was wondering if the Analytics Measurement Protocol is something you could add here. It's unauthenticated so I'm not sure if it falls into the same category as the other APIs you generate the bindings for, but I at least wanted to ask. :)
Google's OAuth docs show pictures of the back-and-forth protocol to get user consent, get a response code, and exchange it for a token that can actually be used for API calls.
(I've been doing this for a while with handa-gdata but without knowing how it works.)
Now I'm staring at the haddock Indices for gogol
, gogol-core
, and gogol-oauth2
and I see very little mention of "token"s. Where are the modules/types to:
OAuthToken
)?First of all, thank all contributors for this great lib!
I am learning Haskell, so bear with me. I encountered TokenRefreshError (Status {statusCode = 400, statusMessage = "Bad Request"}) "Failure refreshing token from accounts.google.com/o/oauth2/v2/auth" Nothing
, when I try bigquery project list.
code looks like
example :: IO BigQuery.ProjectList
example = do
lgr <- Google.newLogger Google.Debug stdout
m <- liftIO (newManager tlsManagerSettings) :: IO Manager
c <- Google.getApplicationDefault m
env <- Google.newEnvWith c lgr m <&>
(Google.envLogger .~ lgr)
. (Google.envScopes .~ BigQuery.bigQueryScope)
runResourceT . Google.runGoogle env $ Google.send BigQuery.projectsList
main :: IO ()
main = do
projects <- example
print projects
i am using authorized_user
type credential generated by gcloud auth application-default login
, i verified the credential is loaded and it is FromUser
, but the refresh endpoint got invoked is
[Client Request] {
host = accounts.google.com:443
secure = True
method = POST
timeout = ResponseTimeoutDefault
redirects = 10
path = /o/oauth2/v2/auth
query =
headers = content-type: application/x-www-form-urlencoded
but the correct endpoint should be https://www.googleapis.com/oauth2/v4/token
accord to this.
i found there is tokenRequest
and accountsRequest
functions, it seems tokenRequest
should be the right one for authorized user token, and exchange
is invoking tokenRequest
. So I cannot find where is the problem. (I tried on two MacOs machines)
Currently only tags are supported. It used to be that tags and labels are coupled somehow. It's no longer the case: https://cloud.google.com/compute/docs/labeling-resources#labels_tags
@brendanhay Could you publish new changes?
I really need this change, #66.
Thanks!
So I'm trying to run all the MonadGoogle
functions I've created, but I get the following error when I try to run everything using runResourceT . runGoogle (storeEnv :: Env '[AuthDatastore])
/Users/rune/IdeaProjects/paychan-datastore/src/Test.hs:40:62: error:
• Couldn't match type ‘Network.Google.Auth.Scope.HasScope'
'[AuthDatastore] (Scopes BeginTransactionResponse)’
with ‘'True’
arising from a use of ‘testDB’
• In the second argument of ‘($)’, namely ‘testDB pid tstData’
In a stmt of a 'do' block:
runResourceT . runGoogle (env :: Env '[AuthDatastore])
$ testDB pid tstData
In the expression:
do { putStrLn $ "Using project: " ++ cs pid;
env <- defaultAppDatastoreEnv;
tstData <- genTestData numPayments;
runResourceT . runGoogle (env :: Env '[AuthDatastore])
$ testDB pid tstData }
Relevant snippet (error appears at testDB pid tstData
in runPaymentTest
- line 40 in this file):
type AuthDatastore = "https://www.googleapis.com/auth/datastore"
runPaymentTest :: ProjectId -> Word -> IO Int
runPaymentTest pid numPayments = do
putStrLn $ "Using project: " ++ cs pid
storeEnv <- defaultAppDatastoreEnv
tstData <- genTestData numPayments
-- Run
runResourceT . runGoogle (storeEnv :: Env '[AuthDatastore]) $
-- ERROR: "Couldn't match type
-- HasScope' '[AuthDatastore] (Scopes BeginTransactionResponse)’ with ‘'True’"
testDB pid tstData
testDB :: ( MonadCatch m
, MonadGoogle '[AuthDatastore] m
, HasScope '[AuthDatastore] BeginTransactionResponse
, HasScope '[AuthDatastore] LookupResponse
, HasScope '[AuthDatastore] RollbackResponse
, HasScope '[AuthDatastore] CommitResponse )
=> ProjectId -> Pay.ChannelPairResult -> m Int
testDB pid Pay.ChannelPairResult{..} = do
let sampleRecvChan = Pay.recvChan resInitPair
sampleKey = Pay.getSenderPubKey sampleRecvChan
paymentList = reverse $ init resPayList
DB.insertChan pid sampleRecvChan
-- Safe lookup + update/rollback
res <- M.forM paymentList (doPayment pid sampleKey)
return $ length res
defaultAppDatastoreEnv :: IO (Env '[AuthDatastore])
defaultAppDatastoreEnv = do
manager <- HTTP.newManager HTTP.tlsManagerSettings
logger <- Google.newLogger Google.Error stderr
Google.newEnv <&>
(envLogger .~ logger) .
(envScopes .~ datastoreScope) .
(envManager .~ manager)
Relevant snippet from gogol-datastore
:
Citing from http://hydra.cryp.to/build/1732331/nixlog/1/raw:
Building gogol-core-0.0.1...
Preprocessing library gogol-core-0.0.1...
src/Network/Google/Data/Time.hs:37:18:
Could not find module ‘Servant.Common.Text’
Use -v to see a list of the files searched for.
I suppose that module has been removed from servant-0.6.x?
When doing a clean stack build:
gogol-datastore-0.0.1: install
gogol-dataflow-0.0.1: install
Progress: 102/158
-- While building package gogol-compute-0.0.1 using:
/home/gambogi/.stack/setup-exe-cache/setup-Simple-Cabal-1.22.4.0-x86_64-linux-ghc-7.10.2 --builddir=.stack-work/dist/x86_64-linux/Cabal-1.22.4.0/ build lib:gogol-compute --ghc-options -ddump-hi -ddump-to-file
Process exited with code: ExitFailure 1
Logs have been written to: /home/gambogi/projects/gogol/.stack-work/logs/gogol-compute-0.0.1.log
Configuring gogol-compute-0.0.1...
Preprocessing library gogol-compute-0.0.1...
[ 1 of 194] Compiling Network.Google.Compute.Types.Sum ( gen/Network/Google/Compute/Types/Sum.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/Network/Google/Compute/Types/Sum.o )
[ 2 of 194] Compiling Network.Google.Compute.Types.Product ( gen/Network/Google/Compute/Types/Product.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.4.0/build/Network/Google/Compute/Types/Product.o )
ghc: out of memory (requested 1048576 bytes)
Not sure if this is primarily a gogol, stack, cabal, or ghc issue, but I figured I'd start here.
I have this function, which I use to make Datastore requests:
sendReq' :: ( DatastoreM m
, HasScope '[AuthDatastore] a
, GoogleRequest a)
=> (ProjectId -> a)
-> m (Rs a)
sendReq' mkReq = do
pid <- getPid
liftGoogle $ Google.send (mkReq pid)
The DatastoreM
monad just has a few things in a config, like the ProjectId
. I use it like this:
runQueryReq :: DatastoreM m
=> Maybe TxId
-> RunQueryRequest
-> m RunQueryResponse
runQueryReq txM req =
sendReq' (projectsRunQuery txReq)
where
txReq = maybe req (`atomically` req) txM
but I get the error below for the above function, and I can't figure out why. It happens for all projects, not just projectsRunQuery.
It's solved by adding HasScope '[AuthDatastore] <Project>
to the constraint list for the function that needs to consume from sendReq'
(eg. runQueryReq
), but then I also need to add it to the next function, and so on, until I'm out in a class instance definition, and I need to use UndecidableInstances
in order to enable it here.
Can you tell me why this happens, and if there's something I'm doing wrong, or if it's GHC acting up?
/Users/rune/IdeaProjects/promissory-note-app/paychan-datastore/src/DB/Request/Query.hs:43:5: error:
• Couldn't match type ‘(("https://www.googleapis.com/auth/datastore"
Data.Type.Equality.== "https://www.googleapis.com/auth/cloud-platform")
Data.Type.Bool.|| ("https://www.googleapis.com/auth/datastore"
Data.Type.Equality.== "https://www.googleapis.com/auth/datastore"))
Data.Type.Bool.|| Network.Google.Auth.Scope.HasScope'
'[]
'["https://www.googleapis.com/auth/cloud-platform",
"https://www.googleapis.com/auth/datastore"]’
with ‘'True’
arising from a use of ‘sendReq'’
• In the expression: sendReq' (projectsRunQuery txReq)
In an equation for ‘runQueryReq’:
runQueryReq txM reqT
= sendReq' (projectsRunQuery txReq)
where
txReq = maybe req (`atomically` req) txM
req = unTagged reqT
Would it be possible to add apikey auth? For example, Translate API doesn't need token exchange, but works by just including a secret key in the url (https://cloud.google.com/translate/v2/getting_started?csw=1).
Note: I tried from devel branch head with a service account, and got Forbidden, so I assume Translate API only accepts the api key or OAuth (but the latter is not needed, since there is no user data here).
Slightly related: is it possible to pass quotaUser and/or userIp parameter for that API? See https://support.google.com/cloud/answer/6158858?hl=en&ref_topic=6262490 .
In Stackage Nightly:
Preprocessing library gogol-affiliates-0.2.0...
[ 1 of 14] Compiling Network.Google.Affiliates.Types.Sum ( gen/Network/Google/Affiliates/Types/Sum.hs, dist/build/Network/Google/Affiliates/Types/Sum.o )
[ 2 of 14] Compiling Network.Google.Affiliates.Types.Product ( gen/Network/Google/Affiliates/Types/Product.hs, dist/build/Network/Google/Affiliates/Types/Product.o )
gen/Network/Google/Affiliates/Types/Product.hs:380:8: error:
Ambiguous occurrence ‘Link’
It could refer to either ‘Network.Google.Prelude.Link’,
imported from ‘Network.Google.Prelude’ at gen/Network/Google/Affiliates/Types/Product.hs:21:1-39
(and originally defined in ‘servant-0.10:Servant.Utils.Links’)
or ‘Network.Google.Affiliates.Types.Product.Link’,
defined at gen/Network/Google/Affiliates/Types/Product.hs:310:1
gen/Network/Google/Affiliates/Types/Product.hs:407:22: error:
Ambiguous occurrence ‘Link’
It could refer to either ‘Network.Google.Prelude.Link’,
imported from ‘Network.Google.Prelude’ at gen/Network/Google/Affiliates/Types/Product.hs:21:1-39
(and originally defined in ‘servant-0.10:Servant.Utils.Links’)
or ‘Network.Google.Affiliates.Types.Product.Link’,
defined at gen/Network/Google/Affiliates/Types/Product.hs:310:1
gen/Network/Google/Affiliates/Types/Product.hs:412:26: error:
Ambiguous occurrence ‘Link’
It could refer to either ‘Network.Google.Prelude.Link’,
imported from ‘Network.Google.Prelude’ at gen/Network/Google/Affiliates/Types/Product.hs:21:1-39
(and originally defined in ‘servant-0.10:Servant.Utils.Links’)
or ‘Network.Google.Affiliates.Types.Product.Link’,
defined at gen/Network/Google/Affiliates/Types/Product.hs:310:1
gen/Network/Google/Affiliates/Types/Product.hs:418:28: error:
Ambiguous occurrence ‘Link’
It could refer to either ‘Network.Google.Prelude.Link’,
imported from ‘Network.Google.Prelude’ at gen/Network/Google/Affiliates/Types/Product.hs:21:1-39
(and originally defined in ‘servant-0.10:Servant.Utils.Links’)
or ‘Network.Google.Affiliates.Types.Product.Link’,
defined at gen/Network/Google/Affiliates/Types/Product.hs:310:1
:
Preprocessing library gogol-classroom-0.2.0...
[ 1 of 46] Compiling Network.Google.Classroom.Types.Sum ( gen/Network/Google/Classroom/Types/Sum.hs, dist/build/Network/Google/Classroom/Types/Sum.o )
[ 2 of 46] Compiling Network.Google.Classroom.Types.Product ( gen/Network/Google/Classroom/Types/Product.hs, dist/build/Network/Google/Classroom/Types/Product.o )
gen/Network/Google/Classroom/Types/Product.hs:641:8: error:
Ambiguous occurrence ‘Link’
It could refer to either ‘Network.Google.Prelude.Link’,
imported from ‘Network.Google.Prelude’ at gen/Network/Google/Classroom/Types/Product.hs:21:1-39
(and originally defined in ‘servant-0.10:Servant.Utils.Links’)
or ‘Network.Google.Classroom.Types.Product.Link’,
defined at gen/Network/Google/Classroom/Types/Product.hs:625:1
gen/Network/Google/Classroom/Types/Product.hs:650:24: error:
Ambiguous occurrence ‘Link’
It could refer to either ‘Network.Google.Prelude.Link’,
imported from ‘Network.Google.Prelude’ at gen/Network/Google/Classroom/Types/Product.hs:21:1-39
(and originally defined in ‘servant-0.10:Servant.Utils.Links’)
or ‘Network.Google.Classroom.Types.Product.Link’,
defined at gen/Network/Google/Classroom/Types/Product.hs:625:1
gen/Network/Google/Classroom/Types/Product.hs:657:15: error:
Ambiguous occurrence ‘Link’
It could refer to either ‘Network.Google.Prelude.Link’,
imported from ‘Network.Google.Prelude’ at gen/Network/Google/Classroom/Types/Product.hs:21:1-39
(and originally defined in ‘servant-0.10:Servant.Utils.Links’)
or ‘Network.Google.Classroom.Types.Product.Link’,
defined at gen/Network/Google/Classroom/Types/Product.hs:625:1
gen/Network/Google/Classroom/Types/Product.hs:661:17: error:
Ambiguous occurrence ‘Link’
It could refer to either ‘Network.Google.Prelude.Link’,
imported from ‘Network.Google.Prelude’ at gen/Network/Google/Classroom/Types/Product.hs:21:1-39
(and originally defined in ‘servant-0.10:Servant.Utils.Links’)
or ‘Network.Google.Classroom.Types.Product.Link’,
defined at gen/Network/Google/Classroom/Types/Product.hs:625:1
gen/Network/Google/Classroom/Types/Product.hs:664:19: error:
Ambiguous occurrence ‘Link’
It could refer to either ‘Network.Google.Prelude.Link’,
imported from ‘Network.Google.Prelude’ at gen/Network/Google/Classroom/Types/Product.hs:21:1-39
(and originally defined in ‘servant-0.10:Servant.Utils.Links’)
or ‘Network.Google.Classroom.Types.Product.Link’,
defined at gen/Network/Google/Classroom/Types/Product.hs:625:1
:
:
First of all thanks for your excellent work. I noticed this problem while I was trying to use gmail-api. mraw
should be base64Url encoded ByteString instead of a Word8. Google description here: https://developers.google.com/discovery/v1/type-format . I think other gogol-* libraries affected as well by this.
Hi there. Thanks for this awesome library!
I'm trying out gogol-compute and I'm getting a somewhat weird error when I try to run the example below. I'm trying to get a list of instances for a particular project.
From what I can tell from the debug output I successfully connect to the google compute api, get a bearer token and make a request but then when the debug output would normally print the body (with I assume, the instances json in there) I get the following error:
Main.hs: base64: input: invalid encoding at offset: 1
I tried setting the logger to Info and Error with no luck. I also tried ignoring the result of the api call
but the same error happens.
{-# LANGUAGE OverloadedStrings #-}
module Main where
import Control.Lens ((&), (.~), (<&>), (?~))
import qualified Data.Aeson as JSON
import qualified Data.ByteString.Lazy as BS
import Data.Maybe (fromJust)
import Data.Text (Text)
import Network.Google
import Network.Google.Auth
import Network.Google.Auth.ApplicationDefault
import Network.Google.Auth.ServiceAccount
import Network.Google.Compute
import System.IO (stdout)
import qualified Data.Text as Text
main = do
lgr <- newLogger Debug stdout
bs <- BS.readFile "service_account.json"
manager <- newManager tlsManagerSettings
let creds = fromJSONCredentials bs
case creds of
Left s -> error s
Right c -> do
env <- (newEnvWith c lgr manager) <&> (envScopes .~ computeReadOnlyScope)
r <- runResourceT . runGoogle env $ do
send $ (instancesList "testing" "us-east1-c")
print r
Hi
I successfully used endpoints like Network.Google.Resource.Container.Projects.Zones.Clusters.List
for instance.
But i cannot seem to figure out where the corresponding *Response definition is for this call:
https://hackage.haskell.org/package/gogol-compute-0.2.0/docs/Network-Google-Resource-Compute-Networks-List.html
Or perhaps it does not exist in the json definition and someone is supposed to add it? I wouldn't mind extending the json spec if needed. Is there some guide that explains how to do this?
Thanks
Hi there
There's a final version of the Datastore API out, v1: https://cloud.google.com/datastore/docs/reference/rest/#collection-v1projects
Will you update gogol-datastore to support this version?
Thank you.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.