fetchByExample error about pyarango

[tariqdaouda]

Hi @gdg87,

I've just tried it and it seems to work. Could you give more information about the code that produced this error?

[gdg87]

Hi,
this is my function that produce this error

`postsCollection = db["posts"]
commentsCollection = db["comments"]
postlinksCollection = db["postlinks"]
posts = postsCollection.fetchByExample({'PostTypeId' : 2}, batchSize = 100)
for post in posts:
parent = postsCollection.fetchFirstExample({'Id': post['ParentId']})
if parent:
parentUID = parent[0]['OwnerUserId'];
G.add_edge(parent[0]['OwnerUserId'], post['OwnerUserId'], key='question_answer', weight=8)

        commentsParent = commentsCollection.fetchByExample({'PostId': parent[0]['Id']}, batchSize = 20)
        for commentParent in commentsParent:
            G.add_edge(parent[0]['OwnerUserId'], commentParent['UserId'], key='comment', weight=4)
    comments = commentsCollection.fetchByExample({'PostId': post['Id']}, batchSize = 20)
    for comment in comments:
        G.add_edge(post['OwnerUserId'], comment['UserId'], key='comment', weight=4)
    link = postlinksCollection.fetchFirstExample({'PostId': post['Id']})
    if link:
        linked = postsCollection.fetchFirstExample({'Id': link[0]['RelatedPostId']})
        G.add_edge(post['Id'], linked[0]['OwnerUserId'], key='link', weight=2)
return G`

[tariqdaouda]

What happens if you do:

for post in posts:
    print post

[gdg87]

Print the object until it give the error

[tariqdaouda]

I am not able able to reproduce the error. Can you tell me how many examples it prints before dying?

[gdg87]

If the batchSize is 100 it dies at 100 example, but if the batchSize is smaller it prints more examples. The number of examples printed is variable.

[tariqdaouda]

If you at the look test here, it creates 100 users, as asks for them all with a batch size of 1 and tries to append 105 of them. Everything seems ok when I run it. Have you tried to update to the last github version? Can you run the test?

[gdg87]

The test works, but I have this error only when the number of documents in the collection is higher then 20000. Using collection with a limited number of documents I do not have any erorr.

[tariqdaouda]

I have just tried it and it works for 20000 users.
Could you give me the contents of posts.request.status_code, and also posts.rawResults?

[gdg87]

Using posts.request.status_code or posts.rawResults it give me the above error with any batch size. Try to increase the number of users in your test I think that the limit number to reproduce the error depends also on the pc where the script run.

[tariqdaouda]

I'm sorry request was not saved as an attribute. I've changed that now, but rawResults should have worked. I've tried it with 100,000 documents. The issue might not be directly related to pyArango, have you tried to update ArangoDB?

[gdg87]

Ok, I make a test removing the internal part of the for and it works. Maybe the problem is related with multiple query in a for, however I will change my code to avoid this error moving the other query out of the for.

[tariqdaouda]

Ok, cool!