Coder Social home page Coder Social logo

Comments (14)

milo-hyben avatar milo-hyben commented on August 15, 2024 1

Try to limit number of threads default is 10, I am using 2 threads and it seems to work, anything above 2 is throwing the Exception as on the first comment.
Example of working call:

"duka EURUSD -s 2017-01-01 -e 2017-12-31 -t 2"

from duka.

smorrel1 avatar smorrel1 commented on August 15, 2024

+1 on this. Any help appreciated please :)

I wonder if Dukascopy started requesting a logon recently which broke this? The behaviour when going incognito supports this theory, i.e. when pressing 'Download' incognito mode for the first time you're asked to log on, but not on subsequent downloads. See this note on incognito.
res = await loop.run_in_executor(None, lambda: requests.get(url, stream=True)) in fetch.py line 24 doesn't send cookies. Here are the docs, but I've not found a way to send them yet.

However sometimes this code has worked for 1 day at a time, so I'm not sure....

from duka.

cjlv91 avatar cjlv91 commented on August 15, 2024

Getting the same errors, anyone have potential solutions?

Anyone with sophisticated enough knowledge maybe can incorporate logins into their system? I.e. maybe bypassing the errors using accountlogin. I just registered an account (not trading account) solely to download the data, and it works wonderfully downloading directly from their website.

from duka.

smorrel1 avatar smorrel1 commented on August 15, 2024

What about logging in manually then including the session cookies with the requests.get()?

from duka.

cjlv91 avatar cjlv91 commented on August 15, 2024

Well it seems that the problem might lie at Dukascopy's own servers, I cannot download data for more than 1 day back on their website using proper login manually. For instance I try download Brent 15min data for 4months. Doesn't load, just stuck at "50%" loading for hours now. This seem to occur at the same time the python give those errors of no data fetched.

Might be something with their servers?

Is there a limit on how much data you can download from their website using proper login manually?

from duka.

smorrel1 avatar smorrel1 commented on August 15, 2024

I could download data for 2003 in some pairs, and two weeks at a time from a month back. But I got asked to log in at apparently random intervals. Which means that the server is fine, but logging in is the problem, provided the downloads are no longer than say 10 days each. Possibly they introduced throttling as well as logon to thwart bulk downloads.

If anyone knows how to get cookies from Chrome’s SQL database then I can amend the code to send these and do a pull request.

from duka.

nick2012 avatar nick2012 commented on August 15, 2024

@smorrel1 can you please implement it for firefox cookies? just add an argument for the cookiefile path. I've tried to do it myself but I'm not that advanced, and I could not make it to work. But I guess this is the way to do it: https://stackoverflow.com/questions/49502254/how-to-import-firefox-cookies-to-python-requests

from duka.

smorrel1 avatar smorrel1 commented on August 15, 2024

Hi, after some investigation I might have missed diagnose the problem... you could try using JFOREX to download historic data quite easily instead, it’s what worked best for me.

from duka.

nick2012 avatar nick2012 commented on August 15, 2024

@smorrel1 Can you please guide me on how to use JFOREX? I need stock data feed.

from duka.

tijo45 avatar tijo45 commented on August 15, 2024

@milo-hyben thanks! that worked for me.

from duka.

silviojaeger avatar silviojaeger commented on August 15, 2024

@nick2012 JFOREX workes fine to download historical data. Just open a demo account on dukascopy, download and install JFOREX and sign in with your demo account.
In the menubar click on "View" --> "Historical Data Manager".
On the popup window you can make your settings and download everything, it's pretty straight forward.
I downloaded the complet Dukascopy dataset in a few hours.

from duka.

hn2 avatar hn2 commented on August 15, 2024

Does it still work for anyone? I get Request failed ... even with 1 thread

from duka.

alanpang1990301 avatar alanpang1990301 commented on August 15, 2024

not working

from duka.

ipray4 avatar ipray4 commented on August 15, 2024

Try to limit number of threads default is 10, I am using 2 threads and it seems to work, anything above 2 is throwing the Exception as on the first comment.
Example of working call:

"duka EURUSD -s 2017-01-01 -e 2017-12-31 -t 2"

wow it really works... Thanks a lot

from duka.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.