atlassian / build-stats Goto Github PK

I ran this in atlaskit and only received around 2000 files

$ build-stats bitbucket:atlassian/atlaskit-mk-2 download
✔ Download completed. Total Builds: 33722

$ ls ~/.config/yarn/global/node_modules/build-stats/.data/bitbucket/atlassian/atlaskit-mk-2/builds | wc -l
2130

I'm thinking this could have been an error response from BB, possibly a rate limit? In either case, we should be catching the error, not returning success.

Steps to reproduce

1. build-stats bitbucket:<user>/<repo> download --auth <username>:<app_password>
   Result:

name: 'HTTPError',
  host: 'api.bitbucket.org',
  hostname: 'api.bitbucket.org',
  method: 'GET',
  path: '/2.0/repositories/<user>/<repo>/pipelines/',
  protocol: 'https:',
  url:
   'https://api.bitbucket.org/2.0/repositories/<user>/<repo>/pipelines/',
  statusCode: 404,
  statusMessage: 'Not Found',
  headers:
   { server: 'nginx',
     'cache-control': 'max-age=900',
     'content-type': 'application/json; charset=utf-8',
     'strict-transport-security': 'max-age=31536000; includeSubDomains; preload',
     date: 'Sun, 08 Dec 2019 15:52:34 GMT',
     'x-served-by': 'app-1107',
     'x-static-version': 'abd48467d96e',
     etag: '"d8570ff7a4314593e35e9bc69e81400a"',
     'x-render-time': '0.0135359764099',
     connection: 'close',
     'x-version': 'abd48467d96e',
     'x-frame-options': 'SAMEORIGIN',
     'x-reads-before-write-from': 'default',
     'x-request-count': '1860',
     'x-cache-info': 'caching',
     'content-length': '52' } }

However when I simply use curl https://api.bitbucket.org/2.0/repositories/<user>/<repo>/pipelines/ --user <username>:<app_password> I receive a JSON response of the 1st page of pipelines data.

Let me know if I can provide any other information.

If you ever get in an inconsistent state (i.e from a bug like #15) where you have the latest data, but not data before it, currently there is no way to get the missing data.

A couple of ways to solve this

• We could either refactor how we decide which builds to download (looking at the list
  of files we have currently and starting from the first missing),
• We could add it as a flag (--force, --all) that works the same as currently (downloading backwards), but instead of stopping when we reach one we already know, we continue going back, each time checking if we have the file before doing the request

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Rate-Limited

These updates are currently rate-limited. Click on a checkbox below to force their creation now.

• Update babel monorepo (@babel/core, @babel/preset-env, @babel/preset-typescript)
• Update dependency @types/node to v16.18.96
• Update jest monorepo (@types/jest, babel-jest, jest)
• Update actions/checkout action to v4
• Update actions/setup-node action to v4
• Update borales/actions-yarn action to v5
• Update dependency @types/meow to v6
• Update dependency @types/node to v20
• Update dependency chai to v5
• Update dependency date-fns to v3
• Update dependency inquirer to v9 (inquirer, @types/inquirer)
• Update dependency meow to v13
• Update dependency mkdirp to v3
• Update dependency ora to v8
• Update dependency p-limit to v5
• Update dependency prettier to v3
• Update dependency typescript to v5
• Update jest monorepo to v29 (major) (@types/jest, babel-jest, jest, ts-jest)
• 🔐 Create all rate-limited PRs at once 🔐

Edited/Blocked

These updates have been manually edited so Renovate will no longer make changes. To discard all commits and start over, click on a checkbox.

• Update dependency string-to-stream to v3

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• Update dependency got to v11 [SECURITY]
• Update dependency @types/lodash.pick to v4.4.9
• Update dependency inquirer to v8.2.6 (inquirer, @types/inquirer)
• Update dependency mkdirp to v0.5.6
• Update dependency ts-jest to v27.1.5
• Update dependency tsc-files to v1.1.4
• Update dependency chai to v4.4.1
• Update dependency prettier to v2.8.8
• Update dependency ts-node to v10
• Click on this checkbox to rebase all open PRs at once

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

• Update dependency chalk to v5

Detected dependencies

• Check this box to trigger a request for Renovate to run again on this repository

Hello @ajaymathur ,

As a user I want to have the build stats for custom builds.

introduction
This project is to help during simcard verification.
problem solving
To prevent frauds from picking IDs and registering more simcards without permission.
conclusion

Will automatically search for builds from a specific build number.

This is more used in recovery situations, but means there will always be a way to force a redownload without losing data.

Same as PR #19 , we should do the same for Travis. Should be a pretty easy change.

Will also work well with #20 , the --since flag. Travis will actually be able to take proper advantage of it since we don't need to rely on rough pagination, you can target exact numbers

Suggested from PR #19 , for anyone that wants downloads to be faster.
Should be a super small change

I believe there is a cwd flag, but most people wouldn't use it.

But it'd be useful to have the equivalent of yarn bin, etc for showing the default directory.

Since, now we are downloading builds in ascending order from 1 to now, we are required to download/ wait for tool to download all the builds before we can start checking latest stats.

@ajaymathur 's suggestion, as discussed.
https://en.wikipedia.org/wiki/Standard_deviation