Breakage: app crash - 2021-01-19 about wikiloop-doublecheck

[xinbenlv]

After debugging, it was due to

Jan 18 18:00:03 wikiloop-doublecheck-prod app[web] info MongoError: you are over your space quota, using 2053 MB of 2048 MB
Jan 18 18:00:03 wikiloop-doublecheck-prod app[web] info     at Connection.<anonymous> (/app/node_modules/mongoose/node_modules/mongodb/lib/core/connection/pool.js:460:61)
Jan 18 18:00:03 wikiloop-doublecheck-prod app[web] info     at Connection.emit (events.js:311:20)
Jan 18 18:00:03 wikiloop-doublecheck-prod app[web] info     at Connection.EventEmitter.emit (domain.js:482:12)
Jan 18 18:00:03 wikiloop-doublecheck-prod app[web] info     at processMessage (/app/node_modules/mongoose/node_modules/mongodb/lib/core/connection/connection.js:368:10)
Jan 18 18:00:03 wikiloop-doublecheck-prod app[web] info     at TLSSocket.<anonymous> (/app/node_modules/mongoose/node_modules/mongodb/lib/core/connection/connection.js:537:15)
Jan 18 18:00:03 wikiloop-doublecheck-prod app[web] info     at TLSSocket.emit (events.js:311:20)
Jan 18 18:00:03 wikiloop-doublecheck-prod app[web] info     at TLSSocket.EventEmitter.emit (domain.js:482:12)
Jan 18 18:00:03 wikiloop-doublecheck-prod app[web] info     at addChunk (_stream_readable.js:294:12)
Jan 18 18:00:03 wikiloop-doublecheck-prod app[web] info     at readableAddChunk (_stream_readable.js:275:11)
Jan 18 18:00:03 wikiloop-doublecheck-prod app[web] info     at TLSSocket.Readable.push (_stream_readable.js:209:10)
Jan 18 18:00:03 wikiloop-doublecheck-prod app[web] info     at TLSWrap.onStreamRead (internal/stream_base_commons.js:186:23) {
Jan 18 18:00:03 wikiloop-doublecheck-prod app[web] info   ok: 0,
Jan 18 18:00:03 wikiloop-doublecheck-prod app[web] info   errmsg: 'you are over your space quota, using 2053 MB of 2048 MB',
Jan 18 18:00:03 wikiloop-doublecheck-prod app[web] info   code: 8000,
Jan 18 18:00:03 wikiloop-doublecheck-prod app[web] info   codeName: 'AtlasError',
Jan 18 18:00:03 wikiloop-doublecheck-prod app[web] info   name: 'MongoError',
Jan 18 18:00:03 wikiloop-doublecheck-prod app[web] info   [Symbol(mongoErrorContextSymbol)]: {}
Jan 18 18:00:03 wikiloop-doublecheck-prod app[web] info }
Jan 18 18:00:03 wikiloop-doublecheck-prod app[web] info Promise {
Jan 18 18:00:03 wikiloop-doublecheck-prod app[web] info   <rejected> MongoError: you are over your space quota, using 2053 MB of 2048 MB
Jan 18 18:00:03 wikiloop-doublecheck-prod app[web] info       at Connection.<anonymous> (/app/node_modules/mongoose/node_modules/mongodb/lib/core/connection/pool.js:460:61)
Jan 18 18:00:03 wikiloop-doublecheck-prod app[web] info       at Connection.emit (events.js:311:20)
Jan 18 18:00:03 wikiloop-doublecheck-prod app[web] info       at Connection.EventEmitter.emit (domain.js:482:12)
Jan 18 18:00:03 wikiloop-doublecheck-prod app[web] info       at processMessage (/app/node_modules/mongoose/node_modules/mongodb/lib/core/connection/connection.js:368:10)
Jan 18 18:00:03 wikiloop-doublecheck-prod app[web] info       at TLSSocket.<anonymous> (/app/node_modules/mongoose/node_modules/mongodb/lib/core/connection/connection.js:537:15)
Jan 18 18:00:03 wikiloop-doublecheck-prod app[web] info       at TLSSocket.emit (events.js:311:20)
Jan 18 18:00:03 wikiloop-doublecheck-prod app[web] info       at TLSSocket.EventEmitter.emit (domain.js:482:12)
Jan 18 18:00:03 wikiloop-doublecheck-prod app[web] info       at addChunk (_stream_readable.js:294:12)
Jan 18 18:00:03 wikiloop-doublecheck-prod app[web] info       at readableAddChunk (_stream_readable.js:275:11)
Jan 18 18:00:03 wikiloop-doublecheck-prod app[web] info       at TLSSocket.Readable.push (_stream_readable.js:209:10)
Jan 18 18:00:03 wikiloop-doublecheck-prod app[web] info       at TLSWrap.onStreamRead (internal/stream_base_commons.js:186:23) {
Jan 18 18:00:03 wikiloop-doublecheck-prod app[web] info     ok: 0,
Jan 18 18:00:03 wikiloop-doublecheck-prod app[web] info     errmsg: 'you are over your space quota, using 2053 MB of 2048 MB',
Jan 18 18:00:03 wikiloop-doublecheck-prod app[web] info     code: 8000,
Jan 18 18:00:03 wikiloop-doublecheck-prod app[web] info     codeName: 'AtlasError',
Jan 18 18:00:03 wikiloop-doublecheck-prod app[web] info     name: 'MongoError',
Jan 18 18:00:03 wikiloop-doublecheck-prod app[web] info     [Symbol(mongoErrorContextSymbol)]: {}
Jan 18 18:00:03 wikiloop-doublecheck-prod app[web] info   }
Jan 18 18:00:03 wikiloop-doublecheck-prod app[web] info }

[xinbenlv]

Mitigate: bump the MongoDB quota from 2GB to 5GB

Further resolution pending:

1. shutdown US 2020 topic feed
2. auto-remove feed revisions after expiration.
3. down size MongoDB quota size