-> breaks plugin installs using git repos or local directories about cordova-fetch

[shankari]

Aha! While creating the MCVE, figured out the issue. I have both cordova and phonegap as dependencies. I was working on upgrading cordova first, and had not yet started upgrading phonegap, which was still at 7.1.1. After upgrading phonegap to 9.0.0+cordova.9.0.0, the error disappeared. I guess 7.1.1 was close enough to 8.0.0 to work, but was too far from 9.0.0

[shankari]

I poked around this a little bit, and the Failed to fetch plugin ... via registry error is from cordova-lib/src/plugman/fetch.js

Code

                .catch(function (error) {
                    var message = 'Failed to fetch plugin ' + plugin_src + ' via registry.' +
                        '\nProbably this is either a connection problem, or plugin spec is incorrect.' +
                        '\nCheck your connection and plugin name/version/URL.' +
                        '\n' + error;
                    return Promise.reject(new CordovaError(message));

So the real error is Could not determine package name from output

[shankari]

Could not determine package name from output is from node_modules//cordova-fetch/index.js

Code

function getTargetPackageSpecFromNpmInstallOutput (npmInstallOutput) {
    const lines = npmInstallOutput.split('\n');
    for (let i = 0; i < lines.length; i++) {
        if (lines[i].startsWith('+ ')) {
            // npm >= 5
            return lines[i].slice(2);
        } else if (lines[i].startsWith('└─') || lines[i].startsWith('`-')) {
            // 3 <= npm <= 4
            return lines[i].slice(4).split(' ')[0];
        }
    }
    throw new CordovaError('Could not determine package name from output:\n' + npmInstallOutput);
}

Essentially, the code expects the npm install output to have a +, but it doesn't. The output is just [email protected] node_modules/phonegap-plugin-contentsync with no +, which is why this fails.

[shankari]

The npm install command is launched using superspawn.spawn.

And now I come to the part that I don't understand. If I use -d, I can see the arguments passed in to npm install

Command line arguments for npm install

No scripts found for hook "before_plugin_add".
Calling plugman.fetch on plugin "https://github.com/phonegap/phonegap-plugin-contentsync.git"
fetch: Installing https://github.com/phonegap/phonegap-plugin-contentsync.git to /Users/kshankar/e-mission/upgrade_platform
Running command: npm install https://github.com/phonegap/phonegap-plugin-contentsync.git --production --save
Command finished with error code 0: npm install,https://github.com/phonegap/phonegap-plugin-contentsync.git,--production,--save
Failed to fetch plugin https://github.com/phonegap/phonegap-plugin-contentsync.git via registry.
Probably this is either a connection problem, or plugin spec is incorrect.
Check your connection and plugin name/version/URL.
Could not determine package name from output:
[email protected] node_modules/phonegap-plugin-contentsync
CordovaError: Failed to fetch plugin https://github.com/phonegap/phonegap-plugin-contentsync.git via registry.
Probably this is either a connection problem, or plugin spec is incorrect.
Check your connection and plugin name/version/URL.
Could not determine package name from output:
[email protected] node_modules/phonegap-plugin-contentsync
    at /Users/kshankar/e-mission/upgrade_platform/node_modules/cordova-lib/src/plugman/fetch.js:146:43
    at processTicksAndRejections (internal/process/task_queues.js:97:5)

If I run the exact same command using superspawn using the node REPL, I get an output with the +

running same command in node REPL

$ node
Welcome to Node.js v14.5.0.
Type ".help" for more information.
> var superspawn = require('cordova-common').superspawn;
undefined
> var dest = "<redacted>"
undefined
> var args = ["install", "https://github.com/phonegap/phonegap-plugin-contentsync.git", "--production", "--save"]
undefined
> superspawn.spawn('npm', args, { cwd: dest }).then(function(output){ console.log(output); });
Promise {
  promiseDispatch: [Function (anonymous)],
  valueOf: [Function (anonymous)],
  inspect: [Function (anonymous)]
}
> (node:59879) [DEP0097] DeprecationWarning: Using a domain property in MakeCallback is deprecated. Use the async_context variant of MakeCallback or the AsyncResource class instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
+ [email protected]
updated 1 package and audited 1707 packages in 11.991s

47 packages are looking for funding
  run `npm fund` for details

found 173 vulnerabilities (45 low, 75 moderate, 53 high)
  run `npm audit fix` to fix them, or `npm audit` for details

But if I add the same kind of output logging in fetch.js, it returns the output without the +

e.g. if I change the code to

New code

function installPackage (target, dest, opts) {
    return isNpmInstalled()
        // Ensure that `npm` installs to `dest` and not any of its ancestors
        .then(_ => fs.ensureDir(path.join(dest, 'node_modules')))

        // Run `npm` to install requested package
        .then(_ => npmArgs(target, opts))
        .then(args => {
            events.emit('verbose', `fetch: Installing ${target} to ${dest}`);
            return superspawn.spawn('npm', args, { cwd: dest });
        })
        .then(function(output){ console.log("Output = "+output); return output; })

        // Resolve path to installed package
        .then(getTargetPackageSpecFromNpmInstallOutput)
        .then(spec => pathToInstalledPackage(spec, dest));
}

I get the following output

non-REPL usage

$ npx cordova plugin -d add https://github.com/phonegap/phonegap-plugin-contentsync.git No scripts found for hook "before_plugin_add". Calling plugman.fetch on plugin "https://github.com/phonegap/phonegap-plugin-contentsync.git" fetch: Installing https://github.com/phonegap/phonegap-plugin-contentsync.git to /Users/kshankar/e-mission/upgrade_platform Running command: npm install https://github.com/phonegap/phonegap-plugin-contentsync.git --production --save Command finished with error code 0: npm install,https://github.com/phonegap/phonegap-plugin-contentsync.git,--production,--save Output = [email protected] node_modules/phonegap-plugin-contentsync Failed to fetch plugin https://github.com/phonegap/phonegap-plugin-contentsync.git via registry. Probably this is either a connection problem, or plugin spec is incorrect. Check your connection and plugin name/version/URL. Could not determine package name from output: [email protected] node_modules/phonegap-plugin-contentsync CordovaError: Failed to fetch plugin https://github.com/phonegap/phonegap-plugin-contentsync.git via registry. Probably this is either a connection problem, or plugin spec is incorrect. Check your connection and plugin name/version/URL. Could not determine package name from output: [email protected] node_modules/phonegap-plugin-contentsync at /Users/kshankar/e-mission/upgrade_platform/node_modules/cordova-lib/src/plugman/fetch.js:146:43 at processTicksAndRejections (internal/process/task_queues.js:97:5)

I can't figure out why superspawn behaves differently in the REPL and in the script. Any thoughts?

[shankari]

Finally, -d also gives us a clue as to why the registry install works (eventually). It uses plugman install instead of npm install

Logs

``` $ npx cordova plugin -d add phonegap-plugin-contentsync No scripts found for hook "before_plugin_add". No version specified for phonegap-plugin-contentsync, retrieving version from package.json Calling plugman.fetch on plugin "phonegap-plugin-contentsync@^1.4.2" Copying plugin "/Users/kshankar/e-mission/upgrade_platform/node_modules/phonegap-plugin-contentsync" => "/Users/kshankar/e-mission/upgrade_platform/plugins/phonegap-plugin-contentsync" Calling plugman.install on plugin "/Users/kshankar/e-mission/upgrade_platform/plugins/phonegap-plugin-contentsync" for platform "android Installing "phonegap-plugin-contentsync" for android Running command: /Users/kshankar/e-mission/upgrade_platform/platforms/android/cordova/version Command finished with error code 0: /Users/kshankar/e-mission/upgrade_platform/platforms/android/cordova/version Finding scripts for "before_plugin_install" hook from plugin phonegap-plugin-contentsync on android platform only. No scripts found for hook "before_plugin_install". Install start for "phonegap-plugin-contentsync" on android. PlatformApi successfully found for platform android Beginning processing of action stack for android project... Action stack processing complete. Install complete for phonegap-plugin-contentsync on android. Finding scripts for "after_plugin_install" hook from plugin phonegap-plugin-contentsync on android platform only. No scripts found for hook "after_plugin_install". Calling plugman.install on plugin "/Users/kshankar/e-mission/upgrade_platform/plugins/phonegap-plugin-contentsync" for platform "ios Installing "phonegap-plugin-contentsync" for ios Running command: /Users/kshankar/e-mission/upgrade_platform/platforms/ios/cordova/version Command finished with error code 0: /Users/kshankar/e-mission/upgrade_platform/platforms/ios/cordova/version Finding scripts for "before_plugin_install" hook from plugin phonegap-plugin-contentsync on ios platform only. No scripts found for hook "before_plugin_install". Install start for "phonegap-plugin-contentsync" on ios. PlatformApi successfully found for platform ios Beginning processing of action stack for ios project... Adding non-custom framework to project... libz.dylib -> {"customFramework":false,"embed":false,"link":true,"weak":false} Non-custom framework added to project. libz.dylib -> {"customFramework":false,"link":true,"weak":false} Action stack processing complete. pods.json found in platforms/ios Podfile found in platforms/ios Install complete for phonegap-plugin-contentsync on ios. Finding scripts for "after_plugin_install" hook from plugin phonegap-plugin-contentsync on ios platform only. No scripts found for hook "after_plugin_install". Adding phonegap-plugin-contentsync to package.json No scripts found for hook "after_plugin_add". ```

[raphinesse]

Thanks for reporting this and for the detailed information and debugging. It's indeed strange that the npm output differs between manual invocations and those performed by cordova-fetch.

Just to be sure that this is still an issue in the current code base, could you please try if you can reproduce this issue when using cordova@nightly?

[shankari]

Ok, so I poked around this some more, and I know the reason behind the difference in superspawn behavior.

After adding log statements to all instances of superspawn, it looks like this is difference is due to the differences in the versions of superspawn included in the project.

$ find node_modules -name \*superspawn.js
node_modules/cordova/node_modules/cordova-common/src/superspawn.js
node_modules/cordova-common/src/superspawn.js
node_modules/cordova-create/node_modules/cordova-common/src/superspawn.js
node_modules/cordova-fetch/node_modules/cordova-common/src/superspawn.js
node_modules/cordova-lib/node_modules/cordova-common/src/superspawn.js
node_modules/phonegap/node_modules/cordova-common/src/superspawn.js
node_modules/phonegap/node_modules/cordova-create/node_modules/cordova-common/src/superspawn.js

While using node directly, we use node_modules/cordova-common/src/superspawn.js, while using plugin add, we use node_modules/cordova-fetch/node_modules/cordova-common/src/superspawn.js. The two versions are different, although I cannot see which of the differences lead to this difference in behavior.

not_working_to_working.diff.gz

[shankari]

so the two versions are different because the versions of cordova-common are different.

$ find node_modules -name package.json | grep "cordova-common.package" | xargs grep version.:
node_modules/cordova/node_modules/cordova-common/package.json:  "version": "3.2.1"
node_modules/cordova-common/package.json:  "version": "4.0.2"
node_modules/cordova-create/node_modules/cordova-common/package.json:  "version": "3.2.1"
node_modules/cordova-fetch/node_modules/cordova-common/package.json:  "version": "3.2.1"
node_modules/cordova-lib/node_modules/cordova-common/package.json:  "version": "3.2.1"
node_modules/phonegap/node_modules/cordova-common/package.json:  "version": "2.1.1"
node_modules/phonegap/node_modules/cordova-create/node_modules/cordova-common/package.json:  "version": "2.2.5"

[shankari]

But if I use the cordova-fetch version superspawn directly in the REPL, it works correctly. Which is consistent with the diffs seeming fairly innocuous. The main difference seems to be running this in cordova.

Logs

> var superspawn = require('./node_modules/cordova-fetch/node_modules/cordova-common/src/superspawn').superspawn;

> superspawn.spawn('npm', args, { cwd: dest }).then(function(output){ console.log(output); return output;})
Promise {
  promiseDispatch: [Function (anonymous)],
  valueOf: [Function (anonymous)],
  inspect: [Function (anonymous)]
}
> + [email protected]
updated 1 package and audited 1707 packages in 19.092s

47 packages are looking for funding
  run `npm fund` for details

found 173 vulnerabilities (45 low, 75 moderate, 53 high)
  run `npm audit fix` to fix them, or `npm audit` for details

I will try with the latest cordova@nightly and if that still doesn't work, I'm going to try to create a MCVE to make it easier for others to debug as well.

[shankari]

@raphinesse Confirmed that this occurs even with cordova@nightly

Logs

$ npx cordova plugin add https://github.com/phonegap/phonegap-plugin-contentsync.git
Warning: using prerelease version 10.0.0-nightly.2020.7.14.d69f1084 ([email protected])
Failed to fetch plugin https://github.com/phonegap/phonegap-plugin-contentsync.git via registry.
Probably this is either a connection problem, or plugin spec is incorrect.
Check your connection and plugin name/version/URL.
CordovaError: CordovaError: Could not determine package name from output:
[email protected] node_modules/phonegap-plugin-contentsync