Support Yarn v3 with ember new Command about ember-cli

[tsengia]

That exception being thrown comes from here in ember-cli-preprocessor, which means for some reason, I am missing some sort of template plugin.

I have a feeling this plugin is some sort of dependency that was not included due to yarn's strict peer dependency handling...

[tsengia]

I did a side by side comparison of two projects: one using npm, and one using yarn.
The yarn project I applied the above steps of with the .yarnrc.yml file.

By using some python scripts, I was able to compare the list of packages installed when using npm versus yarn.

I found that the set of packages installed when using yarn is a subset of the packages installed when using npm.

Everything installed when using yarn is also installed by npm.
Some things installed when using npm are not present in yarn.

Here's a list of packages missing (compared to npm) when installing via yarn:

Click to see list of 735 packages

``` @jridgewell/gen-mapping @jridgewell/set-array @jridgewell/sourcemap-codec @jridgewell/trace-mapping @babel/highlight @babel/helper-validator-identifier js-tokens jsesc @babel/helper-environment-visitor @babel/helper-simple-access @babel/helper-function-name @babel/helper-hoist-variables @babel/helper-string-parser to-fast-properties @babel/helper-annotate-as-pure @babel/helper-member-expression-to-functions @babel/helper-optimise-call-expression @babel/helper-skip-transparent-expression-wrappers ansi-styles color-convert color-name has-flag path-root resolve lru-cache yallist fs.realpath inflight once wrappy inherits path-is-absolute ansi-escapes type-fest cli-cursor restore-cursor onetime signal-exit cli-width external-editor chardet iconv-lite safer-buffer tmp figures mute-stream run-async tslib through ms assert-never babel-import-util object-assign path-posix minimist array-equal broccoli-node-api broccoli-output-wrapper fs-merger @types/symlink-or-copy @types/minimatch matcher-collection merge-trees fs-updater can-symlink os-tmpdir clean-up-path @babel/plugin-syntax-typescript ansi-to-html entities get-stream pump end-of-stream is-stream merge-stream npm-run-path path-key p-finally strip-final-newline jsonfile universalify stagehand at-least-node broccoli-kitchen-sink-helpers copy-dereference async-disk-cache istextorbinary binaryextensions editions textextensions username-sync async-promise-queue async jsonify brace-expansion concat-map regenerator-runtime cliui wrap-ansi escalade require-directory y18n yargs-parser @babel/compat-data @babel/helper-validator-option @babel/plugin-bugfix-safari-id-destructuring-collision-in-function-expression @babel/plugin-bugfix-v8-spread-parameters-in-optional-chaining @babel/plugin-transform-optional-chaining @babel/plugin-syntax-async-generators @babel/plugin-syntax-class-properties @babel/plugin-syntax-class-static-block @babel/plugin-syntax-dynamic-import @babel/plugin-syntax-export-namespace-from @babel/plugin-syntax-import-assertions @babel/plugin-syntax-import-attributes @babel/plugin-syntax-import-meta @babel/plugin-syntax-json-strings @babel/plugin-syntax-logical-assignment-operators @babel/plugin-syntax-nullish-coalescing-operator @babel/plugin-syntax-numeric-separator @babel/plugin-syntax-object-rest-spread @babel/plugin-syntax-optional-catch-binding @babel/plugin-syntax-optional-chaining @babel/plugin-syntax-private-property-in-object @babel/plugin-syntax-top-level-await @babel/plugin-syntax-unicode-sets-regex @babel/helper-create-regexp-features-plugin regexpu-core @babel/regjsgen regenerate-unicode-properties regenerate regjsparser unicode-match-property-ecmascript unicode-canonical-property-names-ecmascript unicode-property-aliases-ecmascript unicode-match-property-value-ecmascript @babel/plugin-transform-arrow-functions @babel/plugin-transform-async-generator-functions @babel/helper-remap-async-to-generator @babel/helper-wrap-function @babel/plugin-transform-async-to-generator @babel/plugin-transform-block-scoped-functions @babel/plugin-transform-class-properties @babel/plugin-transform-class-static-block @babel/plugin-transform-classes @babel/plugin-transform-computed-properties @babel/plugin-transform-destructuring @babel/plugin-transform-dotall-regex @babel/plugin-transform-duplicate-keys @babel/plugin-transform-dynamic-import @babel/plugin-transform-exponentiation-operator @babel/helper-builder-binary-assignment-operator-visitor @babel/plugin-transform-export-namespace-from @babel/plugin-transform-for-of @babel/plugin-transform-function-name @babel/plugin-transform-json-strings @babel/plugin-transform-literals @babel/plugin-transform-logical-assignment-operators @babel/plugin-transform-member-expression-literals @babel/plugin-transform-modules-commonjs @babel/plugin-transform-modules-systemjs @babel/plugin-transform-modules-umd @babel/plugin-transform-named-capturing-groups-regex @babel/plugin-transform-new-target @babel/plugin-transform-nullish-coalescing-operator @babel/plugin-transform-numeric-separator @babel/plugin-transform-object-rest-spread @babel/plugin-transform-parameters @babel/plugin-transform-object-super @babel/plugin-transform-optional-catch-binding @babel/plugin-transform-private-methods @babel/plugin-transform-private-property-in-object @babel/plugin-transform-property-literals @babel/plugin-transform-regenerator regenerator-transform @babel/plugin-transform-reserved-words @babel/plugin-transform-shorthand-properties @babel/plugin-transform-spread @babel/plugin-transform-sticky-regex @babel/plugin-transform-template-literals @babel/plugin-transform-typeof-symbol @babel/plugin-transform-unicode-escapes @babel/plugin-transform-unicode-property-regex @babel/plugin-transform-unicode-regex @babel/plugin-transform-unicode-sets-regex @babel/preset-modules @babel/plugin-proposal-unicode-property-regex babel-plugin-polyfill-corejs2 @babel/helper-define-polyfill-provider lodash.debounce @nicolo-ribaudo/semver-v6 babel-plugin-polyfill-corejs3 core-js-compat babel-plugin-polyfill-regenerator find-cache-dir commondir make-dir pkg-dir locate-path p-locate p-limit p-try path-exists loader-utils big.js emojis-list @types/json-schema ajv-keywords line-column isarray isobject sourcemap-codec parse-static-imports string.prototype.matchall call-bind function-bind get-intrinsic define-properties has-property-descriptors object-keys es-abstract array-buffer-byte-length is-array-buffer available-typed-arrays es-set-tostringtag has-tostringtag has-symbols has es-to-primitive is-callable is-date-object is-symbol function.prototype.name functions-have-names get-symbol-description globalthis gopd has-proto internal-slot is-typed-array is-negative-zero is-regex is-shared-array-buffer is-string for-each is-weakref object-inspect object.assign regexp.prototype.flags safe-regex-test string.prototype.trim string.prototype.trimend string.prototype.trimstart typed-array-byte-offset typed-array-length unbox-primitive has-bigints which-boxed-primitive is-bigint is-boolean-object is-number-object which-typed-array side-channel broccoli-node-info icss-utils postcss-modules-extract-imports postcss-modules-local-by-default postcss-modules-scope postcss-modules-values source-map uglify-js wordwrap ajv-formats json-schema-traverse require-from-string uri-js path-root-regex path-parse supports-preserve-symlinks-flag core-js object-hash find-babel-config pkg-up reselect blank-object fast-ordered-set sync-disk-cache fixturify @types/fs-extra @types/node @types/rimraf @types/glob arr-diff array-unique braces arr-flatten extend-shallow is-extendable fill-range is-number kind-of is-buffer repeat-string to-regex-range repeat-element snapdragon-node define-property is-descriptor snapdragon-util snapdragon split-string to-regex is-accessor-descriptor is-data-descriptor assign-symbols extglob expand-brackets posix-character-classes regex-not fragment-cache map-cache nanomatch is-windows object.pick safe-regex ret base cache-base collection-visit map-visit object-visit component-emitter get-value has-value has-values set-value to-object-path union-value arr-union unset-value class-utils static-extend object-copy copy-descriptor mixin-deep for-in pascalcase source-map-resolve atob decode-uri-component resolve-url source-map-url urix use errlop broccoli-caching-writer sri-toolbox lodash.defaultsdeep terser @jridgewell/source-map source-map-support buffer-from mout osenv os-homedir untildify fast-sourcemap-concat memory-streams readable-stream core-util-is string_decoder amdefine sourcemap-validator lodash.foreach find-index lodash.omit lodash.uniq ansi-html has-ansi ansi-regex @types/chai-as-promised @types/chai @types/express @types/body-parser @types/connect @types/express-serve-static-core @types/qs @types/range-parser @types/send @types/mime @types/serve-static @types/http-errors connect finalhandler encodeurl escape-html on-finished ee-first parseurl statuses unpipe utils-merge esm findup-sync detect-file resolve-dir expand-tilde homedir-polyfill parse-passwd global-prefix ini which isexe https resolve-path http-errors depd setprototypeof @cnakazawa/watch anymatch remove-trailing-separator exec-sh nice-try shebang-command shebang-regex strip-eof fb-watchman walker underscore.string sprintf-js util-deprecate accepts negotiator bytes compressible mime-db on-headers safe-buffer vary dot-prop is-obj unique-string crypto-random-string is-typedarray typedarray-to-buffer xdg-basedir mimic-fn is-fullwidth-code-point ora cli-spinners log-symbols wcwidth defaults through2 broccoli-clean-css clean-css-promise array-to-error array-to-sentence clean-css graceful-readlink pinkie-promise pinkie inline-source-map-comment sum-up xtend process-relative-require node-modules-path human-signals array-flatten body-parser content-type destroy qs raw-body type-is content-disposition cookie-signature cookie etag fresh toidentifier merge-descriptors methods path-to-regexp proxy-addr forwarded ipaddr.js range-parser send mime serve-static media-typer yocto-queue callsites clean-stack extract-stack eventemitter3 follow-redirects requires-port bl buffer base64-js ieee754 is-interactive is-unicode-supported argparse lodash.assign lodash._baseassign lodash._basecopy lodash.keys lodash._createassigner lodash._bindcallback lodash._isiterateecall lodash.restparam lodash._getnative lodash.isarguments lodash.isarray lodash._reinterpolate lodash.templatesettings cardinal ansicolors redeyed esprima cli-table colors linkify-it uc.micro mdurl basic-auth abbrev hosted-git-info proc-log validate-npm-package-name lcid invert-kv mem map-age-cleaner p-is-promise mktemp picomatch bser node-int64 makeerror tmpl detect-indent detect-newline git-hooks-list array-union dir-glob merge2 slash is-plain-obj sort-object-keys @xmldom/xmldom backbone underscore bluebird charm consolidate fireworm is-type lodash.flatten lodash._baseflatten lodash.assignin lodash.castarray lodash.clonedeep lodash.find lodash.uniqby mustache node-notifier growly is-wsl is-docker shellwords npmlog are-we-there-yet delegates console-control-strings gauge aproba color-support has-unicode wide-align set-blocking printf socket.io base64id cors engine.io @types/cookie @types/cors engine.io-parser ws socket.io-adapter socket.io-parser @socket.io/component-emitter spawn-args styled_string tap-parser events-to-array minipass body continuable-cache error string-template safe-json-parse faye-websocket websocket-driver http-parser-js websocket-extensions livereload-js ember-cli-test-info @ember-data/canary-features ember-cached-decorator-polyfill ember-cache-primitive-polyfill babel6-plugin-strip-class-callcheck @types/broccoli-plugin rollup-pluginutils estree-walker rollup npm-git-info @types/acorn acorn-dynamic-import date-time time-zone is-reference locate-character pretty-ms parse-ms require-relative caniuse-lite lodash.memoize whatwg-url tr46 webidl-conversions recast ast-types private @types/eslint proper-lockfile retry upath dequal @glimmer/reference @glimmer/global-context @glimmer/interfaces path-type is-extglob language-subtag-registry @handlebars/parser simple-html-tokenizer mdn-data source-map-js dot-case no-case lower-case regexpp fast-diff @humanwhocodes/object-schema @nodelib/fs.scandir @nodelib/fs.stat run-parallel queue-microtask fastq reusify fast-json-stable-stringify punycode esrecurse acorn-jsx flat-cache flatted parent-module prelude-ls type-check @aashutoshrathi/word-wrap deep-is fast-levenshtein globalyzer globrex parse-json error-ex is-arrayish lines-and-columns @types/minimist camelcase-keys camelcase map-obj quick-lru decamelize-keys decamelize hard-rejection minimist-options arrify normalize-package-data validate-npm-package-license spdx-correct spdx-expression-parse spdx-license-ids spdx-exceptions read-pkg-up read-pkg @types/normalize-package-data redent indent-string strip-indent min-indent trim-newlines cssesc nanoid emoji-regex lodash.truncate slice-ansi astral-regex strip-bom @webassemblyjs/helper-numbers @webassemblyjs/floating-point-hex-parser @webassemblyjs/helper-api-error @xtuc/long @webassemblyjs/helper-wasm-bytecode @webassemblyjs/helper-buffer @webassemblyjs/helper-wasm-section @webassemblyjs/wasm-gen @webassemblyjs/ieee754 @webassemblyjs/leb128 @webassemblyjs/utf8 @webassemblyjs/wasm-opt @webassemblyjs/wast-printer @xtuc/ieee754 electron-to-chromium node-releases update-browserslist-db @jridgewell/resolve-uri jest-worker serialize-javascript randombytes ```

[tsengia]

I figured out that ember-cli actually outputs in debug messages the templates as it search for them.

Here is the (abbreviated) output of DEBUG=* yarn start:

  ember-cli:registry add type: minify-css, name: broccoli-csso, extension:null, options:{ applicationName: 'test-yarn-app', app: [EmberApp] } +0ms
  ember-cli:registry add type: template, name: broccoli-emblem-compiler, extension:[ 'embl', 'emblem' ], options:{ applicationName: 'test-yarn-app', app: [EmberApp] } +1ms
  ember-cli:registry add type: template, name: broccoli-ember-hbs-template-compiler, extension:[ 'hbs', 'handlebars' ], options:{ applicationName: 'test-yarn-app', app: [EmberApp] } +0ms     
  ember-cli:merge-trees [EmberCliMergeTrees(TreeMerger (templates))#13] Returning empty tree +536ms
  ember-cli:merge-trees [EmberCliMergeTrees(Styles)#14] Returning single input tree +1ms
  ember-cli:merge-trees [EmberCliMergeTrees(TreeMerger (lint app))#15] Returning empty tree +0ms
  ember-cli:merge-trees [EmberCliMergeTrees(TreeMerger (lint tests))#16] Returning empty tree +1ms
  ember-cli:registry extensions for type template: [ 'template', 'embl', 'emblem', 'hbs', 'handlebars' ] +77ms
  ember-cli:merge-trees [EmberCliMergeTrees(TreeMerge (templates))#17] Returning upstream merged trees +3ms
  ember-cli:merge-trees [EmberCliMergeTrees(TreeMerger (lint templates))#18] Returning empty tree +0ms
  ember-cli:merge-trees [EmberCliMergeTrees(undefined)#19] Returning upstream merged trees +1ms
  ember-cli:merge-trees [EmberCliMergeTrees(TreeMerger (addon-test-support))#20] Returning empty tree +0ms
  ember-cli:merge-trees [EmberCliMergeTrees(TreeMerger (tests))#21] Returning upstream merged trees +0ms
  ember-cli:merge-trees [EmberCliMergeTrees(TreeMerger (vendor))#22] Returning empty tree +1ms
  ember-cli:merge-trees [EmberCliMergeTrees(TreeMerger (addon))#23] Returning empty tree +0ms
  ember-cli:merge-trees [EmberCliMergeTrees(TreeMerger (ExternalTree))#24] Returning upstream merged trees +1ms
  ember-cli:merge-trees [EmberCliMergeTrees(Public)#25] Returning single input tree +0ms
  ember-cli:merge-trees [EmberCliMergeTrees(TreeMerger (app))#26] Returning single input tree +1ms
  ember-cli:merge-trees [EmberCliMergeTrees(Full Application)#27] Returning upstream merged trees +0ms
  ember-cli:merge-trees [EmberCliMergeTrees(Templates)#28] Returning single input tree +3ms
  ember-cli:registry loading template: available plugins [ 'broccoli-emblem-compiler', 'broccoli-ember-hbs-template-compiler' ]; found plugins []; +9ms
  ember-cli:preprocessors plugins found for templates: [] +0ms

[tsengia]

I figured out how to fix this issue, for those of you encountering the same problem, here are the steps I took to resolve:

1. After running ember new --yarn ..., enter your project directory
2. Add @glimmer/env and @babel/core by running: yarn add @glimmer/env @babel/core
3. Create a file named .yarnrc.yml and enter the following into it:

nodeLinker: "pnpm"

packageExtensions:
  "ember-source@*":
    dependencies:
      "@babel/core": "^7.22.9"
  "@glimmer/vm-babel-plugins@*":
    dependencies:
      "@babel/core": "^7.22.9"
  "ember-data@*":
    dependencies:
      "@glimmer/tracking": "1.1.2"
  "ember-cli-typescript@*":
    dependencies:
      "@babel/core": "^7.22.9"
  "ember-compatibility-helpers@*":
    dependencies:
      "@babel/core": "^7.22.9"
  "ember-auto-import@*":
    dependencies:
      "webpack": "^5.88.1"
  "@ember-data/store@*":
    dependencies:
      "ember-source": "^4.12.3"
      "@glimmer/component": "^1.1.2"
  "@ember-data/model@*":
    dependencies:
      "ember-source": "^4.12.3"
      "@glimmer/component": "^1.1.2"
  "eslint-plugin-qunit@*":
    dependencies:
      "eslint": "^8.44.0"

You may want to change the versions listed in the above file to match the already installed versions of the modules.

4. Run yarn install, this will install the packages into a node_modules folder, which will take some time.

5. Run yarn start and you should be able to build as normal

I am closing this issue, because it is a duplicate of #8164

The root cause of the problem is a lack of support for Plug'n'Play without the node_modules directory existing, which is already an issue at #8164