Coder Social home page Coder Social logo

zce / velite Goto Github PK

View Code? Open in Web Editor NEW
302.0 4.0 17.0 9.38 MB

Turns Markdown / MDX, YAML, JSON, or others into app's data layer with Zod schema.

Home Page: http://velite.js.org

License: MIT License

TypeScript 99.39% JavaScript 0.61%
content contentlayer datalayer esbuild headless-cms markdown mdx typescript zod

velite's Introduction

Turns Markdown / MDX, YAML, JSON, or other files into app's data layer with type-safe schema.

Build Status License NPM Version Node Version Code Style
NPM Downloads Install Size Dependencies Status

What is Velite?

Velite is a tool for building type-safe data layer, turn Markdown / MDX, YAML, JSON, or other files into app's data layer with Zod schema.

Velite Workflow

Naming Origin

"Velite" comes from the English word "elite".

"Velite" itself is the code name for Napoleon's elite army.

Key Features

  • Easy to use: Move your contents into content folder, define collections schema, run velite, then use the output data in your application.
  • Type-safe: Contents schema validation by Zod, and generate type inference for TypeScript.
  • Framework Agnostic: JSON & Entry & DTS output, out of the box support for any JavaScript framework or library.
  • Light-weight: Choose more native APIs instead of bloated NPM modules, less runtime dependencies, so it is fast and efficiently.
  • Still powerful: Built-in Markdown / MDX, YAML, JSON support, relative files & images processing, schema validation, etc.
  • Configurable: Both input and output directories can be customized, and support for custom loaders, hooks, etc.
  • Extensible: Support any file types by custom loaders, Custom field validation and transform by custom schema, and any output formats by hooks.

Check out our detailed Why Velite to learn more about what makes Velite special. ✨

Try Velite Online

You can try Velite directly in your browser on StackBlitz:

Note

You may need a real-world project to start Velite quickly. I have forked the shadcn-ui/taxonomy project as an example, you can try it out. https://github.com/zce/taxonomy

Roadmap

The following are the features I want to achieve or are under development:

  • Full documentation
  • More framework or build tool integration examples.
  • More built-in schemas
  • Unit & E2E tests?
  • Scoffolding tool
  • Incremental build
  • Turborepo?
  • Next.js plugin package? It's currently a snippet.

See the open issues for a list of proposed features (and known issues).

Contributing

  1. Fork it on GitHub!
  2. Clone the fork to your own machine.
  3. Checkout your feature branch: git checkout -b my-awesome-feature
  4. Commit your changes to your own branch: git commit -am 'Add some feature'
  5. Push your work back up to your fork: git push -u origin my-awesome-feature
  6. Submit a Pull Request so that we can review your changes.

Note

Be sure to merge the latest from "upstream" before making a pull request!

License

MIT © zce

velite's People

Contributors

afadlallah avatar git-create-devben avatar jrmurr avatar jzxhuang avatar marcocondrache avatar mateusfg7 avatar renovate[bot] avatar witch-factory avatar writeonlycode avatar zce avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

velite's Issues

Flattened path is not correctly handled for folders

Issue with File Paths

Suppose there is a directory structure as follows:

  • contents/product1/architecture/index.md
  • contents/product1/architecture.md

Both of these paths generate the same path: product1/architecture, despite being distinct files. To differentiate them, the flatten path should be renamed to product1/architecture/index. This change would explicitly highlight it as an index file, aligning with the structure at the base root.

The issue is related to s.path() schema.

Adding an option that allows users to select their preferred format could present an intriguing solution as well.

How to pass in props to the built mdx code?

Hi, I'm coming from contentlayer and just wondering how you can pass props into the component inside the mdx file.

Previously I can just do https://github.com/Adriel-M/adriel.dev/pull/163/files#diff-06f49e3158547ee7b6e5ad074bcb5d474ecd3b8d7b1418c4ab1d89973e22c5edL19 and pass in the props in the mdx file via https://github.com/Adriel-M/adriel.dev/pull/163/files#diff-5cb1a6f54644fd51947374ebebad4a490a7d9cbc2d3d7a288619268911d3367cL31

Looking at the code, I see it's using mdx-js and I don't quite see a way to do this in the documentation: https://mdxjs.com/docs/using-mdx/#props. Am I missing something or is this just not possible with the mdx bundler of this project?

How to reference MDX images from the public folder without copying them to the public directory during the build.

I am trying to use Contentlayer to transform MDX, similar to taxonomy(shadcn-ui example). However, Contentlayer is no longer maintained, so I am attempting to use Velite instead.

According to the Velite example, it suggests placing images and MDX files in the same directory and referencing them using relative paths. I am unable to reference images using absolute paths from the public directory and do not copy images to the public folder, as I could with taxonomy(shadcn-ui example). Is there any solution to this issue?

If I place images in the public folder, like public/images/blog/blog-post-3.jpg and use the path /images/blog/blog-post-3.jpg, velite throws the warning:

[VELITE] issues:
content/blog/deploying-next-apps.mdx
 warning ENOENT: no such file or directory, open 'D:/images/blog/blog-post-3.jpg'  image

⚠ 1 warning

I have to place the images in the same directory as the MDX file and reference them using ./blog-post-3.jpg, and velite will copy them to the public directory during the build.

Imrpove DX on collection import

It would be beneficial to provide general functions getCollection and getXYZbySlug to eliminate the need for defining custom functions, as they often have similar structures for each collection.

The getCollection function should take the collection name as a parameter and return the entire collection.

The getXYZbySlug function should also take a parameter representing the slug. It should return the matching entry or execute specific code if the action fails, such as calling notFound().

[webpack.cache.PackFileCacheStrategy/webpack.FileSystemInfo] Parsing of ... for build dependencies failed at 'import(configUrl.href)'.

This error is logged every time I navigate to a new page. It might not be logged if I am on a page using content? I am guessing that it is related to watching a page where velite is not used?

[webpack.cache.PackFileCacheStrategy/webpack.FileSystemInfo] Parsing of /home/spark/projects/nourish-run/nourish.run/node_modules/velite/dist/velite-hIhfoR9Z.js for build dependencies failed at 'import(configUrl.href)'.

image

repo

Path option for `s.image()`

Hi zce. I was reading the source code of velite to get it as a replacement for contentlayer. I found that the s.image() is cool. But when I started to integrate it with Next/Image for better image optimization and lazy loading. I found it’s quite hard. I just put all my images in the public directory. So I use an absolute path start with /.

So I just wonder if the s.image() can have a option for pointing the image src to the right place.

s.image add option to not copy the file to a folder

Hey

I would like to request a feature so that when we use s.image we can configure it to not copy the file to any folder.

For example...

I have this frontmatter

cover: /media/images/blog/blog-post-cover.jpg

s.image would read that file in ./public/media/images/blog/blog-post-cover.jpg, meaning it is already in that folder, no need to copy it, and return the image metadata it already does.

Maybe cover would need the whole path (./public/media/images/blog/blog-post-cover.jpg) so that s.image checked right there and not try to figure out the root folder.

Thanks in advance!

[feature request] handle contents failing schema validation, or autofixing

Hi I'm new to velite, and in my use case, I plan to assign a random ID to each markdown file in frontmatter, and prefer the generation is automatic: The file is created without ID and then the ID is written to file after saving.

I wanted to make use of prepare velite config to autofix this but looks like the data available in this hook are all contents that pass the schema validation and those who fail are dropped.

So I'm asking if we could add a new hook or some mechanism to allow fixing validation errors on the fly in velite (watch mode maybe). Here's some thoughts

  • this hook should have access to the files that did not pass the validation and respetively the reason(s)
  • this hook should have access to the files that passed the validation because they might be required in the fix (my use case is an example, ID should be unique)
  • we might need a physical path schema to ease the file writing

If this request is acceptable, I can also help if I find some spare time

Handling relationships

Are there any examples on how relationships between Collections are supposed to be handled? The configs prepare hook briefly mentions them but doesn't give any more details.
It's definitely possible to add them in there but this sounds like it would get cumbersome and be prone to errors.

Ideally I would like to define them directly in the schema, similar to Astro's content references to easily ensure data integrity.

v0.1.0 TODOs

  • Swatinem/rollup-plugin-dts#168
  • Better provision of raw data for schemas such as markdowns
  • Improve the speed of cold build
  • full documentation
  • styfle/cancel-workflow-action
  • #8
  • Reduce duplicate markdown parsing in excerpt
  • output each entry to a separate file to improve fast refresh speed
  • init command

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

This repository currently has no open or pending branches.

Detected dependencies

github-actions
.github/workflows/docs.yml
  • actions/checkout v4
  • pnpm/action-setup v4
  • actions/setup-node v4
  • actions/configure-pages v5
  • actions/upload-pages-artifact v3
  • actions/deploy-pages v4
.github/workflows/main.yml
  • actions/checkout v4
  • pnpm/action-setup v4
  • actions/setup-node v4
.github/workflows/publish.yml
  • actions/checkout v4
  • pnpm/action-setup v4
  • actions/setup-node v4
npm
docs/package.json
package.json
  • @mdx-js/mdx 3.0.1
  • esbuild 0.21.3
  • sharp 0.33.4
  • terser 5.31.0
  • @rollup/plugin-commonjs ^25.0.8
  • @rollup/plugin-json ^6.1.0
  • @rollup/plugin-node-resolve ^15.2.3
  • @types/hast ^3.0.4
  • @types/mdast ^4.0.4
  • @types/micromatch ^4.0.7
  • @types/node ^20.12.12
  • @zce/prettier-config ^0.4.0
  • chokidar ^3.6.0
  • fast-glob ^3.3.2
  • hast-util-raw ^9.0.3
  • hast-util-to-string ^3.0.0
  • lint-staged ^15.2.4
  • mdast-util-from-markdown ^2.0.0
  • mdast-util-to-hast ^13.1.0
  • mdast-util-toc ^7.0.1
  • micromatch ^4.0.7
  • prettier ^3.2.5
  • rehype-raw ^7.0.0
  • rehype-stringify ^10.0.0
  • remark-gfm ^4.0.0
  • remark-parse ^11.0.0
  • remark-rehype ^11.1.0
  • rollup ^4.18.0
  • rollup-plugin-dts ^6.1.1
  • rollup-plugin-esbuild ^6.1.1
  • simple-git-hooks ^2.11.1
  • tsx ^4.11.0
  • typescript ^5.4.5
  • unified ^11.0.4
  • unist-util-visit ^5.0.0
  • vfile ^6.0.1
  • vfile-reporter ^8.1.1
  • yaml ^2.4.2
  • node ^18.17.0 || >=20.3.0
  • pnpm 9.1.2

  • Check this box to trigger a request for Renovate to run again on this repository

type inference doesn't work when using defineCollection

When using defineCollection, the resulting output type is any[] in .velite/index.d.ts

This works:

import { defineConfig, s } from 'velite'

// `s` is extended from Zod with some custom schemas,
// you can also import re-exported `z` from `velite` if you don't need these extension schemas.

export default defineConfig({
  collections: {
    posts: {
      name: 'Post', // collection type name
      pattern: 'posts/**/*.md', // content files glob pattern
      schema: s
        .object({
          title: s.string().max(99), // Zod primitive type
          slug: s.slug('posts'), // validate format, unique in posts collection
          date: s.isodate(), // input Date-like string, output ISO Date string.
          cover: s.image().optional(), // input image relpath, output image object with blurImage.
          video: s.file().optional(), // input file relpath, output file public path.
          metadata: s.metadata(), // extract markdown reading-time, word-count, etc.
          excerpt: s.excerpt(), // excerpt of markdown content
          content: s.markdown() // transform markdown to html
        })
        // more additional fields (computed fields)
        .transform(data => ({ ...data, permalink: `/blog/${data.slug}` }))
    },
    others: {
      // other collection schema options
    }
  }
})

.vellite/index.d.ts:
image

However this doesn't work:

import { s } from 'velite'

const posts = defineCollection({
  name: "Post",
  content: "posts/**/*.md",
  schema: s.object({
    slug: s.slug('posts'),
    date: s.isodate(),
    cover: s.image(),
    video: s.file().optional(),
    metadata: s.metadata(),
    excerpt: s.excerpt(),
    content: s.markdown()
  })
})

export default defineConfig( { collections: { posts } })

image

type of Post is any[]

Auto-generate a unique id for every content item

In brief, every content item generated by Velite should have an id key auto-generate.
For example:

.velite/posts.json

[
  {
    "_id": "9b1deb4d-3b7d-4bad-9bdd-2b0d7b3dcb6d", 
    "title": "...",
    "description": "...",
    "type": "...",
    "date": "2024-04-06T19:00:00.000Z"
  }
]

Unable to support advanced features with code highlighting

TL;DR

At present, some common advanced features of code highlighting plug-ins cannot work properly, such as highlighted lines.

Explains

Because the most of code-highlighting plug-ins implement these advanced features by code blocks meta, like this:

```js {1,3-4}
// highlighted-line
// 
console.log('foo') // highlighted-line
console.log('bar') // highlighted-line
```

But in Velite's markdown process, rehype-raw is used to process the rehype tree before loading the custom rehypePlugins.

However, rehype-raw will lead to the loss of metadata, refer to:

Include image metadata from content source

Hi there! Can image metadata be included when extracted from content instead of front matter?

I need the image width and height in the MDX component for the Next.js Image component. Currently, only src and alt properties are available.

Also, is it possible to import images directly for use with a custom component like this:

import image from './example.png'

# Title

See the image below:

<Figure>
  <FigureImage src={image} alt="" />
  <FigureCaption>
    Caption
  </FigureCaption>
</Figure>

The requested resource isn't a valid image

⨯ The requested resource isn't a valid image for /static/journey-b48700.png received text/html; charset=utf-8

I believe this may also be happening in the style guide post and the contact page in the Next.js example?

image

Can see how I am adding it to my repo here

I can display the main content - SyntaxError: Unexpected token '<'

Hey guys! I moved from contentlayer and everything seems to be working well except for one thing, I'm unable to display the main content (body/content) of my MDX file.
I've used the MDXContent compoment from the demo

Do you have any thoughts on how to tackle this issue?

 ○ Compiling /blog/[...slug] ...
 ✓ Compiled /blog/[...slug] in 503ms
 ⨯ src/components/content/mdx-content.tsx (11:12) @ useMDXComponent
 ⨯ SyntaxError: Unexpected token '<'
    at new Function (<anonymous>)
    at useMDXComponent (/saas-template/.next/server/chunks/[root of the server]__f067be._.js:2176:16)
    at MDXContent (/saas-template/.next/server/chunks/[root of the server]__f067be._.js:2182:23)
    at stringify (<anonymous>)
   9 |
  10 | const useMDXComponent = (code: string) => {
> 11 |   const fn = new Function(code)
     |            ^
  12 |   return fn({ ...runtime }).default
  13 | }
  14 |

next.config.js

import('./env.mjs')

/** @type {import('next').NextConfig} */
module.exports = {
  reactStrictMode: true,
  images: {
    remotePatterns: [
      {
        protocol: 'https',
        hostname: 'avatars.githubusercontent.com',
      },
      {
        protocol: 'https',
        hostname: 'lh3.googleusercontent.com',
      },
      {
        protocol: 'https',
        hostname: 'images.unsplash.com',
      },
      {
        protocol: 'https',
        hostname: 'source.unsplash.com',
      },
      {
        protocol: 'https',
        hostname: 'i.pravatar.cc',
      },
      {
        protocol: 'https',
        hostname: 'res.cloudinary.com',
      },
    ],
  },
  transpilePackages: ['html-to-text'],
  experimental: {
    serverComponentsExternalPackages: ['@prisma/client'],
  },
  webpack: (config) => {
    config.plugins.push(new VeliteWebpackPlugin())
    return config
  },
}

class VeliteWebpackPlugin {
  static started = false
  apply(/** @type {import('webpack').Compiler} */ compiler) {
    // executed three times in nextjs
    // twice for the server (nodejs / edge runtime) and once for the client
    compiler.hooks.beforeCompile.tapPromise('VeliteWebpackPlugin', async () => {
      if (VeliteWebpackPlugin.started) return
      VeliteWebpackPlugin.started = true
      const dev = compiler.options.mode === 'development'
      const { build } = await import('velite')
      await build({ watch: dev, clean: !dev })
    })
  }
}

mdx-content.tsx

import * as runtime from 'react/jsx-runtime'

import Image from 'next/image'

interface MdxProps {
  code: string
  components?: Record<string, React.ComponentType>
}

const useMDXComponent = (code: string) => {
  const fn = new Function(code)
  return fn({ ...runtime }).default
}

export function MDXContent({ code, components }: MdxProps) {
  const Component = useMDXComponent(code)
  return <Component components={{ Image, ...components }} />
}

velite.config.js

import rehypeAutolinkHeadings from 'rehype-autolink-headings'
import rehypePrettyCode from 'rehype-pretty-code'
import rehypeSlug from 'rehype-slug'
import { defineCollection, defineConfig, s } from 'velite'

const allAuthors = defineCollection({
  name: 'Author',
  pattern: 'authors/**/*.mdx',
  schema: s
    .object({
      title: s.string(),
      description: s.string().optional(),
      avatar: s.string().optional(),
      twitter: s.string().optional(),
    })
    .transform((data) => ({
      ...data,
      slug: data.title
        .toLowerCase()
        .replace(/ /g, '-')
        .replace(/[^\w-]+/g, ''),
    })),
})

const allDocs = defineCollection({
  name: 'Doc',
  pattern: 'docs/**/*.mdx',
  schema: s.object({
    slug: s.path(),
    title: s.string(),
    description: s.string(),
    body: s.markdown(),
    published: s.boolean().default(true),
  }),
})

const allGuides = defineCollection({
  name: 'Guide',
  pattern: 'guides/**/*.mdx',
  schema: s.object({
    slug: s.path(),
    title: s.string(),
    description: s.string(),
    body: s.markdown(),
    date: s.isodate(),
    published: s.boolean().default(true),
    featured: s.boolean().default(false),
  }),
})

const allPages = defineCollection({
  name: 'Page',
  pattern: 'pages/**/*.mdx',
  schema: s.object({
    slug: s.path(),
    title: s.string(),
    description: s.string(),
    body: s.markdown(),
  }),
})

const allPosts = defineCollection({
  name: 'Post',
  pattern: 'blog/**/*.mdx',
  schema: s
    .object({
      slug: s.path(),
      title: s.string(),
      description: s.string(),
      body: s.markdown(),
      date: s.isodate(),
      published: s.boolean().default(true),
      image: s.string().optional(),
      authors: s.array(s.string()),
      tags: s.array(s.string()).optional(),
      category: s.array(s.string()).optional(),
      hasToc: s.boolean().default(false),
      toc: s.toc(),
    })
    .transform((data) => ({
      ...data,
      slug: data.slug.replace('blog/', ''),
    })),
})

export default defineConfig({
  root: 'content',
  output: {
    data: '.velite',
    assets: 'public/static',
    base: '/static/',
    name: '[name]-[hash:6].[ext]',
    clean: true,
  },
  collections: {
    allAuthors,
    allDocs,
    allGuides,
    allPages,
    allPosts,
  },
  mdx: {
    rehypePlugins: [
      rehypeSlug,
      [rehypePrettyCode, { theme: 'github-dark' }],
      [
        rehypeAutolinkHeadings,
        {
          behavior: 'wrap',
          properties: {
            className: ['subheading-anchor'],
            ariaLabel: 'Link to section',
          },
        },
      ],
    ],
    remarkPlugins: [],
  },
})

allPosts.jon

[
  {
    "slug": "10-things-you-most-likely-didnt-know-about-health",
    "title": "10 Things You Most Likely Didn't Know About Health.",
    "description": "Culpa laboris aliquip ea consectetur mollit ea ipsum sint qui culpa laboris dolor adipisicing proident. Et officia consequat do nulla tempor cupidatat elit.",
    "body": "<p>Cupidatat voluptate deserunt non ea exercitation sit consequat ullamco ex nostrud elit magna. Nulla id proident labore pariatur pariatur ex ut ad enim et labore. Est do minim eiusmod culpa. Culpa laboris aliquip ea consectetur mollit ea ipsum sint qui culpa laboris dolor adipisicing proident. Et officia consequat do nulla tempor cupidatat elit. Consequat proident magna dolor labore et esse aute dolor sit ea.</p>\n<p>Sint amet deserunt commodo aute consectetur Lorem qui aliqua tempor nulla. Velit non ea qui aliquip. Qui laborum labore excepteur duis velit velit enim enim veniam. Pariatur laboris commodo est laboris. In incididunt pariatur aliquip ut elit irure magna anim sunt. Anim labore ut sit magna ex proident dolor anim cupidatat adipisicing.</p>\n<h2>5 Things</h2>\n<ul>\n<li>Minim est mollit commodo ad sit quis.</li>\n<li>Tempor id magna eu veniam sint et mollit magna laboris reprehenderit Lorem proident.</li>\n<li>Laborum id sint ex magna nulla est labore non.</li>\n<li>Pariatur qui qui ex duis nulla et aute magna incididunt cupidatat commodo.</li>\n<li>Sint amet deserunt commodo aute consectetur.</li>\n</ul>\n<p>Aliquip veniam aliquip nisi non amet pariatur quis. Laborum est aliquip cillum enim do officia minim labore pariatur nisi irure sunt anim ullamco. Mollit ullamco sint qui enim non Lorem aliquip nulla sint. Aute laborum tempor adipisicing officia magna fugiat sint cupidatat. Enim Lorem officia anim cillum ea tempor dolore voluptate.</p>\n<p>Consequat occaecat excepteur esse eu et ex officia adipisicing laborum qui duis tempor. Amet do pariatur elit aute fugiat eu ad. Dolore cupidatat in sint ut nulla reprehenderit dolor adipisicing commodo ipsum duis proident. Lorem anim veniam id aliqua.</p>\n<p>Excepteur aliqua minim Lorem officia ullamco pariatur. Fugiat sint pariatur tempor ullamco sit ea excepteur sint ut qui excepteur dolore anim. Eiusmod id dolor sit fugiat eu eiusmod tempor proident. Officia enim dolore excepteur proident incididunt et sint. Non laboris veniam nisi adipisicing magna.</p>\n<p>Nulla ut irure aliqua ex aliquip nisi non amet excepteur ipsum laboris voluptate elit. Duis cupidatat mollit ea ipsum tempor consectetur. Aliquip proident magna Lorem amet esse laborum cillum. Culpa aute laborum velit velit in do. Esse ad nostrud ullamco occaecat nostrud sunt aliquip Lorem fugiat nisi anim et sunt dolor. Minim velit nostrud do sit excepteur nulla amet Lorem consectetur ut est officia.</p>\n<p>Nisi ad aliquip minim quis cupidatat eu minim voluptate tempor consequat irure eu. Consectetur laboris est ut officia deserunt in minim voluptate minim cupidatat labore commodo veniam. Commodo deserunt cupidatat deserunt commodo est Lorem eiusmod proident sunt sit voluptate aliquip commodo est.</p>\n<p>Consectetur aliqua eu veniam consequat eu adipisicing id ullamco incididunt. Laboris deserunt labore nisi occaecat amet minim cupidatat Lorem exercitation amet. Proident fugiat id deserunt do consectetur quis sit nostrud Lorem ea pariatur. Occaecat et esse sunt dolore nisi aliquip et non do sint. Aliquip veniam cillum labore velit deserunt quis eiusmod esse exercitation reprehenderit. Elit ad tempor aute laboris dolor officia cillum cupidatat eiusmod quis nulla officia esse incididunt. Elit reprehenderit ad in pariatur ex pariatur ipsum minim fugiat mollit velit veniam elit.</p>\n<p>Qui duis excepteur culpa officia Lorem aliquip duis nisi id. Consequat ut eiusmod sit eiusmod. Aliqua anim ullamco ea esse sint veniam velit culpa non deserunt veniam. Eu elit amet reprehenderit ipsum eu ex do ullamco.</p>",
    "date": "2023-12-07T00:00:00.000Z",
    "published": true,
    "image": "/images/blog/vegetables.jpg",
    "authors": [
      "Curtis Lopez"
    ],
    "tags": [
      "Health",
      "Likely",
      "Didn't Know"
    ],
    "category": [
      "Health"
    ],
    "hasToc": false,
    "toc": [
      {
        "title": "5 Things",
        "url": "#5-things",
        "items": []
      }
    ]
  },
...
]

Property 'content' does not exist on type 'ZodMeta'.

I'm following the website guide, and use the custom transform on defineCollecion schema, to get a raw text of content:

content: s.custom().transform((data, { meta }) => meta.content),

But this line gives me the error Property 'content' does not exist on type 'ZodMeta'.
Also, if I use meta.plain, I receive the same error, but for plain.


All works fine and I can get the raw content even with TS Server warnings, it's just a type error.

Weird Page Freeze when Navigating via Table of Contents Links

I've encountered a peculiar issue on the live version of the website hosted at Deployed Live & this Github, specifically related to navigating through the Table of Contents (TOC) links. Whenever I click on any link within the TOC, the entire page freezes momentarily, and after a few seconds, it resumes normal operation, executing the intended event corresponding to the clicked link

i think that the issue may be rooted in the implementation of the MDXContent component or its interaction with other components or scripts on the page idk .. can u guide little bit how to resolve this

RecordedVideo.webm

Virtual Collection Request

One nice to have would be a way to avoid passing in a pattern when the collection you want doesn't have any files and is just a collection that will be calculated in the prepare step.

For example: I don't want to have any tag files and generate it from my posts: https://github.com/Adriel-M/adriel.dev/pull/163/files#diff-a2cacdd0436b3267547f4383a14ca08e5dd4d8369c083c869a2ea079aa5d7404R167

While it would be nice if this was also be able to be a singleton since it's more of a metadata object.

The provided config in #37 resolves warning in dev mode but still present in deployment

When the type in package.json is set to module, you should import velite by static import declaration:

import { build } from 'velite'

/** @type {import('next').NextConfig} */
export default {
  // othor next config here...
  webpack: config => {
    config.plugins.push(new VeliteWebpackPlugin())
    return config
  }
}

class VeliteWebpackPlugin {
  static started = false
  apply(/** @type {import('webpack').Compiler} */ compiler) {
    // executed three times in nextjs
    // twice for the server (nodejs / edge runtime) and once for the client
    compiler.hooks.beforeCompile.tap('VeliteWebpackPlugin', async () => {
      if (VeliteWebpackPlugin.started) return
      VeliteWebpackPlugin.started = true
      const dev = compiler.options.mode === 'development'
      await build({ watch: dev, clean: !dev })
    })
  }
}

Originally posted by @zce in #37 (comment)

image

raw.flattenPath equivalent

First of all great substitute for contentlayer thank you for that.
I am testing out velite by migrating a contentlayer project.

In contentlayer my slugs were based on the filepath. For a nested MDX file in e.g. content/firstfolder/secondfolder/test.mdx that meant the slug would be /firstfolder/secondfolder/test

To achieve that in contentlayer we had raw.flattenedPath like

const computedFields: ComputedFields = {
  slug: {
    type: "string",
    resolve: (doc) => `/${doc._raw.flattenedPath}`,
  },

I did read the velite documentation but how would that be achieved in velite?
I understand you can either use .transform for collection based computed fields or prepare for all collections.

If there is a solutions to this might consider adding it to the docs

Thank you

Unhandled Runtime Error When using remarkCodeHike

Hello, I'm moving from contentlayer to velite and everything is great until I tried adding remarkCodeHike to remarkPlugins.

The error would occur by simply adding the remark plugin

import { defineConfig } from "velite";
import { remarkCodeHike } from "@code-hike/mdx";

export default defineConfig({
    root: "content",
    collections: {
        // collections
    },
    mdx: {
        remarkPlugins: [
            remarkCodeHike
        ]
    }
})
Unhandled Runtime Error
Error: await is only valid in async functions and the top level bodies of modules

Source
src\components\ui\MDX\index.tsx (5:15) @ useMDXComponents

  3 |
  4 | const useMDXComponents = (code: string) => {
> 5 | const fn = new Function(code)
    |           ^
  6 | return fn({ ...runtime }).default
  7 | }
  8 |

Here is a fork of next.js example with codehike added to show the error:
https://stackblitz.com/edit/velite-nextjs-ladcsb?file=readme.md
navigate to /about page
I'm thinking this is caused by how mdx is compiled in velite.

Is it possible to use files stored inside github repo?

Hello, I really like velite, it's a great project so please keep working on it :).

I would like to store my files on github repo, so I could use this as hosting for my .mdx files and each website that will consume these files will be able to pass it's own react components to render to html.

I know it's possible to get files from github using contentLayer, but I would like to use velite.

content not building

Hello

I'm trying Velite first time, nice project!
But I'm having a difficulty with building the content.

I'm running

pnpm velite dev --debug --verbose      
[VELITE] using config 'velite.config.ts' in 15.42ms
[VELITE] created entry file in '.velite/index.js' 
[VELITE] created entry dts file in '.velite/index.d.ts' 
[VELITE] output entry file in '.velite' in 1.04ms
[VELITE] initialized in 19.54ms
[VELITE] resolving collections from 'content' 
[VELITE] resolve 0 files matching '/blog/**/*.md' in 3.59ms
[VELITE] resolved 0 posts 
[VELITE] wrote '.velite/posts.json' with 0 posts 
[VELITE] output 0 posts in 0.48ms
[VELITE] output 0 assets in 0.00ms
[VELITE] resolved 1 collections in 4.92ms
[VELITE] build finished in 24.95ms
[VELITE] watching for changes in 'content' 

But nothing is showing up and from the output I can see "output 0 posts" and I don't see why or what is the root cause.
My posts.json inside .velite folder is indeed empty as []
I don't get any errors anywhere.

It's a simple Nextjs setup, I have 2 .mdx files with correct markup.

This is my velite config:

import { defineConfig, defineCollection, s } from "velite";
import rehypeSlug from "rehype-slug";
import rehypePrettyCode from "rehype-pretty-code";
import rehypeAutolinkHeadings from "rehype-autolink-headings";

const computedFields = <T extends { slug: string }>(data: T) => ({
    ...data,
    slugAsParams: data.slug.split("/").slice(1).join("/"),
});

const posts = defineCollection({
    name: "Post",
    pattern: "/blog/**/*.md",
    schema: s.object({
        slug: s.path(),
        type: s.string().max(50).default("blog").optional(),
        title: s.string().max(100),
        description: s.string().max(2000).optional(),
        date: s.isodate(),
        published: s.boolean().default(true),
        body: s.mdx(),
    })
        .transform(computedFields),
});

export default defineConfig({
    root: "content",
    output: {
        data: ".velite",
        assets: "public/static",
        base: "/static/",
        name: "[name]-[hash:6].[ext]",
        clean: true,
    },
    collections: { posts },
    mdx: {
        rehypePlugins: [],
        remarkPlugins: [],
    }
});

This is my next.config.mjs:

// /** @type {import('next').NextConfig} */
// const nextConfig = {};

// export default nextConfig;

import { build } from "velite";

/** @type {import('next').NextConfig} */
export default {
  // othor next config here...
  webpack: (config) => {
    config.plugins.push(new VeliteWebpackPlugin());
    return config;
  },
};

class VeliteWebpackPlugin {
  static started = false;
  constructor(/** @type {import('velite').Options} */ options = {}) {
    this.options = options;
  }
  apply(/** @type {import('webpack').Compiler} */ compiler) {
    // executed three times in nextjs !!!
    // twice for the server (nodejs / edge runtime) and once for the client
    compiler.hooks.beforeCompile.tapPromise("VeliteWebpackPlugin", async () => {
      if (VeliteWebpackPlugin.started) return;
      VeliteWebpackPlugin.started = true;
      const dev = compiler.options.mode === "development";
      this.options.watch = this.options.watch ?? dev;
      this.options.clean = this.options.clean ?? !dev;
      await build(this.options); // start velite
    });
  }
}

The only thing I get is a warning when running pnpm run dev as following:

<w> [webpack.cache.PackFileCacheStrategy/webpack.FileSystemInfo] Parsing of /Users/xxxx/development/xxxxx-nextjs/xxxxx-nextjs/node_modules/.pnpm/[email protected]/node_modules/velite/dist/velite-B1HL3saI.js for build dependencies failed at 'import(configUrl.href)'.
<w> Build dependencies behind this expression are ignored and might cause incorrect cache invalidation.

My versions:

  "dependencies": {
     ...
    "next": "14.1.3",
    "react": "^18",
    "rehype-autolink-headings": "^7.1.0",
    "rehype-highlight": "^7.0.0",
    "rehype-pretty-code": "^0.13.0",
    "rehype-slug": "^6.0.0",
    "shiki": "^1.1.7",
    "zod": "^3.22.4"
}

  "devDependencies": {
    "@types/node": "^20",
    "@types/react": "^18",
    "@types/react-dom": "^18",
    "autoprefixer": "^10.0.1",
    "eslint": "^8",
    "eslint-config-next": "14.1.3",
    "postcss": "^8",
    "tailwindcss": "^3.3.0",
    "typescript": "^5",
    "velite": "0.1.0-beta.11"
  }

Any clue where I need to start checking or what this is causing?

Thanks!

Error on top level array for JSON/YAML

Given this config file:

import { defineConfig, s } from "velite";

export default defineConfig({
	collections: {
		data: {
			name: "Data",
			pattern: "data/index.json",
			single: true,
			schema: s.array(s.string()),
		},
	},
});

and this JSON file:

[
	"foo",
	"bar",
	"baz"
]

Running velite gives the following warnings and fails to produce any output.

Screenshot 2024-04-03 at 8 41 40 PM

Is there a way to add id to heading tags so that they can be used as anchor links?

If i have a markdown like

## Some Text

I want it to generate html code like

<h2 id="some-text">Some Text</h2>

so by using any anchor tags like <a href="#some-text">Go to some text</a> ,the page will scroll to above h2 position.

currently only that tag with children is being generated, is there an easy way to add slugified value of children as id too to heading elements only? or something like Github markdown.

content not rebuilding when modify/change files under the 'content' directory using next 14.1.3

Hello, im trying velite for the first time, its working and nice projects!
But i have weird issue, i try to modify or change the files under the 'content' directory, but not rebuilding and not showing anything on the terminal.

image

Integration with Next 14.1.3.
This is my velite config: here

Im on node v20.12.0
Try use node v.20.9.0, v18.19.0 and still same.

I try next.config same as velite documentation, but still same.

import { build } from 'velite'

/** @type {import('next').NextConfig} */
export default {
  // othor next config here...
  webpack: config => {
    config.plugins.push(new VeliteWebpackPlugin())
    return config
  }
}

class VeliteWebpackPlugin {
  static started = false
  constructor(/** @type {import('velite').Options} */ options = {}) {
    this.options = options
  }
  apply(/** @type {import('webpack').Compiler} */ compiler) {
    // executed three times in nextjs !!!
    // twice for the server (nodejs / edge runtime) and once for the client
    compiler.hooks.beforeCompile.tapPromise('VeliteWebpackPlugin', async () => {
      if (VeliteWebpackPlugin.started) return
      VeliteWebpackPlugin.started = true
      const dev = compiler.options.mode === 'development'
      this.options.watch = this.options.watch ?? dev
      this.options.clean = this.options.clean ?? !dev
      await build(this.options) // start velite
    })
  }
}

Array type pattern confuses typing

I noticed that when trying to pass a glob array to the pattern option within a collection definition this leads to the typing of each collection just becoming any.

Funnily enough, although the pattern item accepts type string it will still parse the glob array fine but break the resulting type definitions for each collection.

The docs refer to fast-glob being used for this pattern item which seems to support an array.

My main use case for this is to be able to ignore some items in production builds, but still be able to see them in development.

I see the docs have inbuilt ignore rules for items starting with _ and . but it would be nice to have more granularity.

For reference here's a snippet of what I'm doing:

const IS_DEV = process.env.NODE_ENV === 'development';
const pattern = ['fragments/**/*.{md,mdx}',].concat(!IS_DEV ? [] : ['!fragments/docs/**/*.{md,mdx}'])

export default defineConfig({
  root: './',
  collections: {
    fragments: {
      name: 'Fragment',
      // used here
      pattern: pattern,
      schema: s
        .object({
          name: s.string(),
          raw: s.raw(),
          path: s.path()
        })
    }
  }
});

This will create collections fine. But the typing of the collections becomes any[]

What is the purpose of the Recipes(especially typescript) in the documents?

While reviewing the documentation, I noticed that the articles on Typescript, Markdown, and Asset Handling in the "Recipes" section are still in progress. What is the purpose of these items? Given the content of other Recipes documents, such as MDX Support and Code Highlighting, it seems that the intention might be to explain code handling the respective resources (type, markdown, assets).

However, types are well described at https://velite.js.org/reference/types, and if it's about schema types, they are detailed at https://velite.js.org/guide/velite-schemas. Additionally, the content in the Markdown document could be easily adapted from the MDX Support article with minor modifications. What is the intended purpose of the Typescript and Markdown documents in Recipes?

If it's okay, I would like to help with writing the documentation. Perhaps starting with the seemingly lacking Asset Handling could be a good place to start. Could you tell me what content you expect in those sections(Typescript, Markdown, and Asset Handling in "Recipes")?

I want to use file without --- --- on top of it

I want to use my CHANGELOG.md as a content file, but when I don't provide a fields section on top, the file is not loaded by Velite.

To work, I need to add an empty field section but my CHANGELOG.md is automatically generated, so this is not ideal.

---

---
# Etc...

Working with 3000+ files. Out of Memoy

Hello,

working with a larger amount of Mdx files (3.000+).
That seems to be the maximum on my machine (32GB ram).
Nextjs 14.2, pnpm
When i do more than that i run into out of memory issues when building.

Is that the maximum or is there anything i can do to push that limit?
Also some warnings in the console:
image

Doesn't rebuild collections when modifying methods outside of velite config file

One thing I notice is there's no rebuild when modifying methods outside the config file while not touching the actual config file.

For example, if you modify some utility methods (such as local remark/rehype plugins or methods used in the transformation), it doesn't detect any changes and require restarting the dev command.

Revise name of generated type

I think we should append something like Meta, Metadata, or Type to our generated collection types. Currently it can be quite difficult to differentiate:

import { Post } from '#site/content';

is less self describing than

import { ProjectMetadata } from '#site/content';

or

import { ProjectType } from '#site/content';

The name of the collection alone might mislead into thinking that the imported property represents the actual content of the collection, which is not the case.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.