Coder Social home page Coder Social logo

narugo1992 / gchar Goto Github PK

View Code? Open in Web Editor NEW
17.0 17.0 2.0 11.69 MB

Crawler and cleaner of data for novelai embedding's training

Home Page: https://narugo1992.github.io/gchar/

License: Apache License 2.0

Makefile 0.26% Shell 0.46% Python 99.29%

gchar's People

Contributors

narugo1992 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

Forkers

kimiko-ai

gchar's Issues

todolist for 2023

  • Game Character Database
    • Arknights
    • FGO
    • Genshin
    • Azuelane
    • Girls' Frontline
  • Crawlers
    • Pixiv Crawler Module
    • Danbooru Crawler Module
  • Image Tagging & Featuring
    • Onnx-based deepdanbooru
    • Onnx-based waifudiffusion1.4 (tagging)
    • Image Feature Similarity
    • Images Clustering
  • Object Detection
    • yolo5-based anime person detection
  • Image Tools
    • Smart Crop to Given Size
    • Pad and Resize to Given Size
    • Color Clustering (not necessary for now)
    • Image Similarity
    • Edge Detection & Similarity
    • GrayScale Image Detection
    • Mosaic (Pixelate & Blur)
  • Censoring
    • R18 Detection (based on nudenet)
    • Danbooru Rating Detection (based on tagging methods)
    • nudenet-based genital detection (for censoring)
  • Pipeline
    • Load Image Set from Local
    • Load Image Set from Crawler
    • Information-embedded Image Model
    • Scalable Pipeline Design
    • Processed Image Set Export

Pixiv-based Illustration Count

Get pixiv-based illustration count of characters.

PS: The api in pixivpy3 (illust_search) can only offset no more than 5000.

Refactor of Games Models

  • Simplify CLI interface
  • Simplify get index function (crawler session)
  • Simplify some properties (such as gender, maybe the enum can be frameworked)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.