Coder Social home page Coder Social logo

sreerag-ibtl / jina Goto Github PK

View Code? Open in Web Editor NEW

This project forked from jina-ai/jina

0.0 1.0 0.0 15.45 MB

An easier way to build neural search in the cloud

Home Page: https://docs.jina.ai

License: Apache License 2.0

Python 94.11% HTML 3.46% Shell 0.93% Dockerfile 0.59% JavaScript 0.38% CSS 0.53%

jina's Introduction

Jina banner

Jina Python 3.7 3.8 PyPI Docker Docker Image Version (latest semver) CI CD Release Cycle Release CD API Schema codecov

English日本語FrançaisDeutschРусский язык中文

WebsiteDocsExamplesHub (beta)Dashboard (beta)Jinabox (beta)TwitterWe are Hiring

Jina is an open-source deep-learning powered search framework, empowering developers to create cross-modal or multi-modal search systems for text, images, video, and audio. Jina is a cloud-native project, and provides long-term support from a full-time, venture-backed team.

⏱️ Time Saver - Bootstrap an AI-powered system in just a few minutes with cookiecutter.

🧠 First-Class AI models - Jina is a new design pattern for neural search systems with first-class support for state-of-the-art AI models like Faiss, Annoy, Onnx, and more.

🌌 Universal Search - Large-scale indexing and querying data of any kind on multiple platforms. Video, image, long/short text, music, source code, and more.

🚀 Production Ready - Cloud-native features out-of-the-box, like containerization, microservice, distributing, scaling, sharding, async IO, REST, gRPC.

🧩 Plug & Play - With Jina Hub, you can extend Jina with simple Python scripts or Docker images optimized for your search domain.

Contents

Get Started

From PyPi

On Linux/MacOS with Python >= 3.7:

pip install jina

To install Jina with extra dependencies, or install on Raspberry Pi please refer to the documentation.

With Cookiecutter

pip install cookiecutter && cookiecutter gh:jina-ai/cookiecutter-jina

With Cookiecutter you can easily create a Jina project from templates with one terminal command. This creates a Python entrypoint, YAML configs and a Dockerfile. You can start from there.

In a Docker Container

We provide a universal Docker image that supports multiple architectures (including x64, x86, arm-64/v7/v6). Simply run:

docker run jinaai/jina --help

Jina "Hello, World!" 👋🌍

As a starter, you can try out our "Hello, World" - a simple demo of image neural search for Fashion-MNIST. No extra dependencies needed, just run:

jina hello-world

...or even easier for Docker users, no install required:

docker run -v "$(pwd)/j:/j" jinaai/jina hello-world --workdir /j && open j/hello-world.html  # replace "open" with "xdg-open" on Linux
Click here to see console output

hello world console output

The Docker image downloads the Fashion-MNIST training and test dataset and tells Jina to index 60,000 images from the training set. Then it randomly samples images from the test set as queries and asks Jina to retrieve relevant results. The whole process takes about 1 minute, and eventually opens a webpage and shows results like this:

Jina banner

The implementation behind it is simple:

Python API or use YAML spec or use Dashboard
from jina.flow import Flow

f = (Flow()
        .add(uses='encoder.yml', parallel=2)
        .add(uses='indexer.yml', shards=2, 
             separated_workspace=True))

with f:
    f.index(fashion_mnist, batch_size=1024)
!Flow
pods:
  encode:
    uses: encoder.yml
    parallel: 2
  index:
    uses: indexer.yml
    shards: 2
    separated_workspace: true

Flow in Dashboard

Explore sharding, containerization, concatenating embeddings, and more

Adding Parallelism and Sharding

from jina.flow import Flow

f = (Flow().add(uses='encoder.yml', parallel=2)
           .add(uses='indexer.yml', shards=2, separated_workspace=True))
from jina.flow import Flow

f = Flow().add(uses='encoder.yml', host='192.168.0.99')
from jina.flow import Flow

f = (Flow().add(uses='jinahub/cnn-encode:0.1')
           .add(uses='jinahub/faiss-index:0.2', host='192.168.0.99'))

Concatenating Embeddings

from jina.flow import Flow

f = (Flow().add(name='eb1', uses='BiTImageEncoder')
           .add(name='eb2', uses='KerasImageEncoder', needs='gateway')
           .needs(['eb1', 'eb2'], uses='_concat'))
from jina.flow import Flow

f = Flow(port_expose=45678, rest_api=True)

with f:
    f.block()

Intrigued? Play with different options:

jina hello-world --help

Tutorials

Jina 101 Concept Illustration Book, Copyright by Jina AI Limited      English日本語FrançaisPortuguêsDeutschРусский язык中文عربية
Level Tutorials

🐣

Search South Park scripts and practice with Flows and Pods

🐣

Extract feature vector data using any deep learning representation

🐣

Search images, define your own executors, and run them in Docker

🕊

Improve performance using prefetching and sharding

🚀

Search Pokemon with state-of-the-art visual representation

Documentation

The best way to learn Jina in depth is to read our documentation. Documentation is built on every push, merge, and release of the master branch.

The Basics

Reference

Are you a "Doc"-star? Join us! We welcome all kinds of improvements on the documentation.

Documentation for older versions is archived here.

Contributing

We welcome all kinds of contributions from the open-source community, individuals and partners. We owe our success to your active involvement.

Contributors ✨

All Contributors

Community

  • Slack workspace - a communication platform for developers to discuss Jina
  • YouTube channel - subscribe to the latest video tutorials, release demos, webinars and presentations.
  • LinkedIn - get to know Jina AI as a company and find job opportunities
  • Twitter Follow - follow us and interact with using hashtag #JinaSearch
  • Company - know more about our company and how we are fully committed to open-source.

Open Governance

GitHub milestones lay out the path to Jina's future improvements.

As part of our open governance model, we host Jina's Engineering All Hands in public. This Zoom meeting recurs monthly on the second Tuesday of each month, at 14:00-15:30 (CET). Everyone can join in via the following calendar invite.

The meeting will also be live-streamed and later published to our YouTube channel.

Join Us

Jina is an open-source project. We are hiring full-stack developers, evangelists, and PMs to build the next neural search ecosystem in open source.

License

Copyright (c) 2020 Jina AI Limited. All rights reserved.

Jina is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.

jina's People

Contributors

alexcg1 avatar allcontributors[bot] avatar anish2197 avatar antonkurenkov avatar bingho1013 avatar boussoffara avatar emmaadesile avatar ericsyh avatar festeh avatar fhaase2 avatar generall avatar guiferviz avatar hanxiao avatar jamestang616 avatar jina-bot avatar joanfm avatar joaopalotti avatar kavan72 avatar maanavshah avatar morriaty-the-murderer avatar nan-wang avatar phamtrancsek12 avatar redram avatar roccia avatar rohan1chaudhari avatar rutujasurve94 avatar shivam-raj avatar xiongma avatar yiksanchan avatar yueliu1415926 avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.