jina-ai / jinabox.js

A lightweight, customizable omnibox in Javascript, for use with a Jina backend.

jinabox.js is a lightweight, customizable set of components for querying and displaying results from Jina flows. You can use it for searching text, images, videos, audio or any kind of data with a Jina backend.

• Learn Jina: the fastest way to build cloud-native neural search
• Features
• Jinabox in Action
• Adding Jinabox to Your Project
• Supported MIME types
• Configuration

Features

• <jina-searchbar></jina-searchbar>

  • perform queries and view results in a single component

• <jina-results></jina-results>

  • place results from searchbar in a separate area

• <jina-floater></jina-floater>

  • standalone floating icon to enable searches from anywhere in the application

• <jina-floater-chat></jina-floater-chat>

  • jina floater in the style of a chat application

• Easy-to-use web components for Jina search backend

• Users can drag and drop multi-modal data as search queries

• Webcam and microphone support (in progress, contributions welcome! ❤️)

• Visualize search results in a dropdown list or separate container

• Highly customizable

jinabox.js in Action

Spin up a Jina Docker Image

jinabox.js is a frontend for querying Jina flows. You will need to spin up a Jina backend to get search results. Based on what you want to search, run one of our example Jina backends:

• Pokedex (real image data): docker run -p 65481:65481 -e "JINA_PORT=65481" jinaai/hub.app.bitsearch-pokedex search
• Southpark (real text data): docker run -p 45678:45678 jinaai/hub.app.distilbert-southpark
• MP4 mock (placeholder video data): docker run -p 65481:65481 -e "JINA_PORT=65481" jinaai/hub.app.iomock.mp4 search
• Wav mock (placeholder audio data): docker run -p 65481:65481 -e "JINA_PORT=65481" jinaai/hub.app.iomock.sound search

Open jinabox.js in your Browser

Just go to https://jina.ai/jinabox.js/ in your browser to open up the search interface.

Set Server Endpoint

Based on the Docker port you are exposing (i.e. 65481 or 45678 in above Docker images), set jinabox's server endpoint to:

http://localhost:<port_number>/api/search

Important: If you are using a remote endpoint (non-localhost), make sure it is using HTTPS, not HTTP. Otherwise, JinaBox will not be able to make requests to your flow.

Start Searching!

Drag and drop images, video, or audio, or type in text to search.

Adding jinabox.js to Your Project

In your HTML source, include the minified script and initialize it with your endpoint:

<script src="https://unpkg.com/jinabox"></script>
<script>
    jb = window.JinaBox
    jb.init('http://localhost:65481/api/search');  // http://localhost:65481/api/search is the endpoint of the REST gateway of a Jina flow with REST enabled.
</script>

Then add the jinabox web components into your application:

<jina-searchbar></jina-searchbar>

<!--or-->

<jina-floater></jina-floater>

Now you can drag anything from anywhere (local/browser/webpage) to it and conduct the search.

Ready to learn Jina? Read our 101 tutorials.

Installation via package manager

npm install jinabox

or

yarn add jinabox

Supported MIME types

MIME type	Input (Query)	Output (Result)	Description
text	Typing	List only	Text-only data including any human-readable content, source code, or textual data such as comma-separated value (CSV) formatted data. Examples include text/plain, text/csv, and text/html.
image	Drag-and-drop	<img> Containers in list/grid view	Image or graphical data including both bitmap and vector still images as well as animated versions of still image formats like animated GIF or APNG. Common examples are image/jpeg, image/png, and image/svg+xml.
video	Drag-and-drop, webcam	<video> containers in list/grid view	Video data or files, such as MP4 movies (video/mp4).
audio	Drag-and-drop, webcam, mic	<audio> containers in list view	Audio or music data. Examples include audio/mpeg, audio/vorbis.

Configuration

Play with and preview jinabox configurations here: https://jina.ai/jinabox.js/

Setting	Default	Type	Description
theme	default	string	Color theme: default, persian, pompelmo, honeybee
searchIcon	color	string	Searchbar icon: color, mono, inverse, or a URL
showDropzone	true	boolean	Show expanding drop zone when dragging files
resultsLocation	dropdown	string	Where to display results: dropdown or external. When selecting external, results will be rendered in the element corresponding to the results-area-id
resultsAreaId	jina-results-area	string	ID of where results will be rendered if resultsLocation is set to external
typewriterEffect	false	boolean	Enable typewriter effect on the placeholder
typewriterDelayItem	1000	number	Time (ms) delay between every placeholder switch when typewriterEffect is enabled
typewriterDelayCharacter	50	number	Time (ms) delay between each character when typwriterEffect is enabled
userMediaHeight	500	number	Webcam feed and capture height
userMediaWidth	300	number	Webcam feed and capture width
acceptAudio	true	boolean	Allow search with audio queries
acceptVideo	true	boolean	Allow search with video queries
acceptText	true	boolean	Allow search with text queries
acceptImage	true	boolean	Allow search with image queries

Expected Response Structure

jinabox.js expects query responses to contain either of the following structures

{
  "search": {
    "docs": [
      {
        "topkResults": [
          {
            "matchDoc": {
              "docId": 14704,
              "weight": 1,
              "uri": "data:image/png;charset=utf,...",
              "mimeType":"image/png"
            }
          }
        ],
        "uri":"data:image/jpeg;...",
        "mimeType":"image/jpeg"
      }
    ]
  }
}

or

{
  "search": {
    "docs": [
      {
        "matches": [
          {
            "docId": 14704,
            "weight": 1,
            "uri": "data:image/png;charset=utf,...",
            "mimeType":"image/png"
          }
        ],
        "uri":"data:image/jpeg;...",
        "mimeType":"image/jpeg"
      }
    ]
  }
}

Browser Support

jinabox.js runs on modern browsers including Chrome, Firefox, Safari (desktop and mobile) and Opera. Certain features (like recording audio/video) are not available on all browsers and jinabox will make these features available accordingly.

Troubleshooting

SSL Error on Localhost

Try http://localhost:65481/api/search, make sure to use http and localhost, instead of 0.0.0.0.

Requests hanging or failing

Query your api with curl to ensure it is properly receiving, processing, and responding to requests. Ensure responses meet jinabox expected response structure.

Contributing

We welcome all kinds of contributions from the open-source community, individuals and partners. We owe our success to your active involvement.

• Contributing guidelines
• Release cycles and development stages

Community

• Code of conduct - play nicely with the Jina community
• Slack workspace - join #general on our Slack to meet the team and ask questions
• YouTube channel - subscribe to the latest video tutorials, release demos, webinars and presentations.
• LinkedIn - get to know Jina AI as a company and find job opportunities
• - follow and interact with us using hashtag #JinaSearch
• Company - know more about our company and how we are fully committed to open-source.

Open Governance

As part of our open governance model, we host Jina's Engineering All Hands in public. This Zoom meeting recurs monthly on the second Tuesday of each month, at 14:00-15:30 (CET). Everyone can join in via the following calendar invite.

• Add to Google Calendar
• Download .ics

The meeting will also be live-streamed and later published to our YouTube channel.

Join Us

Jina is an open-source project. We are hiring full-stack developers, evangelists, and PMs to build the next neural search ecosystem in open source.

License

Copyright (c) 2020 Jina AI Limited. All rights reserved.

Jina is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.

jinabox.js's Issues

Update documentation to show new configurations options and requirements

┆Issue is synchronized with this Jira Story by Unito

UI: Drop zones not working on Firefox

Describe your problem

In Firefox, drag and drop does not work. Dragging a Pokemon image from the left side doesn't cause drop zones to pop up, or causes the lower left one to pop up when mouse is not near it (closing the pop-up when the mouse gets closer)

Dropping the file leads to expected result (i.e. search results appearing)

What is your guess?
I guess it was coded for Chrome first and not tested on FF?

---

Environment

• Firefox 77.01 on Manjaro Linux
• Javascript enabled
• Server endpoint: http://localhost:65481/api/search
• Pokedex example

Screenshots
Can't capture a screenshot when dragging the mouse unfortunately

UI: Two dropzones is confusing

Describe your problem
Dragging and dropping a Pokemon image (or anything else) results in two dropzones (search bar and action button on bottom left). This is a little confusing - where should I drop?

I mean, I know I can drop anywhere, but it's not clear to new users straight away. It'd be like your browser having two refresh buttons that both do same thing.

Suggestion

• Remove one drop zone or expand dropzone to whole area

---

Environment
Chrome on Linux (dropzones work inconsistently in Firefox, see #4)

Screenshots
No screenshot because it won't work while dragging

scale the display images to a proper size

I've tried flowersearch example with jinabox. Currently, the images totally break the UI.

drag png problem summary

• ✅ drag a png (in data uri) from same page
• ✅ drag a png from local
• ❌ drag a png (with http src) from the same page -> broken, expected behavior need to be discussed
  • option 1, convert it to datauri in the frontend and
  • option 2, recognize it as url and download it at the backend
    

      chunk_seg@ 8[E]:expecting data URI, but got http://pngimg.com/uploads/pokemon/pokemon_PNG149.png
      chunk_seg@ 8[E]:unknown exception: 'NoneType' object has no attribute 'items'
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/jina/peapods/pea.py", line 358, in run
    self.loop_body()
  File "/usr/local/lib/python3.7/site-packages/jina/peapods/pea.py", line 323, in loop_body
    msg = self.zmqlet.recv_message(callback=self.msg_callback)
  File "/usr/local/lib/python3.7/site-packages/jina/peapods/zmq.py", line 223, in recv_message
    return callback(msg)
  File "/usr/local/lib/python3.7/site-packages/jina/peapods/pea.py", line 301, in msg_callback
    return self._callback(msg)
  File "/usr/local/lib/python3.7/site-packages/jina/peapods/pea.py", line 291, in _callback
    self.pre_hook(msg).handle(msg).post_hook(msg)
  File "/usr/local/lib/python3.7/site-packages/jina/peapods/pea.py", line 179, in handle
    self.executor(self.request_type)
  File "/usr/local/lib/python3.7/site-packages/jina/executors/__init__.py", line 537, in __call__
    d()
  File "/usr/local/lib/python3.7/site-packages/jina/drivers/craft.py", line 74, in __call__
    for k, v in ret.items():
AttributeError: 'NoneType' object has no attribute 'items'

• ❌ drag a png (with http src) from another tab -> no effect, backend receive nothing
• ❌ drag a png (in data uri) from another tab (you can drag one from https://github.com/jina-ai/examples/blob/master/pokedex-with-bit/result.html) -> no effect, backend receive nothing

UI: Server endpoint text displayed by default but not set

Describe your problem
I tried for ages to get jinabox working with endpoint http://localhost:65481/api/search. I thought it was already set to that because that's already displayed in the server endpoint textbox, but turns out it's just placeholder text and not actually set.

This is really not obvious :/

What is your guess?

• Placeholder text is too dark. Lighten it to ensure it's clear that's placeholder
• Pop up an error if there is no server endpoint set, and highlight the server endpoint box in red

---

Environment
Firefox and Chrome on Manjaro

Screenshots

Image upload from "browse" button does nothing

Describe your problem

• Go to 'Upload' tab
• Browse for image, select image, click OK
• Nothing happens

What is your guess?

No idea. Dragging and dropping images from file folder works fine though

---

Environment
Chrome/Firefox on Manjaro Linux

Screenshots
Since nothing happens, nothing to screenshot

Can't drag and drop bottom left image

On https://jina.ai/jinabox.js/ it is not possible to drag and drop the image on the bottom left

Update jinabox to show cross modal results with appropriate rankings

┆Issue is synchronized with this Jira Story by Unito

Parse new data structure to work with cross modal examples

┆Issue is synchronized with this Jira Story by Unito

can't set server endpoint from https://jina.ai/jinabox.js

I'm trying examples/southpark-search. As tutorial, I get everything ready and opened https://jina.ai/jinabox.js in Chrome browser, but I can't set the server endpoint since no place to fill in the text.

The below is the pic in tutorial, which might be an older version.

add CLA bot

fix: search request top_k parameter parsing

When trying to use the Jina 2.0 API with jinabox, a problem occurred.
Currently in Jinabox, the top_k parameter is passed in the search request like this:

xhr.send(JSON.stringify({ data, top_k: parseInt(top_k), mode: 'search' }));

This leads to the parameter not being passed by the Jina 2.0 API.
To be able to access the parameter through the Jina 2.0 requests decorator, the line needs to be changed to:

xhr.send(JSON.stringify({ data, parameters: {top_k: parseInt(top_k)}, mode: 'search' }));

Add grid/list view toggle for single item queries

┆Issue is synchronized with this Jira Story by Unito

Jinabox doesn't work

The error occurs when jinabox changes from 0.1.49 to 0.1.50. There seems to be a problem when loading the script, but it works if i use 0.1.49 script. It would be good if you check the 0.1.50 script once.

Can't drag and drop bottom left image

On https://jina.ai/jinabox.js/ it is not possible to drag and drop the image on the bottom left

Add error handling to parsing

┆Issue is synchronized with this Jira Story by Unito

dropping text is not supported yet

As a user, I would like to be able to select a large corpus of text and drop it into jinabox so that I don't need to copy and paste.

Add document meta_info representation to jinabox front-end

For the Jina AI hackathon, I wanted to see if there was a way to add decoded document meta_info onto the front-end portion of the static Vue appl that was supplied within the mutlires-lyrics-search example.

Based on a conversation with @nan-wang, it would seem that this feature doesn't currently exist, therefore, creating this issue to track this feature request.

┆Issue is synchronized with this Jira Task by Unito

Create debug mode to show all logs

┆Issue is synchronized with this Jira Story by Unito