Coder Social home page Coder Social logo

rapture's Introduction

Rapture

Go Report Card v0.3.6 DeepSource
No-nonsense data-breach search interface
Read the Guide


Table of Contents

Information

About

Rapture is a simple and cross-platform, "no-nonsense" data-breach search interface designed to enable individuals to effortlessly index and query compromised data to identify exposure and protect personal information and assets.

Attribution

This software is heavily inspired by MiyakoYakota's search.0t.rocks (Github | Archive), uses the same Solr configuration, and remains consistent with their core tenants of simplicity and usability. Thank you for your grand contribution to the development of open-source intelligence software and for providing a means for the community to expand on your developments.

Disclaimer

It is the end user's responsibility to obey all applicable local, state, and federal laws. Developers assume no liability and are not responsible for any misuse or damage caused by this program. This software comes AS IS with NO WARRANTY and NO GUARANTEES of any kind. By using Rapture, you agree to the previous statements.

AI Disclosure

The Rapture social card/preview uses content generated by OpenAI's DALL-E 2 AI system.

Prerequisites

  • Git
  • Docker Engine w/ Docker Compose
  • Java 8 or later

Getting Started

A more comprehensive guide to configuring Rapture will be created in the near future. For now, the following instructions should suffice.

  1. Fetch the repository: git clone https://github.com/ax-i-om/rapture.git
  2. Navigate into Rapture's root directory: cd rapture
  3. Run: docker compose build
  4. Run: docker compose up -d

The initialization process may take ~1min, but that's it! You can now navigate to the Rapture web page at http://localhost:6175.

Data Conversion

Below is an example of a valid/cleaned CSV file titled header.csv:

id,emails,passwords,firstName,lastName
ficticiousbreach-03122024-1,[email protected],VK9RSuK0wSSXNc0gF8iYW1f6,axiom,estimate
ficticiousbreach-03122024-2,[email protected],N7cblKU9ypU727lwiTr9espw,garble,vortex
ficticiousbreach-03122024-3,[email protected],eP0u9cM0jAa2QeUVI3d88rYn,vertiable,slap
ficticiousbreach-03122024-4,[email protected],QxtyRMAx3KniskzjGDg6tHdl,axiom,terrific
ficticiousbreach-03122024-5,[email protected],cirSMQZp7Enh98KLb6r8JT1I,garble,eighty
ficticiousbreach-03122024-6,[email protected],9J53HQEetTv5E2xCJKe4tdaP,veritable,tumble
ficticiousbreach-03122024-7,[email protected],3sZ7NPb54Fk0Qy2LXlLejwCu,axiom,slipper
ficticiousbreach-03122024-8,[email protected],OxoZGbn3v0tvBMyWA0Jds0Ea,garble,chord
ficticiousbreach-03122024-9,[email protected],et8ggkgUeyQ3ge7ua2YNsOLd,veritable,baffle

Only the fields specified during the execution of setup.sh are valid, and fields are case sensitive; therefore, firstName is a valid key/field, whereas firstname and FIRSTNAME are invalid. Likewise, the passwords field is valid; however, the singular iteration of the word (password) is invalid.

Note

Each data point is required to have an id field. It is up to you how to format this field; however, it is important that each and every occurrence of an id is unique. In the case of a duplicate id, data will be overwritten.

Data Importation

The process for importing data is straightforward. In the example below, we will use the Solr post tool to post the data from the header.csv file exhibited above.

Note

Supported file types: xml,json,csv,pdf,doc,docx,ppt,pptx,xls,xlsx,odt,odp,ods,ott,otp,ots,rtf,htm,html,txt,log

Note

The Solr post tool requires a Java runtime greater than version 8, and can be installed like so: sudo apt install openjdk-19-jre-headless

  1. Download Solr binary release from: https://solr.apache.org/downloads
  2. Extract binaries: tar zxvf solr-9.5.0.tgz
  3. Post data to Solr collection: /home/axiom/Desktop/solr-9.5.0/bin/post -c BigData -p 8983 -host 127.0.0.1 header.csv

Ensure that you change the command specified in step 3 to accomodate:

/home/axiom/Desktop/solr-9.5.0/bin/post - Path to Solr post binary extracted (extracted in step 2)

-c BigData - Collection name (Leave this as is, unless you modified setup.sh)

-p 8983 - Solr service port (Leave this as is, unless you modified setup.sh)

-host 127.0.0.1 - Solr service host address (Leave this as is, unless you modified setup.sh)

header.csv - File containing clean data you wish to post. This is the file we demonstrated in Data Conversion

After you have posted the data, you should be able to refresh the Rapture web page and successfully query the information!

rapture's People

Contributors

ax-i-om avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

Forkers

jermanuts

rapture's Issues

java.io.FileNotFoundException when sending data via Solr post tool 9.x.x

When trying to send data to the Rapture instance with locally installed solr tools I encounter an error java.io.FileNotFoundException.

This was edited as I was sending a request to a local IP with a filename censored, for the purpose of the report I'm sending the file to 127.0.0.1 and the file is file.jsonl.

I have done the appropriate setup commands of sudo docker compose build and sudo docker compose up -d to start the containers.

Command Output

$ /opt/solr/bin/post -c BigData -p 8983 -host 127.0.0.1 file.jsonl
The bin/post script is deprecated in favour of the bin/solr post command. Please update your scripts.
java -classpath /opt/solr/server/solr-webapp/webapp/WEB-INF/lib/solr-core-9.6.1.jar -Dauto=yes -Dport=8983 -Dhost=127.0.0.1 -Dc=BigData -Ddata=files org.apache.solr.cli.SimplePostTool file.jsonl
SimplePostTool version 9.6.1
Posting files to [base] url http://127.0.0.1:8983/solr/BigData/update...
Entering auto mode. File endings considered are xml,json,jsonl,csv,pdf,doc,docx,ppt,pptx,xls,xlsx,odt,odp,ods,ott,otp,ots,rtf,htm,html,txt,log
POSTing file file.jsonl (application/jsonl) to [base]/extract
SimplePostTool: WARNING: Solr returned an error #404 (Not Found) for url: ...
SimplePostTool: WARNING: Response: <p>
  Searching for Solr?<br/>
  You must type the correct path.<br/>
  Solr will respond.
</p>
SimplePostTool: WARNING: IOException while reading response: java.io.FileNotFoundException: ...
1 files indexed.
COMMITting Solr index changes to http://127.0.0.1:8983/solr/BigData/update...
Time spent: 0:01:42.727

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.