Coder Social home page Coder Social logo

zk / clojuredocs Goto Github PK

View Code? Open in Web Editor NEW
1.0K 25.0 80.0 11.38 MB

clojuredocs.org web app

Home Page: http://clojuredocs.org

License: Eclipse Public License 1.0

Shell 0.40% JavaScript 0.05% Clojure 86.59% HTML 1.55% Less 11.40% Procfile 0.01%
clojure clojuredocs

clojuredocs's Introduction

clojuredocs

The clojuredocs.org webapp

Contributing

A few ways to contribute:

Let's use GH issues for discussion for now

If you're looking for a project:

  • Content for namespaces (see src/md/namespaces) needs to be added / edited. Example of clojure.core.async
  • Listing of clojure training / classes / events on home page
  • Stand-alone example page, maybe have the var info (signature, doc string, etc) at the top.
  • Source-linking on libs not included in the standard library e.g. core.async.

Deploy

Production is deployed on an AWS t2.micro instance. There's an nginx process running on the box, balancing to two JVMs managed by Upstart to support zero-downtime deploys.

To regenerate the upstart scripts:

cd $REPO
sudo foreman export -a clojuredocs -e ./.env -u ubuntu -c "web=2" upstart /etc/init/

To start the app processes:

sudo service clojuredocs-web-1 start
sudo service clojuredocs-web-2 start

To redeploy:

# in $REPO
sudo service clojuredocs-web-1 stop
git pull origin master
# This will compile assets & run tests
bin/build
sudo service clojuredocs-web-1 start
# Wait for proc 1 to start serving requests
sudo service clojuredocs-web-2 restart

Reqs

Dev

Run bin/dev, which will start all the things (repl, web process, scss compiler, etc). See Procfile for more info.

Connect to the repl and / or visit http://localhost:4000

You'll notice that var information is already populated. In an effort to not make the same mistakes again, all core-related var info is loaded from the runtime version of Clojure on start up.

OTOH, examples, see-alsos, and notes (and any other user-generated content) are stored in the database.

Local Data

The app uses a MongoDB database named clojuredocs to store data. Run bin/db-reset to seed the database with a recent production export (you must be running mongod for this to work).

Prod Local

Occasionally you'll need to compile and run things as they would be in production (checking advanced cljs compilation, for example): bin/prod-local.

CLJS Source Maps

The ClojureDocs project is set-up to emit source-maps for compiled javascript. To enable in Chrome, check the 'Enable JS source maps' option in the Developer Tools settings pane.

Clojure Version

Clojure vars are pulled directly from the runtime, and are not stored in the database. When new versions of Clojure are released:

  • Change the Clojure dep in project.clj
  • Update the version string, source base url, and gh tag url in clojuredocs.search/clojure-lib
  • Update the version for the mobile nav in clojuredocs.pages.common
  • Update the github URL in clojuredocs.pages.vars/source-url.

App Structure

Interesting files:

  • src/clj/clojuredocs/main.clj -- Main entry-point into the app, starts the jetty server
  • src/clj/clojuredocs/entry.clj -- Root routes and middleware
  • src/clj/clojuredocs/pages.clj -- User-facing HTML pages
  • src/clj/clojuredocs/api.clj -- API endpoints for ajax calls from the frontend.
  • src/cljs/clojuredocs/main.cljs -- Main js entry-point into the app

Conventions

  • Functions that return hiccup structures should be prefixed with a $, like $layout.
  • Mutable state should be prefixed with a !, ex: !my-atom.

Adding Functions, Macros, Special Forms, Namespaces & Other Vars

Most vars are picked up from Clojure at runtime, using core namespace and var introspection utilities. Special forms and namespaces to include on the site are specified explicitly in the clojuredocs.search.static namespace.

Vars are picked up automatically based on the namespaces specified in clojuredocs.search.static/clojure-namespaces vector. Any namespace in this vector will be queried for public vars to be made searchable and displayable on the site.

Special forms are specified in the clojuredocs.search.static/special-forms list, and require a server restart to be picked up in a dev environment.

Adding Core Libraries

Sometimes we'd like to add core libraries that are not part of the standard Clojure distribution (like core.async) to the site. Here's how to do that:

  1. Add dependency to project.clj
  2. Add ns sym to clojure-namespaces in clojuredocs.search.static
  3. Add a short description + links to community articles / videos / other resources to src/md/namespaces/

Dev-Prod differences

  • Dev starts the environment using lein repl :headless, prod uses lein run -m clojuredocs.main. See :repl-options in project.clj for initialization options.

Data Exports

Contributors

Zachary Kim, Lee Hinman, and more.

License

Copyright © 2010-present Zachary Kim

Distributed under the Eclipse Public License either version 1.0 or (at your option) any later version.

clojuredocs's People

Contributors

bfontaine avatar cantido avatar cindywu avatar cldwalker avatar dakrone avatar dimmyjr avatar dotemacs avatar ertugrulcetin avatar eval avatar glts avatar jeremyvdw avatar jgertm avatar jim-sokoloff avatar jstepien avatar kochb avatar magemasher avatar mneise avatar msporleder avatar nb avatar oskarkv avatar rakyi avatar reborg avatar sanrodari avatar seanmalloy avatar svenschoenung avatar szemek avatar taravancil avatar tippenein avatar vale981 avatar zk avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

clojuredocs's Issues

Searching for an exact keyword doesn't return that keyword at the top.

Hi,

I noticed that if you search for a function with it's literal name, it doesn't always show up at the top of the results while it does show up at the top of the search dropdown. Is this expected behaviour?
For example when searching for simple functions like when, do, or if , those only show up at respectively the 6th, 3rd or 5th place in the queries.

External Link Section

Would be useful, after comments, to have an external link section for pointing to other blogs who have introduced the var being documented. Currently comments are used for that, but that just feels like a hack.

Don't cache page headers

[1:16 PM] <coopernurse> srid: clojuredocs is also good, and has a quick ref that groups by data structure http://clojuredocs.org/quickref/Clojure%20Core
 [1:16 PM] <coopernurse> I like the examples on clojuredocs
 [1:26 PM] <srid> coopernurse: that's very handy! examples are what I most like about clojuredocs as well.
 [1:26 PM] <srid> coopernurse: btw, when I clicked on your link I see "Logged in as coopernurse" on the top-right corner.
 [1:26 PM] <srid> session caching bug?
 [1:26 PM] <coopernurse> srid: woah. on the clojuredocs site it says that?
 [1:27 PM] <srid> yes
 [1:27 PM] <srid> I hit the same bug while developing code.activestate.com
 [1:27 PM] <srid> then I remembered the I mistakenly cached the page header too. so if one user loaded the page, and when the next did the same immediately he would see the former user's page header
 [1:27 PM] <coopernurse> interesting.. so this is just a cosmetic issue?
 [1:27 PM] <coopernurse> for example, can you change my email address?
 [1:28 PM] <coopernurse> on my clojuredocs profile page
 [1:28 PM] <srid> no, it is only a page caching bug. it just loads it from the cache for me (which cache was last-updated when you did)

Var versioning

Adding version support requires changes in several parts of the app:

  • Schema changes
    • Moving everything except for libarary, ns, and name from the functions table to a function_versions table.
    • Adding a version column to the function_versions table.
  • Updating site logic to use function.latest_version / function.versions where appropriate.
  • Updating UI to provide links on the function.html.erb page to view previous versions.
    • Only having links to versions which have changed from the previous version, keying off the docstring.
  • Updating cd-analyzer
    • Importing to new schema
    • Still removing vars that are no longer in the codebase (?)

New user signup fails with OpenID provider hyves.nl

log output:

Processing UserSessionsController#new (for 76.25.241.113 at 2010-10-15 09:05:52) [POST]
  Parameters: {"openid.mode"=>"id_res", "openid.op_endpoint"=>"https://openid.hyves-api.nl/", "openid.return_to"=>"http://clojuredocs.org/login?for_session=1&_method=post", "openid.ns.sreg"=>"http://openid.net/extensions/sreg/1.1", "openid.sig"=>"Mth1eqCjmMqIOx94pLyWeuRf9cTs3E051sb5Z2tfnF8=", "openid.response_nonce"=>"2010-10-15T08:57:53ZMTQyNDU0OTM3Nw==", "openid.ns"=>"http://specs.openid.net/auth/2.0", "action"=>"new", "for_session"=>"1", "openid.identity"=>"http://zacharykim.hyves.nl/", "openid.ns.ax"=>"http://openid.net/srv/ax/1.0", "openid.assoc_handle"=>"SE1BQy1TSEEyNTZfbm8tZW5jcnlwdGlvbl8xMjg3MjEyODU5XzE4NzI3NDI5MDdfTzZxUXZTekd0QVZTZjJxOVBKc1U0QmNzZUZnPQ==", "openid.signed"=>"assoc_handle,ax.mode,claimed_id,identity,mode,op_endpoint,response_nonce,return_to,ns,ns.ax,ns.sreg,signed", "controller"=>"user_sessions", "openid.ax.mode"=>"fetch_response", "openid.claimed_id"=>"http://zacharykim.hyves.nl/#NzgzZjI4Nj"}

ActionController::InvalidAuthenticityToken (ActionController::InvalidAuthenticityToken):
  passenger (2.2.15) lib/phusion_passenger/rack/request_handler.rb:92:in `process_request'
  passenger (2.2.15) lib/phusion_passenger/abstract_request_handler.rb:207:in `main_loop'
  passenger (2.2.15) lib/phusion_passenger/railz/application_spawner.rb:441:in `start_request_handler'
  passenger (2.2.15) lib/phusion_passenger/railz/application_spawner.rb:381:in `handle_spawn_application'
  passenger (2.2.15) lib/phusion_passenger/utils.rb:252:in `safe_fork'
  passenger (2.2.15) lib/phusion_passenger/railz/application_spawner.rb:377:in `handle_spawn_application'
  passenger (2.2.15) lib/phusion_passenger/abstract_server.rb:352:in `__send__'
  passenger (2.2.15) lib/phusion_passenger/abstract_server.rb:352:in `main_loop'
  passenger (2.2.15) lib/phusion_passenger/abstract_server.rb:196:in `start_synchronously'
  passenger (2.2.15) lib/phusion_passenger/abstract_server.rb:163:in `start'
  passenger (2.2.15) lib/phusion_passenger/railz/application_spawner.rb:222:in `start'
  passenger (2.2.15) lib/phusion_passenger/spawn_manager.rb:253:in `spawn_rails_application'
  passenger (2.2.15) lib/phusion_passenger/abstract_server_collection.rb:126:in `lookup_or_add'
  passenger (2.2.15) lib/phusion_passenger/spawn_manager.rb:247:in `spawn_rails_application'
  passenger (2.2.15) lib/phusion_passenger/abstract_server_collection.rb:80:in `synchronize'
  passenger (2.2.15) lib/phusion_passenger/abstract_server_collection.rb:79:in `synchronize'
  passenger (2.2.15) lib/phusion_passenger/spawn_manager.rb:246:in `spawn_rails_application'
  passenger (2.2.15) lib/phusion_passenger/spawn_manager.rb:145:in `spawn_application'
  passenger (2.2.15) lib/phusion_passenger/spawn_manager.rb:278:in `handle_spawn_application'
  passenger (2.2.15) lib/phusion_passenger/abstract_server.rb:352:in `__send__'
  passenger (2.2.15) lib/phusion_passenger/abstract_server.rb:352:in `main_loop'
  passenger (2.2.15) lib/phusion_passenger/abstract_server.rb:196:in `start_synchronously'

Rendering /var/www/clojuredocs/public/422.html (422 Unprocessable Entity)

Update rss feed to output html from code-only examples

As of now, items in the rss feed for recent updates (http://clojuredocs.org/feed/recent_updates) are showing up as walls of text due to the new examples changes (http://bit.ly/ca2lRR).

Basically, line breaks aren't being translated correctly, and needs to be changed so that:

user=> (map inc [1 2 3 4 5]) (2 3 4 5 6) ;; map can be used with multiple collections. Collections will be consumed ;; and passed to the mapping function in parallel: user=> (map + [1 2 3] [4 5 6]) (5 7 9) ;; When map is passed more than one collection, the mapping function will...

looks like:

user=> (map inc [1 2 3 4 5]) (2 3 4 5 6)

;; map can be used with multiple collections. Collections will be consumed
;; and passed to the mapping function in parallel:
user=> (map + [1 2 3] [4 5 6]) (5 7 9)

;; When map is passed more than one collection, the mapping function will...

in your rss reader.

What URL to use for API and Clojure functions +, /, .. ?

3rd party lib requests

3rd party lib support is still a ways off, so this issue will be used to track 3rd party library import requests.

Create Ability to Export to PDF

I'd like to use ClojureDocs when I'm offline, is there a way to down the sit as a pdf or locally(without building from source) to have it when I need some examples without the internet.

THANKS!

Automated DB backup

I'm thinking hourly, AES encrypted backups to S3. I'm on the fence about incrementals.

clojuredocs doesn't handle libraries with '/' in the title well

Playing around with cd-analyzer, I had it analyze my API client, which has the full name org.thnetos/cd-client. The analyze went fine, going to the actual page (http://localhost:3000/org.thnetos/cd-client) causes a problem:

Processing MainController#ns (for 127.0.0.1 at 2010-09-09 22:03:52) [GET]
Parameters: {"action"=>"ns", "lib"=>"org.thnetos", "ns"=>"cd-client", "controller"=>"main"}
Namespace Columns (4.5ms)   SHOW FIELDS FROM `namespaces`
Namespace Load (2.7ms)   SELECT * FROM `namespaces` WHERE (`namespaces`.`name` = 'cd-client') LIMIT 1
User Columns (1.8ms)   SHOW FIELDS FROM `users`
Library Columns (1.8ms)   SHOW FIELDS FROM `libraries`
Library Load (0.3ms)   SELECT * FROM `libraries` WHERE (`libraries`.`name` = 'org.thnetos') LIMIT 1

It should be loading the "lib"=>"org.thnetos/client", we need something to munge slashes in fully-qualified package names.

Either that, or I imported it improperly :)

Add special forms

Link to special form url in the docstring / source, similar to the output of (doc ):

user=> (doc if)
-------------------------
if
Special Form
  Please see http://clojure.org/special_forms#if
nil

I'll probably just add them to the database manually, maybe script it. There are only 16 or so.

Searching with hyphens doesn't seem to work

Searching for a string with hyphens doesn't seem to work, it returns empty results. Maybe the hyphen is being interpreted by the search engine instead of searching for the whole string? For example: as-file.

Inconsistency between clojure.core/let web page and comments API results

I do not know which is considered correct, but I wanted to point out the inconsistency in case it was a bug.

When I use the API for Clojure 1.2.0 or 1.3 to get the comments for clojure.core/let, I get the same 1 comment beginning "Note Bene".

http://api.clojuredocs.org/comments/1.2.0/clojure.core/let
http://api.clojuredocs.org/comments/clojure.core/let

I also see that comment when I go to the 1.2.0 version of the web page here:

http://clojuredocs.org/clojure_core/1.2.0/clojure.core/let

but it is not there when I go to the 1.3 version of the let web page:

http://clojuredocs.org/clojure_core/clojure.core/let

Which of the 1.3 versions, the API or the web page, is considered correct?

Create clean db rake task

Need a way to clean a database dump.

User table:

  • login
  • email
  • crypted_password
  • password_salt
  • persistence_token
  • current_login_ip
  • last_login_ip
  • openid_identifier

Add open search meta information to the site e.g. for mozilla search engine

Hi,

I think it would be a win for many users if you could include a autodiscovery link for a search plugin information.

I created a search engine information on mycroft http://mycroft.mozdev.org/search-engines.html?name=clojuredocs.org but direct integration would be nice I think.

Mycroft suggest to copy and modify the xml file in case of auto detection:
http://mycroft.mozdev.org/developer/hosting.html

Integration Example:
<meta>
...
<link rel="search" type="application/opensearchdescription+xml" title="ClojureDocs" href="http://mycroft.mozdev.org/installos.php.html/40721/clojuredocs_keywords.xml">

I figured the link tag would go into main.html.erb but I am not so sure about that and wanted to "ask" first.

https://github.com/zkim/clojuredocs/blob/14e927c74ff04429ce27610fac9897f65c94742c/app/views/layouts/main.html.erb

What do you think?

regards

Ben.

Search of "take-nth"

Hello,

The problem is that if you just search for "take-nth", the site doesn't find anything. However, the auto-complete works fine, and it's the only option to open this function's page.

Cheers,
Anton

Search should prioritize clojure.core results

When searching for a function such as when or case, the first result is not the one in clojure.core, but often a function of the same name found in clojure.contrib or other libraries.

Possible issue with gzip stream

I tried to read from the API with clj-http 0.2.1 and I got the following exception:

EOFException Unexpected end of ZLIB input stream  java.util.zip.InflaterInputStream.fill (InflaterInputStream.java:223)

This is with the examples API, but it seems to happen everywhere. This happens when Accept-Encoding contains gzip. Now, this could possibly be a clj-http problem, but given the simplicity of the way it handles gzip streams, I'm leaning towards it being a clojuredocs problem.

Can anybody with more knowledge of the internals of clojuredocs confirm whether or not this is a clojuredocs problem?

API for function page ?

I'm writing a mobile application for clojuredocs. So I just wonder is it possible to expose API for function includes docs, usages and even sources.

Thank you.

Index page redesign

The goal here is to get rid of the overwhelming wall of text currently shown on the main CD page by:

  • Moving 'New to ClojureDocs?' section to dedicated page.
  • Cleaning up 'Core Language' section
  • Moving 'Other Libs' section to dedicated page
  • Add 'New to Clojure? Start Here.' section w/ embedded 2 minute intro, links to 20 minute intro & add'l resources (cherry pick a few videos & in-depth overview links)

Search API mixes 1.2 and 1.3 results indistinguishably

Example of search API URL:

http://api.clojuredocs.org/search/pmap

Results:

[
{"id":1579,"name":"zipmap","ns":"clojure.core","url":"http://clojuredocs.org/v/1579"},
{"id":1952,"name":"pmap","ns":"clojure.core","url":"http://clojuredocs.org/v/1952"},
{"id":5659,"name":"pmap","ns":"clojure.core","url":"http://clojuredocs.org/v/5659"},
{"id":5854,"name":"zipmap","ns":"clojure.core","url":"http://clojuredocs.org/v/5854"}
]

One of the pmap results is the Clojure 1.2 page for pmap, and the other is for Clojure 1.3, but there is no way to tell from the search result which is which.

Would it be difficult to add something like a key :lib_version with value "1.2.0" or "1.3.0" in the maps of the search result, similar to how the examples API returns?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.