3b1b / 3blue1brown.com Goto Github PK
View Code? Open in Web Editor NEWHome Page: https://www.3blue1brown.com
License: Other
Home Page: https://www.3blue1brown.com
License: Other
Now that our repository structure has settled down somewhat and #47 is almost done, I think it makes sense to purge our git history of big files. This will drastically reduce the overall size of the repo (currently ~42MB download).
Full admission, I also accidentally committed some large files to the repo when working with Git LFS, but it probably makes sense to do this anyways.
I used this script to generate a list of objects in the repo's history. For example, here are the largest 10 files in the history. Note, this list contains duplicate entries (same object id) when the file path was changed for an object.
100644 blob 831ebc200f6db517c090eccc4deada769ee08813 9882127 static/images/special/pi-fourier.mp4
100644 blob 831ebc200f6db517c090eccc4deada769ee08813 9882127 images/special/pi-fourier.mp4
100644 blob 0eb0c85d73c59292653314f82fd7802c4b5fc477 5051228 content/lessons/2021/git-lfs-test/animation-10.00-10.39.mp4
100644 blob 86983c9d0941a85963f6c2b523ad892eee059631 956891 static/images/store/short-3blue1brown-pi-plushie.png
100644 blob 86983c9d0941a85963f6c2b523ad892eee059631 956891 images/store/short-3blue1brown-pi-plushie.png
100644 blob dccdfc3f20f88b865172603c4aa69dc3e1dce540 804017 static/images/store/tall-3blue1brown-pi-plushie.png
100644 blob dccdfc3f20f88b865172603c4aa69dc3e1dce540 804017 images/store/tall-3blue1brown-pi-plushie.png
100644 blob c8268e87c9dfd5c891a42b946616fdb79cf58bc7 735316 static/images/store/fractal-curve-poster-set.png
100644 blob c8268e87c9dfd5c891a42b946616fdb79cf58bc7 735316 images/store/fractal-curve-poster-set.png
100644 blob 704c663661314540075925125d090f36058c5e08 733084 static/images/topics/differential-equations.jpg
See the full list here.
I've used the BFG Repo Cleaner in the past to programmatically purge large files from a repo. Here are the steps for installing and using BFG.
Start by installing bfg
.
brew install bfg
Clone a fresh copy of your repo, using the --mirror
flag:
git clone --mirror https://github.com/3b1b/3Blue1Brown.com
Identify a size threshold to use when purging large files. For example, purging files above 1M would help quite a bit. @vincerubinetti should we use Git LFS to manage the images in the static
directory?
bfg --strip-blobs-bigger-than 1M 3Blue1Brown.com.git
I think we can be pretty aggressive with the threshold, because BFG treats the files in the current commit as "sacred", meaning that it should only clean files from old commits. For example, running the command with a 1M threshold currently deletes the three files below:
Filename Git id
---------------------------------------------
animation-10.00-10.39.mp4 | 0eb0c85d (4.8 MB)
pi-fourier.gif | ef6a1192 (5.7 MB)
pi-fourier.mp4 | 831ebc20 (9.4 MB)
Then, update the history for the mirrored repository.
cd 3Blue1Brown.com.git
git reflog expire --expire=now --all && git gc --prune=now --aggressive
Check that the site still builds, the size and history of the mirrored repository. I think you could also compare the new list of objects sorted by size.
If this all looks good, push the changes.
git push
Afterwards, it is recommended that people ditch their old copies of the repo and make fresh clones.
You can also remove objects by its id. For example, the pi-fourier.mp4
file has an object id of 831ebc200f6db517c090eccc4deada769ee08813
bfg --strip-blobs-with-ids <blob-ids-file>
A simple and really destructive method to make the repo is to create a new branch with an unrelated history and then delete the old branch.
This guide has the steps for setting up Linode object storage.
If you would like me to setup the bucket I'll need an login with a payment method. Otherwise, I think the general steps are:
Some thoughts on the table of contents. These are mostly (entirely?) shamelessly lifted from the design of the sidebar on Wikiwand.
maxresdefault.jpg
instead of hqdefault.jpg
I'm Just leaving a note here for what content would be referenced on the extras page. Completely open to suggestions on the best design surrounding it.
Stuff to put on extras page:
Put list of Patreon supporters at bottom of each lesson. Those who contributed between a month leading up to the lesson's release?
Maybe pull from Patreon API? Or perhaps more simply, just make _data/patrons.yaml
like:
# a supporter
- name: Adrienne Wilson
start: 2020-10-24
end: 2018-08-10
star: true
# another supporter
...
Maybe have a special
field for people who have supported > 1 year or high donation amounts, and highlight them in bold or gold or something?
Don't autoplay video until switch to that tab. Pause video when switching back to image tab.
This is vague and will encompass many things, but it's worth recording some thoughts on the best way to include translated content into the site. Honestly, this is more about managing the 3blue1brown YT channel(s) than the website development, but there is some crossover.
Add sad pi with heart broken. Maybe something else fun.
Include site-wide search when we have that ready per #24
The about page should have some mention of each contributor to the site, perhaps with links to their preferred pages.
Choose safe and reliable comment service, such as Disqus or some other service. Then paste its embed code into the comments.html
include.
Probably want to wait until the site is live to do this so we don't get comments on the in progress site, and because the page urls (which the services hook into) will change (e.g. https://vincerubinetti.github.io/3Blue1Brown.com/lessons/test-lesson
-> https://3blue1brown.com/lessons/test-lesson
).
Some kind of automatic numbering and id'ing of figures, equations, and more, similar to Manubot.
Cursory thought on implementation: have the appropriate components be able to accept an id
field, like {% include figure.html id="fft-derivation" image="fft.png" %}
, then you can link to it elsewhere in the doc simply like [Link Text](#fft-derivation)
.
Then we can have a javascript plugin or ruby plugin replace "Link Text" with automatic numbering. May need prefixes somewhere like fig:
or eq:
to distinguish between the types.
Such a long word, really takes up a lot of space in the header. Perhaps put it on the "Extras" page? Or have it as a separate page, but don't link to it in the header, but instead link to it at the top of Extras or About with a big button?
Unless I am mistaken, the figure caption remains the same for both the Still and Animation tabs, but it makes sense to have a separate caption for certain cases where the still needs a bit more explanation.
Currently the 404 page has a link to the search page, but the link is https://3blue1brown.netlify.app/search%20%7c%20relURL%20, which serves another 404 page because of the extra characters at the end of the search URL.
I'm just recording a few small notes here so I don't forget.
Some method (probably another component) to put citations into lessons. Cursory thought: we have a site-wide citations.yaml
in /_data
that can be populated like:
- id: doi:10.10101
title: How to add numbers
author: Isaac Newton
publisher: Me
year: 2030
...
and displayed like {% include components/citation.html id="doi:10.10101" %}
, and we make the citation component look how we want and have the fields we want.
This component will probably need to integrate with the "aside" component in #19 . Think holistically here.
Make the credits at the top of each post clickable. Link to bio pages for each author. Make new _members
collection with bios like grant-sanderson.md
, james-schloss.md
, etc.
Automatically find corresponding bio in post layout.
Right now, the site deploys to GitHub Pages automatically, with no extra configuration. Downside is GitHub Pages significantly restricts what Jekyll functionalities you can use, for security reasons.
We will likely want some Jekyll plugins that gh pages doesn't support, like last-modified-at (automatically assigns a last-modified date to each page) and paginate-v2 (allows a statically generated page for each tag).
All this will cost us is: 1) the repo will have a separate gh-pages
branch in addition to main
which holds the built version of the site and 2) we'll need a small yaml file to config GitHub actions to build the site and push the result to the gh-pages
branch every time there's a push to main
.
Normally GitHub does these things by default, behind the scenes, hidden. Originally, I wanted to keep things as clean as possible, but these things are easy to do and add more functionality. A side benefit is we'll also get better build logging.
Today we were talking about the value of the pi characters for expressing emotion.
It would be really great to have one or more components that enable us to use the pi characters to communicate emotion and whimsy. In particular, we discussed two ways the pi characters might be used:
Any component for these characters should definitely make it possible to specify the emotion of the pi character (as well as the associated text).
We have chips for chapter coming in the next PR. It'd be good to have chips for other important things like, "this lesson has a text version ready", and less importantly, "this lesson has interactive elements".
Make sure lessons can be printed cleanly, e.g. with only still images, no interactive things like tooltips, even margins, etc. Do this at end when we have all the components we know we'll have.
In JS + SVG for clean res
From a previous email:
What are your thoughts on populating the home page with a gallery of thumbnails, something with minimal text and just imagery? One could toggle between seeing them organized by category or just listed by date created, and upon clicking a thumbnail it expands to show the title and a short description.
We could add a dropdown/toggle for this. But perhaps let's consider this an enhancement, since 1) we already have the latest video above the topics and that's probably what most people are looking for in terms of chronology? 2) the topics are probably a better jumping off point than date? 3) YouTube already has a functionality to list things by upload date.
Also, perhaps each card should have two links on hover: a link to the video on youtube and a link to the written blog post?
Also perhaps simplify the Follow and Support sections? Less text, maybe collapse them into a single, two-column section below the topics section? Ultra-minimal.
GitHub pages is a free web hosting service that can host static things like images, html websites, etc. That's (so far) what we're using to host this website itself and its images/assets.
But we'll need more robust (paid) website hosting for other purposes:
Homework for Grant:
Find a website hosting service. I use Inmotion Hosting. Run your choices by me to make sure it has the functionality we'll need, but most popular services should be fine.
I'm opening this issue about error reporting in Hugo with the goal to prevent silent failures and generate more meaningful errors for authors and developers while working with Hugo. This can be pretty subtle when managing the dot context, but when setup correctly it's no big deal and will make our lives better.
TLDR: Give more useful errors to content authors and prevent silent failures when content changes.
As a general example, this example figure shortcode matches a page resource (image) based on a path supplied by the user. For example, the user might write the shortcode shown below in their markdown file.
{{ figure image="example.png" }}`
The shortcode template looks like this:
{{ with .Page.Resources.GetMatch $path }}
{{/* TODO: render here */}}
{{ else }}
{{ errorf (printf "%s: Unable to find resource given the path:\"%s\"" .Position $path) }}
{{ end }}
Then, when the author types a bad file path they'll get a shorter, better error. Specifically, the line number where the failure occurred and the input that caused the error.
ERROR 2021/05/25 10:27:29 "~/Sites/3Blue1Brown.com/content/lessons/2021-05-22/index.md:12:1": Unable to find resource given the path:"bad-file-path.png"
There are a number of other considerations for the actual figure shortcode. For example, what happens if the "image" or "video" attribute is misspelled or not present.
In addition to making the error messages more helpful, managing errors and warnings ourselves gives us the added benefit of preventing silent failures when editing the site. For example, the lesson gallery shortcode is a currently a good example, because if you change data in the data/featured.yaml
file or, just as likely, change the path of a lesson in the content
directory, the current behavior is silent failure - the bad id fails silently and no lesson card is rendered.
Currently on line 34-36 in layouts/partials/lesson-gallery.html
the lesson card partial is used. If a bad id is passed to the partial, this renders nothing when it probably should report an error.
{{ range $featured := site.Data.featured }}
{{ partial "lesson-card" (dict "id" .) }}
{{ end }}
An error can be thrown in the partial, and probably should throw one if the partial expects and string id and calls GetPage
. However, I would recommend that we provide a more better error message here and instead pass the lesson page context and additional values to the partial.
{{ range site.Data.featured }}
{{ with site.GetPage . }}
{{ partial "lesson-card" (dict "context" .) }}
{{ else }}
{{ errorf (printf "data/featured.yaml: Unable to find the lesson associated with the id:%s" . )}}
{{ end }}
{{ end }}
The same applies for the data/topics.yaml
file. For example, before the last commit, adding this check to lines 43-45 in layouts/partials/lesson-gallery.html
reports the three errors shown below while testing.
ERROR 2021/05/25 13:57:42 data/topics.yaml: Unable to find the lesson associated with the id: "quick-eigen"
ERROR 2021/05/25 13:57:42 data/topics.yaml: Unable to find the lesson associated with the id: "qa1"
ERROR 2021/05/25 13:57:42 data/topics.yaml: Unable to find the lesson associated with the id: "qa2"
In general, I agree with Vince that first priority is errors that content authors run into, usually associated with shortcodes, data files and adding/editing lessons in the content
directory. If we run into an obscure error in the future, we can always review it and see if we can provide a better one.
Maybe incorporate a table of contents plugin -- for the page, not for the site-wide lessons and topics -- somehow into the side? Perhaps useful for longer lessons where jumping between sections would require a lot of scrolling. Along with this, a floating jump-to-top button?
Need some kind of footnote or aside component. There are several ways to do this.
One is by having a tooltip, but this is difficult and a bit hacky to do. Another is by having an accordion the reader can expand/collapse, but this, design wise, only makes sense for non-mid-sentence asides (it'd have to go after a full paragraph).
A perhaps temporary solution we came up with is: have a superscript icon or number that when clicked on expands a hidden paragraph below the current paragraph with rich content.
Need to experiment with this to figure out what the best doable solution is.
If we change the content width on the site to be a number with more factors than 1000
, such as 960
or 1080
, then it will make designing grid layouts and custom illustrations a little easier and IMO more pleasing. Namely, 1000
is missing the factor of 3
.
Note: This is pretty nit-picky since 1000px is perfectly fine.
To illustrate why this might be nice, consider that 960
can be evenly divided by common numbers and the results also have a lot of factors too.
960/2 = 480
960/3 = 320
960/4 = 240
960/5 = 192
960/6 = 160
For completeness, shown below are the prime factorizations of the three numbers 960, 1000 and 1080.
960=2*2*2*2*2*2*3*5
1000=2*2*2*5*5*5
1080=2*2*2*3*3*3*5
Add a short text description to each of the lesson topics (Calculus, Linear Algebra, etc.), which will become visible upon expanding a given topic.
Splitting this out from #4 into it's own separate issue. Maybe have a button to toggle between showing the topic cards, and just a chronological listing of lesson cards. Wouldn't actually be too hard to do.
Link to another lesson, with tooltip on hover that shows description or other metadata. Separating this from the basic components in #5 because I think the tooltip should/will look similar to what we do for the topic expansions on the homepage, which I will be doing next in a separate PR.
Right now we're leaving it up to the author to size each image/video manually, eg. width: 60%
or height: 300px
. Maybe, to enforce consistency, we can figure out some algorithm that automatically sizes it such that it always looks good and consistent. We could always keep the manual sizing there for special cases, I suppose.
@3b1b thoughts?
Also, for the sake of detailed figures, we could have the image/video pop out and become full screen on click. Though I don't recommend making a figure so detailed that this is absolutely necessary, because if it has that much fine detail, it won't be legible on mobile even when full screen. Though, still a nice option to have. Browsers already have built-in controls for fullscreening videos so we'd only need to implement it for images.
https://gohugo.io/hosting-and-deployment/
@kurtbruns , would be good if you could list some bullet points of things you like about it. I've personally gotten frustrated with Jekyll more than once, I just chose it because I thought it would be easier for Grant to maintain in the future.
Hugo does seem to have a "jekyll import" function that hopefully works nicely.
<iframe>
embedSomehow normalize.css
is being included in the GitHub Pages build, even though it is no where in my repo. It's screwing up some header and footer styles.
Some groups of topics (linear algebra, calculus, differential equations, neural networks) are meant to be a contiguous series, whereas all the others are more just collections of topics with similar topics/genres. It might make sense for the design under the "by topics" tab to reflect this.
Then, for those groups which are meant to be series, it might make sense to display the chapter numbers with a separate design around their titles.
Don't show correct answer when selecting wrong one. Show reset button after submitting.
Our workflow should be robust, easy to us and leverage the benefits of git
and Github.
I.e. an author in a local environment should be able to view and make changes without worrying about the live site or data constraints. Then, when it comes time to push changes to the live site, they are able to publish changes quickly and confidently through a pull request. They should also have the ability to roll back to a previous commit.
Currently we use Github Actions to publish the live site to Github Pages. We also use Vercel to preview the site via a pull request. This workflow is simple, easy to use and well integrated into Github.
Due to high number of expected users, we plan to use store large static assets such as images and animations (.mp4, .mov, .png) on a Cloud Object Storage service. The proposed workflows below are suggestions for how we can extend our current workflow to use buckets.
This workflow uses two buckets called development
and production
.
While editing content, authors interact with the development
bucket to add/edit/delete files. This would be done through the Linode CLI or Cyberduck. I tested this workflow using Cyberduck and it is easy to setup and use. We would mirror the _posts
directory structure in the bucket.
To push changes to the live site, a pull request is created that generates a preview of the site with assets that use the development
bucket. Then, either through manual action, a script or github action, the production
bucket is synced with the development
bucket and the pull request is merged.
Considerations:
A solution for the two considerations above is to take a snapshot of the production
bucket before a pull request is merged. This snapshot would be saved with the commit hash and would allow us to recover/roll the live site back to a previous commit.
This workflow has authors edit and commit large assets into VCS. Github actions would be used orchestrate local, staging and production environments using feature branches, a development
branch and a main
branch.
The author would edit local markdown files and assets that live next to each other.
example-post/
├── YYYY-MM-DD.md
├── animation-1.mp4
├── animation-2.mp4
├── figure-1.png
├── figure-2.png
├── ...
When the author is satisfied with their changes they push to the development
branch. This creates a development site available at a public URL. If the author wants to roll back changes, they can revert to a specific commit and push to the development
branch again.
Once changes have been staged in the development branch, a pull request for the main
branch is created. Once accepted, this applies the changes in the development branch to the live site. This is the same workflow as pushing changes to the development
branch.
This workflow uses git large file storage in combination with Linode Object Storage (see #38) to manage the static files in two buckets: a development bucket and a production bucket. When pushing changes to the development
or main
branch a git action is run that syncs the bucket with the current commit.
See s3 sync action as an example.
The local site would serve local assets while authoring content using something like Jekyll Environments. These assets would be published in the build directory for the local
, but not the development
or production
environments. I know this is possible in Hugo #34 , but may not be with Jekyll.
As per #60, and also #58, the question component should have an explanation, and this explanation should have some styling that makes it stand out clearly from the article. I'm guessing lower width and clearly distinct background color?
The explanation is likely to be larger than the question, and if we have to choose which one gets full markdown, it should be the explanation.
Separate question: Should "show explanation" be a separate operation from "check answer"?
Dear Grant, even though I do not have the internship opportunity, could I still make some contributions on this project? Thank you!
Pi creatures get covered up by video on small screens. Just remove.
Also per meeting on 5/9 we are getting rid of spotlight all together? Are we sure? Then there will be nothing directly watchable on the homepage.
Method to search site for content. Related: greenelab/lab-website-template#2
Probably just want to opt for an embedded google search. Grant will need to register/create a search console that we can paste into the template somewhere.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.