Everything you expect in a blogging platform, and more!
Home Page: https://blogthedata.com
License: MIT License
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
https://www.youtube.com/watch?v=vdFm0J-tizI&list=PLvJ_dXFSpd2tThwk2t057HdK9hMLKd5k8&index=8
https://www.youtube.com/watch?v=G_9-AkZch4k&list=PLvJ_dXFSpd2tThwk2t057HdK9hMLKd5k8&index=8
https://www.youtube.com/watch?v=JlXoMbBJF9s&list=PLvJ_dXFSpd2tThwk2t057HdK9hMLKd5k8&index=9
Also check out the Audit from AHref in my email.
A robots.txt file instructs crawlers on whether they are or are not allowed to crawl your site. It is also useful for giving sitemaps to the crawlers so they can better understand how the site is structured.
I should review some of these resources to learn about best practices and how others have implemented it.
https://developers.google.com/search/docs/advanced/robots/create-robots-txt
https://www.youtube.com/watch?v=abgxvrNqNkw
https://www.youtube.com/watch?v=xAXMqiPSY34
When people search for websites, the meta description is the text they see under the link. Also, SEO web crawlers use the meta description to learn more about a site. Each of my indexed pages should have a meta description.
Check out this post about it
https://stackoverflow.com/questions/66930647/can-you-add-html-meta-description-to-django-blog-post-model
I'd like to have the social media share buttons within the post-detail page. This would let viewers easily share posts to their chosen social media platform (LinkedIN, Facebook, etc)
Users must copy/paste the url into a social media post.
User clicks a share button corresponding to the social media platform they wish to share to.
This django library looks very promising. It includes:
'Tweet This'
'Share this on Facebook'
'Share on Google+'
'Share on LinkedIn'
'Share on Telegram'
'Share on WhatsApp'
'mailto://'.
'Save to Pinterest'
Currently, users have to pan their phone screen sideways when they want to see larger images. This looks weird and makes the layout inconsistent.
I like the way code snippets work. If the line is too long, a horizontal scrollbar appears and the user can scroll sideways without shifting the page around
I am not sure this is the same experience I'd want on devices with larger screens.
Currently, blogthedata.com gets pretty bad latency when accessed from other countries. I hosted the server in Dallas, Texas because I figured most of my traffic would be from the US and Texas is a pretty good midpoint in the country.
My understanding is that a CDN would distribute my site across many regions, moving the content closer to potential users. Here is a breakdown of the country IP addresses of people who have visited blogthedata.com
[show pie chart]
According to this Geekflare tool, users from countries such as Singapore can expect a time to first byte of about 1.5 seconds!
Server in Dallas, Texas serves all content to all users
Content can be served in at least a few continents (Maybe one for each major region?) NA, EMEA, LATAM, APAC
Since content is being distributed geographically, we can't guarantee each region will be synced up with the rest (unless we enforce that as a constraint). Seeing this is a blog site, I am personally OK, if there is lag time for all the servers to get the latest and greatest.
Not sure if it makes more sense to implement cache control BEFORE implementing this issue.
According to this Geekflare article, there are several CDNs to choose from, with Cloudfare being the most popular. Might want to do a little research to see what best works for me. Exciting to see that Cloudfare has a free tier!
I think this could have security implications, so adding the security label.
I also think this could have SEO impacts as it would definitley improve load time which is a metric for Google Lighthouse.
I've written articles on LinkedIN that I would like to include on blogthedata.com. The only issue is that I haven't get included the ability to upload images that are hosted on my server.
I really don't want to rely on external servers for images, so once that is implemented, transferring the articles shouldn't be too difficult.
A proper CI-CD workflow would be to run all unittests before code is committed. I've done something similar using Git Hooks.
This ensures that code is not merged without unit testing.
Tests are run manually via the VScode UI or running unittest via terminal.
Unittest coverage is assessed manually using Python's coverage module. It outputs this:
At a most basic level, unittests should be run whenever code is committed.
A second requirement could be to mandate that there is a certain code coverage before code is allowed to be committed.
Since I am using TDD now, enforcing code coverage shouldn't be a problem.
Might be able to leverage this approach:
https://www.codingforentrepreneurs.com/blog/django-github-actions
Other resources
https://www.youtube.com/watch?v=odUAJws0sPU
https://www.youtube.com/watch?v=qJPLFDtEi1I&t=4s
Users often want sites to display on Light/Dark mode based on their preferences. Not totally sure how this could be advertised, but this could be through:
It might make sense to keep the theme stored in a cookie if the user wants to get the theme regardless of their system setting. The initial implementation should just match the user's system's theme.
icons, favicons
Peoples of the internet sometimes like to donate to projects. I would like to give them that option!
I imagine a component in the sidebar that lets a user quickly donate.
Would also like to add this to the readme.
Whenever code is committed to source control that adds dependencies or other static conect (images, fonts, etc), automatically handle all the CI-CD stuff without manual intervention.
We must run:
python3 manage.py collectstatic to move assets into the right folders.
1 - Commit code
2 - Static files automatically move to correct folders (collectstatic command is run)
ERROR
Uncaught (in promise) DOMException: Failed to register a ServiceWorker for scope ('https://viewer.diagrams.net/') with script ('https://viewer.diagrams.net/service-worker.js'): The user denied permission to use Service Worker.
One of the performance opportunities put forth by Google Lighthouse was to implement Cache-Control for static assets.
Once implemented, I can get a performance score of 100 (And hopefully better performance).
No cache control
Static assets all have a cache control policy.
My understanding is that cache control will allow clients to hold on to static assets (images, fonts, etc) instead of having to keep fetching them. One danger is that they might not get up-to-date static content after an update...but it looks like this can be mitigated by using hashing on all the assets.
If the hashes don't match, the client will ask for the resource again!
Resources:
https://www.youtube.com/watch?v=TgZnpp5wJWU
https://jakearchibald.com/2016/caching-best-practices/
https://www.youtube.com/watch?v=ZDftZcgEEZo
https://www.youtube.com/watch?v=usN-tbrQynA&list=PLrFzFX2lgkuy79RYoQHekuqnHDRrkUgXx&index=13
This is blocked by #63
Some pages should have an automatic focus (Post Create, Comment Create, Edit Post, etc). When these pages initially load, the cursor should already be inside ckeditor.
This will break the UI tests
Blog posts are often shared to places like LinkedIN via 'posts.' Whenever I have done them, the post either has:
1 - No image
2 - My profile picture as the image
3 - The first picture embedded in a post
It would be great if authors could choose the photo that is featured in places like LinkedIN posts.
If you place an image very close to the top of your post, it appears to be used as the preview. For example, this post shares pretty well on LinkedIN.
I imagine an image field where you can choose what you want the preview to be. Once this is implemented, we can check the result using the LinkedIN post inspector which will show a preview of what your post will look like when it is published. There is also a facebook post inspector
I know little about social media sharing. WIll need to research this before attempting to implement. Check on this site on how they suggest doing it. This SO post agrees with the site.
It appears that I need to plug into the OpenGraph protocol.
In order to get the mimetype for the uploaded image, looks like I need to do some more work...like adding a validator
https://djangomango.com/blog/detail/checking-mime-types-for-uploaded-files-05a0d53e-4d5f-440f-a454-a5f5891adf01/
Also need to make sure GZIP compression is turned on
https://www.giftofspeed.com/gzip-test/
I recently discovered that Django template test coverage can be assessed with a plugin for the coverage module
Some coverage for templates.
Complete coverage for templates
I am noticing some pages don't look great in google. The home page, for example, is using meta description text that is from a blog post, not what I added to the metadesc for the home page.
I think this is because the text I put in was too short.
https://search.google.com/test/rich-results
https://developers.google.com/search/docs/advanced/sitemaps/overview
Could we also take a second look at sitemaps in this issue? I think we need to add category pages to the sitemap (/life_advice, /site_updates)
Most websites have a footer. This is an area at the bottom of the page that contains a bunch of information and links.
One reason I want a footer is that I need a place to reference some of the free resources and tools I've used to build the site. These open source contributors deserve recognition!
No footer = (
A footer that looks good in Desktop and Mobile
Not really sure how a footer is supposed to look on Mobile.
https://www.orbitmedia.com/blog/website-footer-design-best-practices/
Make sure to add these to the works cited page
https://favicon.io/emoji-favicons/bar-chart/
https://creazilla.com/nodes/55015-bar-chart-emoji-clipart
Django, bootstrap
Thank Corey Schafer and Code Academy
I noticed bootstrap has a way of adding counts to categories in list items.
Let's add these to our categories list
The Mozilla Observatory audit revealed that we have not implemented SRI.
Subresource integrity is a recent W3C standard that protects against attackers modifying the contents of JavaScript libraries hosted on content delivery networks (CDNs) in order to create vulnerabilities in all websites that make use of that hosted library.
Source: Mozilla infosec
The crux of SRI is generating a unique cryptographic hash for each file which locks the resource into a specific point in time.
There appears to be a python module, django-sri, that accomplishes this. Should look into it and see how easy it is to implement.
I often want to write posts about Health and Fitness, but I don't have a category for it! This should be pretty easy to add!
Whenever images are uploaded when creating a post or uploading a profile picture, they are not cleaned up when a post is deleted or a user is deleted.
Image resources are uploaded in the following ways
These all stick around forever
Image resources should be deleted when
I'll have to come up with a strategy for deleting existing resources that are no longer tied to a user or post. Might be able to use this approach.
When people visit a category (or view all posts)...they might want to sort the posts based on views/likes so they can see the most popular content first. They also might want to see oldest/newest posts.
There's no way to do this now. Posts are automatically sorted by timestamp (latest first). There is no way to change this.
Sort functionality toward the top of the page. Should include:
1 - Sort by time (Descending/ Ascending)
2 - Sort by Likes (High/Low)
3 - Sort by Views (High/Low)
4 - Sort by Tag? ...there is a discussion about this in #2
Should be pretty straightforward. I imagine this would be done by leveraging the get_queryset( ) function to fetch sorted/filtered posts. This is how I did it for the CategoryVIew
def get_queryset(self):
cat = self.kwargs.get("cat").replace("-", " ")
posts = Post.objects.active()
if self.request.user.is_staff or self.request.user.is_superuser:
posts = Post.objects.all()
return posts.filter(category=cat)One of the biggest hits I am taking with Mozilla's Observatory tool is that blogthedata.com does not have a CSP.
The Mozilla Foundation has created a Django plugin called django-csp. After installing it, I can use these guides to create a CSP.
https://www.geeksforgeeks.org/adding-csp-headers-in-django-project/
https://dev.to/thepylot/django-web-security-checklist-before-deployment-secure-your-django-app-4jb8
I've done some initial testing and have found that adding some of the reccomendations (like removing inline styles) completely breaks the app's UI.
Will need to create a branch for this one and work on it incrementally.
Google Lighthosue is reporting that Mailchimp's js file is not compressed. This negatively affects performance.
I found this guide which involves hosting the JavaScript locally and compressing it myself. I am a bit wary of this because it means I have to make sure it gets updated whenever mailchimp updates their JS.
Would be great if Mailchimp could provide a minified version of their JS CDN link!
I seem to have made a mistake furnishing the publish date to using the OpenGraph protocol. Need to investigate to find out why it's not being picked up.
When using the LinkedIN post inspector with this link:
https://blogthedata.com/post/integration-tests-are-a-scam-i-wrote-them-anyway/
It says that it cannot be found.
This is how I am encoding the publish date:
<meta property="article:published_time" content="{{ post.date_posted|date:'c' }}">
We all know about those 'accept cookies' banners that show up on sites these days. I believe this is due to the EU's GDPR laws. It would be great if blogthedata.com conformed to these rules.
I'm not really tracking anyone aside from logging uniue IP addresses to do view/like counts on posts. Will need to research what is needed for a basic site like mine.
My username is displayed on posts. It would be nice if the underscore could be removed cause it looks weird.
A space should replace the underscore
I don't want to change the username because I don't think it's allowed to have spaces, but there's probably a way to do this in the template. If not, we can do something like we do to slugify/unslugify urls and categories
context["cat"] = Category.objects.get(name=self.kwargs["cat"].replace("-", " "))
Whenever a user likes a post, a page reload happens. Essentially, this line of code is fetching the post-detail page again so the user can see that the post either went up 1 or down 1.
return HttpResponseRedirect(reverse("post-detail", args=[str(slug)]))Would be great if the page didn't re-load.
1 - User Likes/Unlikes a Post
2 - Page re-loads
3 - Like count is correct, Like button state is correct
This isn't so noticeable on Desktop, but it's really wonky for mobile users. After liking a post, it refreshes the screen and you're scrolled back up to the top of the post-detail page.
The only thing that needs to be refreshed is the Like button and the number of likes.
1 - User Likes/Unlikes a post
2 - Like count updates and Like button state changes WITHOUT reload
I believe there's two ways I could solve it. The first is to use a library called django-htmx which I understand to allow me to make AJAX calls without actually writing any Javascript.
The second approach would be to write async JS to pull this off.
I will need to research these two approaches to figure out what is the best choice.
It would be nice if there was some kind of UML application diagram for how blogthedata.com is structured in terms of classes/functions/etc
None
An editable diagram that can be easily shared.
Might want to use Draw.io ?
One goal of this site is to teach me web development frameworks. Now it's entirely possible to write the whole app without a single line of JavaScript, but that's silly if I want to get into the Web Dev world.
Thus, a big step for blogthedata.com is to separate the front and backend. There are plenty of tutorials online about 'reactifying' a Django app.
I imagine a React.js app that controls what we show to the user. When the app needs to interact with a Database, it should make REST calls to Django which handles the request, makes any needed db changes, and then responds with what React needs.
There will need to be a lot of architecture changes. We should finish the architecture diagram before taking on this issue.
Might want to pick up this book.
Django 4.0 came out this past fall and it may be nice to upgrade it to the newest version.
Upgrade Docs
Depreciation Timeline
One benefit is that 4.0 implements a setting for Cross-Origin Opener Policy
One of the modules we use, django-admin-honeypot, seems to have issues with django 4.0 ...There's a sketchy unmerged PR about fixing errors in 4.0.
I also notice that HTMX is not supported on 4.0
https://django-htmx.readthedocs.io/en/latest/installation.html
A CI workflow was created in this issue. Now it's time we moved toward CD (Continuous Deployment) where production is updated without manual intervention.
I think the best way to do this would be to create a release branch and use a methodology such as Git Flow. to manage the SDLC.
Once code is pushed into the release branch, production should be updated within a reasonable ammount of time (a few minutes?)
We are already collecting static files using Git Hooks which was implemented in this issue. We will probably want to wrap that work into this workflow.
Might want to do more research on modern ways to manage branches.
I think we are going to have to move off of Linode node and move to something that has a starter workflow for GitActions if I am still using GitActions at the time this is implemented. My options today are Alibabacloud, AWS, Azure, Google, IBM, Openshift, Tencent, and Terraform.
Adding Security and performance labels because a whole new infrastructure requires new consideration of these factors.
Might want to check out a tool called Dokku
In this issue, I temporarily fixed password reset emails. Unfortunately, this fix will stop working May 2022.
Google changed the rules on the 'less secure' method of authenticating via username and password.
The setting 'allow less secure apps' automatically turns itself off if it's not actively being used. So if no one tries to reset their password, I assume the password reset with break again.
Google is planning to shut down my 'less secure' way of authenticating with the Gmail API in May 2022, so we need to switch to a more secure way of authenticating.
Uses 'less secure' API call using username/passsword
Oauth
Ideally, the password reset emails should be using my blogthedata.com domain, but I am not sure how to pull that off yet. I think it makes sense to implement using Google and then switch over to the blogthedata domain when the time is right.
I think I can use this approach.
https://www.youtube.com/watch?v=7X3fBlMw_1k
https://www.youtube.com/watch?v=44ERDGa9Dr4
We need a way for authors to know when they are over the character limit for a field.
Each field has a character limit.
title = models.CharField(max_length=60)
snippet = RichTextField(max_length=300, blank=True, null=True)
metadesc = models.CharField(max_length=140, blank=True, null=True)
category = models.CharField(max_length=100, default='uncategorized')
These are all based on SEO best practices.
Unfortunately, there's no way for authors to know how many characters they have typed until attempting to save the post.
When attempting to save a post where one of the field character limits has been exceeded, you get an error.
This error shows if you click 'Post'
Would be great to have a running counter that keeps track of every character (spaces, linebreaks, html, etc). That way, the author knows when they are approaching or have exceeded the limit without needing to attempt a save.
One solution would be to just stop the text area from adding characters above the limit. I don't like this option because I often want to finish my sentence or copy/paste text in and then reduce it down with edits.
I am currently manually adding '...' to snippets.
'...' is added without any manual intervention
I've already added them manually, so I probably need to manually take them out, too, so I don't have six periods. I could script it, but there's only like 30 posts, so it shouldn't be a huge pain.
Currently, there are two folders with static files /static and /staticfiles. Both appear to have the same files in them. My understanding is that my local dev environment leverages /staticfiles whereas production is using /static.
When I've tried to remove one or the other, it will break Dev or Prod.
# Settings.py
STATIC_ROOT = os.path.join(BASE_DIR, "staticfiles")
STATIC_URL = "/static/"
# Extra places for collectstatic to find static files.
STATICFILES_DIRS = [
os.path.join(BASE_DIR, "static"),
]There shouldn't be duplicate files in source control...nor should there be local copies that could get out of sync.
We will want to take care of this issue before implementing #19
I can't just embed Github into my site because of the X-frame restrictions....but I can use their content API to pull issues into the page.
See
https://docs.github.com/en/rest/reference/issues
Once I pull issues in, I can put the closed ones under a 'Completed' section and the open ones under 'TODO'....might need to create more labels and/or a project to further organize and display issues.
A user might want his post to be automatically saved (at some interval) without any manual intervention. This prevents any content from being lost should the page accidentally be closed, hard drive failure, nuclear explosion.
I can save drafts by ticking the draft checkbox and submitting the post. You then need to re-edit the post to continue where you left off.
This isn't a great experience because I've lost progress from forgetting to periodically do this. Plus it's annoying as hell.
I think it makes sense to save a draft or check for changes every 15 seconds.
I might want to look into other things that could improve this experience (like preventing the page from being navigated away from)
I am also unsure of what happens if someone leaves their browser open for years and years on end. Should there be some kind of timeout? Maybe I need to implement a sessiontimeout and automatic logout so my server doesn't get pinged every 15 seconds until the end of time.
If the author wants to discard their changes, they should be able to do that.
https://classroom.udacity.com/courses/ud884
https://openfolder.sh/django-faster-speed-tutorial
https://www.honeybadger.io/blog/caching-in-django/
I should check out some performance profiling tools
According to the Godaddy cert checker, my site doesn't implement this. According to digicert,
OCSP stapling can be used to enhance the OCSP protocol by letting the webhosting site be more proactive in improving the client (browsing) experience. OCSP stapling allows the certificate presenter (i.e. web server) to query the OCSP responder directly and then cache the response. This securely cached response is then delivered with the TLS/SSL handshake via the Certificate Status Request extension response, ensuring that the browser gets the same response performance for the certificate status as it does for the website content.
Digicert has a guide on how to do this in Apache
https://www.digicert.com/kb/ssl-support/apache-enable-ocsp-stapling-on-server.htm
There is also this cool tool put out by Mozilla that appears to add OCSP stapling to an apache config boilerplate
https://ssl-config.mozilla.org/#server=apache&version=2.4.41&config=intermediate&openssl=1.1.1k&guideline=5.6
After making some layout changes, we now have a pretty high CLS according to Google Lighthouse
The biggest issue is that the body is shifting a lot.
Will need to learn more about how bootstrap renders elements to debug this one
In addition to a robot.txt, there appears to also be a security.txt file that has something to do with reporting security vulnerabilities. It is documented here:
https://securitytxt.org/
Will need to look into this more to see if it makes sense to implement!
I like the idea of unit tests doing more 'checking' then testing. There's really no need to involve the database and external servers when stubs and mocks will do.
This will also greatly increase the speed of the tests and hopefully improve my software design.
Unit tests are more 'classisit' where they make real connections to servers and databases to get results.
unit tests are completely disconnected from databases and servers which will make them more 'unit testy' as opposed to being more 'integraty'
Probably want to wait on this issue #20 to be implemented before taking this on.
https://pypi.org/project/django-webtest/
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.