opsschool / curriculum

Ops School Curriculum

Ops School Curriculum

The current documentation based on these sources can be seen at: http://www.opsschool.org/

Welcome!

If you have arrived here, you are probably interested in helping out. So thank you for your time.

Things you should know:

• This project is written in reStructuredText
• Hosted by Read the Docs
• Tested by rendering in Sphinx on Travis CI

This is the only Markdown file in the repository, as it's not meant to be included in the documentation itself.

If you are looking to add content, fix formatting, syntax, typos or other wonderful things, please follow this process:

• Read the Style Guide
• Install Sphinx: pip install -r requirements.txt (This is best done in a virtual environment.)
• Fork the opsschool/curriculum repository to your own account
• Check out a branch to make your changes on: git checkout --branch <my_topic>
• Execute make html to build the docs in to _build/
• Make your changes
• Execute make html again and verify your changes don't cause any warnings/errors
• Commit with a descriptive message, and submit a pull request from your branch to master
• One of the editors will review the change, and either merge it or provide some feedback. Community review is also encouraged.

If you submit a pull request and would like to have your name associated with the project, add it to the meta/contributions.rst file!

Some cool things:

• vim-common contains a reStructuredText syntax highlighter
• The Emacs support via rst-mode comes as part of the docutils package under /docutils/tools/editors/emacs/rst.el

Note about internal links

When creating a link to another page inside the curriculum, use Sphinx's built-in cross-reference mechanisms to achieve this.

Examples:

• linking to a top-level document such as "Text Editing 101" should be done inline, like so:

  See :doc:`text_editing_101` for details...
  

  where the name provided is the RST file path. The title will be linked to the document like so:

  See <em>Text Editing 101</em> for details...
  

• linking to a reference point, such as a sub-topic on a page should be done by creating a globally unique reference point, such as:

  .. _gnu-screen:
  
  GNU Screen
  ----------
  

  and using this inline anywhere in the docs like so:

  See :ref:`gnu-screen` for more details.
  

  The sub-topic's title will be replaced in the hyperlink as well.

After modifications to links, please run make linkcheck to see the status of all the links in the docs.

More on cross-referencing can be read in the Sphinx Docs.

curriculum's Issues

See about some automated testing via Travis for pull requests

As references in #15, having some automated testing around the code we write can prove valuable.
Even more so when some automated system can di it for you.

CDN content?

I'm happy to write about how content delivery networks work, probably for a 101 piece to begin with to at least get something written. I can't see anywhere in the repo right now where it would naturally fit; anyone have any thoughts on that?

(I do see some reference to Varnish, which is cool, but that's a caching layer and although central to at least one CDN, is a different topic to write about).

in shell_tools_101.rst, the section on cat seems to encourage UUoC

• The section covering cat seems to indirectly encourage a Useless Use of Cat

SMTP section - RFC on moving sendmail to 201

I'd like to suggest moving sendmail to 201 - it's overly complex for the "junior" sysadmin by modern standards, and horribly inappropriate for most modern mail configurations due to it's complexity, security history, and large amount of poorly understood legacy features for mail setups that largely don't exist anymore, like uucp routing.

With that done, I'd like to either put exim in its place, or leave postfix alone in 101.

Thoughts?

Coverage of other systems in addition to Linux

I work in a very heterogeneous environment--some clients are strictly Linux, others a hodgepodge; the majority of our internal machines run our illumos derivative, though our firewalls and OpenVPN boxes run OpenBSD; the list goes on. What sort of plans are in place for covering other systems alongside Linux? I'm fine with doing the documentation for these, but I'd like to know the how so I can get started with the what.

Cover sound programming practices

This is more than just scripting/hacking to get a solution done. There's a difference in mindset and goals between writing code to automate something quickly and writing code for the business's main application.

make releases and keep a changelog

I love this project. Unfortunately, i don't have time to review every commit and pull request that is merged in, even on a monthly basis.

Instead, I would like if actual tags (ie. also tarball releases!) would be laid down, with a descriptions of broad changes that have been factored into those releases.

I would of course suggest semantic versionning for the versionning scheme, and would suggest you start with a 1.0.0 release. :)

Thanks!

Section on Testing?

Looking over the curriculum, there doesn't seem to be a Chapter on testing. Is this something to explore? I'd be happy to help.

Create some sort of blog

Make it easy to fork & fix, publish via github pages

Is this project dead?

I am just wondering if we should expect updates to this nice project or if it is dead.

ctime should be change time

In chapter File System, paragraph "File System Options", the explain of ctime should be change time, but not create time.

tcpdump

Where tcpdump is mentioned, it might be good to link to a helpful blog post such as this one

undocumented TODOs in package management 101

Hi,
The document says:
"Creating packages
There are two todos here."
but I couldn't figure out what the two todos are from the to do list.
http://www.opsschool.org/en/latest/package_management_101.html
http://www.opsschool.org/en/latest/index.html?highlight=todo

Add SaltStack to Configuration Management

Hi guys,
I don't know whats the procedure (no readme or something...) when it comes to suggestions so that's why this question.

How do you feel about adding @saltstack to the list of configuration management tools?

@thatch45 already wrote plenty of documentation and if needed I could share a bunch of state files that can be used as a real world examples.

Thanks in advance.

TODO List - issues and best practices

It occurs to me that it may be useful to "mirror" the TODO List in github issues and introduce it as a contributing note. This will prevent contributors from working on the same TODO item, at the same time.

2 contributor submissions on 2 TODO List items is probably better than 2 contributions on 1 TODO List items. At the moment to ensure that the TODO List is achieving the best possible MTR.

So for each item added to the TODO List a corresponding github issue would be raised for the item. When someone wants to pickup a TODO they check the TODO issue on github and if not active make a comment saying "In progress ..."

I am happy to pick that up and do it if that makes sense, just update here.

I am going to put in into practice myself now :)

Style Guide

So, one thing i keep thinking of - I never know whether to write operations professional or system administrator. I mean, they're realistically interchangeable, but, i think having some sort of "hey, here are some conventions" are a thing that will make this work stronger.

Thoughts?

Clean build environment produces warnings

On current master (17a0ca1)

make html produces this warning repeated frequently:

deprecation warning: io.FileInput() argument `handle_io_errors` is ignored since "Docutils 0.10 (2012-12-16)" and will soon be removed.reading sources... [

Environment is also pretty outdated, and could use a fresh coat of paint.

$ python --version
Python 2.7.15
$ pip list
Package          Version
---------------- -------
docutils         0.14
Jinja2           2.10
MarkupSafe       1.1.0
pip              18.1
Pygments         2.3.1
setuptools       40.6.3
Sphinx           1.2
sphinx-rtd-theme 0.1.5
wheel            0.32.3
$ pip list --outdated
Package          Version Latest Type
---------------- ------- ------ -----
Sphinx           1.2     1.8.3  wheel
sphinx-rtd-theme 0.1.5   0.4.2  wheel

So it appears that we likely want to update the baseline Sphinx installation, as well as move the entire project to Python 3, as Python 2 will end of life at the end of 2020 - see https://www.python.org/dev/peps/pep-0373/#update

Add an .editorconfig file and testing

As a Contributor, I want to have the editing rules auto-apply when editing where possible.
If I don't have an editor that applies EditorConfig on save, then the CI should also validate that the changes enforce the rules, and fails with feedback as to why.

https cert invalid

Could not find another way to report this. The ssl cert on https://www.opsschool.org is invalid for the domain.

www.opsschool.org uses an invalid security certificate.

The certificate is only valid for the following names:
  *.readthedocs.org, readthedocs.org  

Error code: SSL_ERROR_BAD_CERT_DOMAIN

Add a README to the project

Some contribution guidelines, etc.

Add my own curriculum?

Is there a way to add my own curriculum, as in specific classes?

I've completed dozens of online classes, many are linked here and 8 classes away from having my requirements for entrance into a PhD program so there is long term thinking.

We could create a simple csv format for ingestion of the class and people type stuff into a csv and OPSschool imports it?

kindle export

I would love to read this on my kindle. It looks like great content. Anyone know if there's a way to do this?

Can't make html

Didn't have too much time to look into this, but I am trying to set this up to read locally:

$ pip install Sphinx
omitted
$ make html
sphinx-build -b html -d _build/doctrees   . _build/html
Making output directory...
Running Sphinx v1.2.2

Exception occurred:
  File "conf.py", line 97, in <module>
    import sphinx_rtd_theme
ImportError: No module named sphinx_rtd_theme
The full traceback has been saved in /var/folders/gq/ns_zjvwd76b4d7kxmmvbmktw0000gn/T/sphinx-err-lPf2MF.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://bitbucket.org/birkenfeld/sphinx/issues/>. Thanks!
make: *** [html] Error 1

Overhaul "importance of documentation" section

http://www.opsschool.org/en/latest/soft_skills_101.html#the-importance-of-documentation

Probably ought to discuss:

• Ops team vs company-wide vs customer-facing docs, and their different goals and audiences
• Tradeoffs between...
  • official/authoritative/approved vs up to date right now
  • persistent (wikis) vs ephemeral (irc logs)
  • controlling who has access vs avoiding redundancy (the hazards of mismatched versions)
  • pretty vs easy to edit
  • tribal knowledge vs written out
• pros and cons of automatically generated documentation (code, APIs)
• if nobody can find a doc, it effectively doesn't exist. Develop a system for finding the information you need when you need it, or it's useless. (anecdote: I've worked at a company where there search features of the multiple internal documentation sites got so bad we just used an irc bot plugin to search all of them. works pretty well actually.)
• If people find a doc that's wrong but looks like it should be correct, it's often worse than if they had no doc at all because it makes them think they don't need to ask questions. However, an obsolete doc that's clearly marked as obsolete is better than nothing because it can help people understand the problem and ask better questions ("what changed in how X, Y, and Z interact since $DATE?" vs "what's X do?")
• good rule of thumb for what to document: if you had to ask a coworker for a piece of information and it's likely to still be true the next time a team member doesn't know it (either via forgetting or being new to the team), document it in the place where you looked before asking them. If it's something that will change constantly, document how to get the information.
• Checklists for repetitive tasks that can't be scripted -- update the checklist every time you use it if you find any flaws
• "passive" vs "active" documentation -- once someone has used a document several times they might not look it up every future time they perform that tasks. If you update the doc to change how things should be done, notify the relevant people in a way that they'll notice (email, IRC, meetspace). Similarly don't spam people with notifications of irrelevant changes unless you want them to ignore you (though that's kind of already covered in the section on effective email)

Ops School videos

The Contributions page says that a number of videos have been filmed already, including DNS 101 (for example), but when I look at the DNS 101 page it's still a scaffold. If that page is blank, then where does the material for the videos come from? Just want to make sure I'm not reinventing the wheel if I contribute. Thanks.

Error when running `make html`

When running make html following error is received:

Extension error:
Could not import extension sphinx.builders.latex (exception: cannot import name 'contextfunction' from 'jinja2'
make: *** [Makefile:45: html] Error 2

This prevent from building docs

Multiple `WARNING: Could not lex literal_block` on new fork

I forked with the intention of making a few contributions and encountered a number of warnings when running make html on a clean master. I was able to resolve one of the issues by installing graphviz. However, the following warnings persist:

/Users/bjs/code/curriculum/databases_101.rst:429: WARNING: Could not lex literal_block as "sql". Highlighting skipped.
/Users/bjs/code/curriculum/logs_101.rst:20: WARNING: Could not lex literal_block as "bash". Highlighting skipped.
/Users/bjs/code/curriculum/logs_101.rst:32: WARNING: Could not lex literal_block as "bash". Highlighting skipped.
/Users/bjs/code/curriculum/sysadmin_tools.rst:442: WARNING: Could not lex literal_block as "ini". Highlighting skipped.

Some googling suggests this is related to improper characters in blocks, but I haven't made any changes. Just want to ensure any subsequent PRs aren't impeded by this, or perhaps something is wrong with my local setup.

Thanks!

GitHub Pages or Website rendering as GUI?

This was previously discussed in regards to a blog but there were no forwarding links, just a mention of tasks management I believe. #82

While the trello boards are nice its one more app for me to have open and instead I wondered about a html rendered page that showed, dashboard, class, metrics of completion etc.

Would there be a way to add as an enhancement a blog of sorts that has a "You are 27% done with your goal" or "You have achieved 88% of your goal 99% on time" metric list of sorts.

Since tool building and dashboards makes up a big part of my client work I loved what your group has created and thought to ask.

Pivotal/Trello/whatever backlog?

My own two-pence/cent/whatever, but does this project have a Pivotal/Trello/whatever backlog that's open to the world? I think it would be really useful as a to-do tracker.

Organization questions

I have a few topics I'd like to tackle, at least to get the ball rolling, but I'm not sure where they'd best fit.

I think netstat, netcat, tcpdump and lsof fit well on the troubleshooting page, although I may have to add more items to the "common errors" list. Does it make sense to expand common errors or to think about an advanced network troubleshooting page or section?

I notice there's no troubleshooting 201 page, and troubleshooting 101 isn't identified as such (just "troubleshooting"); is that so that the more advanced troubleshooting goes into the more focused pages, like networking 101/201? I don't see a troubleshooting section in either of those pages (yet).

I'd also like to talk a bit about SSH port forwarding, SSH keys, and perhaps other more advanced ways to use SSH. Where would that best fit?

Perils of plain text authentication (HTTP, FTP, telnet): security? networking? 101 or 201?

Thank you very much.

Adding new section about operation systems.

I've noticed that there aren't any chapters about operation systems in general. I suggest the following content to be added:

OS 101

1. What is OS, Why do we need OS
2. What is kernel
   A short description of what is a kernel
3. User land vs kernel land
   Differences and how they communicate
4. User interfaces
   Different ways to interact with the machine. CLI and GUI
5. File systems
   Short introduction about file systems without going in depth
6. Application software
   Description of some common application software which often ships with the OS - File browser, web browser, text editors

OS 102

1. Architectures
2. File systems in depth
   More information about file systems like partitioning, block sizes etc..
3. Kernel in depth
   A few things about processes, scheduling, networks(maybe), how multi-user OS works. Maybe a few more things about kernels
4. Boot process
   Short description of generic boot process. Detailed description must be added in the sections for Unix and Windows

If the community agrees with my proposal, I'm ready to start writing.

Certifications

I'm thinking about writing a section covering certification but don't know of a suitable place to put this content. Any advice or suggestions where this topic should be placed?

Section on ethics

As systems administrators, we have a great deal of access to information - and more so with the Bring Your Own Device trend. I'm sure that we've all discovered things in the course of doing our jobs that we either wish we hadn't known, or have found it difficult to keep secret (examples include that coworkerA is going out with coworkerB, or that the company is definitely not doing as well financially as they make it seem).

I believe that the opsschool curriculum should spend some time (not necessarily a lot, but some) on dealing with this.

Add resources discussed in #hangops 2013-02-15

Links to mailing lists, irc channels, etc.

Feedback: Thanks!

Thank you all so much!

Especially for: "Learning and the Community" http://www.opsschool.org/en/latest/learning.html

TODO List - firewalls on host vs network

In progress

Also known as section - Firewalls and packet filters

Will discuss a some real world examples of “Crunchy outer shell, soft center is bad”.
Should be done in 28-48 hrs.

Broken RTD PDF download (looks like a graphviz rendering)

Steps to reproduce:

1. Click v:latest on sidebar
2. Click 'PDF' under 'Downloads' (https://media.readthedocs.org/pdf/ops-school/latest/ops-school.pdf)

opsschool.org doesn't redirect

On mobile, at least on mine (iOS 5.1 with Chrome and Safari), when I go to opsschool.org, it gives me a 'Page does not exist' error from Read The Docs, whereas going to 'www.opsschoo.org' works just fine.

However, opsschool.org redirects properly on my computer. I don't know why it works on my laptop, but not on my phone.

Need to discuss tech debt

How to identify it, how to address it, when to accept it.