szepeviktor / debian-server-tools

Tools and living docs 🧬 for Debian-based servers and Web Applications

Debian server tools

You find all my knowledge on GitHub in form of Bash and PHP scripts and Markdown documents.
Updated daily as I work.

Featured documents ⭐

1. Too much of a website
2. Web application development
3. Setting up your infrastructure and your application
4. All things WordPress
5. WordPress Security
6. Running Laravel
7. Running Drupal
8. Monitoring on paranoid level
9. Continuous Integration and Continuous Delivery
10. Debian setup on UpCloud server including Resize root filesystem during boot
11. Live list of Hostile networks
12. Bulk mail sending

Magyar nyelvű kiemelt dokumentumok 🇭🇺

1. Csapatának tagja szeretnék lenni
2. Ismerkedés Új ügyfelekkel
3. Teljesítménycentrikus webhely tervezése
4. Honlap jogi dolgai + GDPR
5. Kézi Tesztelési Kézikönyv
6. Képek életciklusa
7. Magyar email szolgáltatók
8. (direct) / (none) Google Analytics-ben

Support my work

Please consider supporting my work.

Thank you!

Superior cloud hosting

Install Debian on UpCloud

How UpCloud was chosen? Read https://github.com/szepeviktor/wordpress-speedtest/blob/master/README.md#how-to-choose-vps-provider

Ergonomics

How to Set Up Your Desk

Model of how systems work

Richard Cook at Velocity NY 2013

Directories

• /debian-setup - Debian installation scripts including debian-setup.sh
• /backup - Tools related to archiving
• /image - Tools related to image optimization
• /input - Tools related to the terminal
• /mail - Tools related to email account management
• /monitoring - Tools related to server monitoring, alert and statistics emails
• /mysql - Tools related to database management
• /package - Tools related to Debian packages and general packaging
• /security - Security and SSL certificate related tools
• /tools - Various small tools
• /virtualization - Docker containers
• /webserver - Tools related to building webservers

Debian install with UTC as timezone

Select Expert install.

Script development

• Check shell scripts: http://www.shellcheck.net/
• Code styling: https://github.com/git/git/blob/master/Documentation/CodingGuidelines
• Bashism: https://wiki.ubuntu.com/DashAsBinSh checkbashisms -f bash-script.sh

Install your own SSH key

S="${HOME}/.ssh";mkdir --mode 0700 "$S";editor "${S}/authorized_keys"
ssh-keygen -v -l -f "${S}/authorized_keys"

authorized_keys parameters:

# restrict == no-port-forwarding,no-agent-forwarding,no-X11-forwarding,no-pty,no-user-rc
restrict,command="echo 'Please login as the user \"debian\" rather than the user \"root\".';echo;sleep 10" ssh-rsa AAAA...

Install a user's SSH key

u bash -c 'S="${HOME}/.ssh";mkdir --mode 0700 "$S";editor "${S}/authorized_keys"'
U="$(stat -c %U .)";S="$(getent passwd $U|cut -d: -f6)/.ssh";mkdir -m 0700 "$S";editor "${S}/authorized_keys";chown -R $U:$U "$S"

Retrieve public key from private key

ssh-keygen -y -f ~/.ssh/id_ecdsa

Display SSH access details

printf 'host: %s\nport: %s\nuser: %s\n' "$(hostname)" "$(/usr/sbin/sshd -T -C user=root -C host=localhost -C addr=localhost|sed -n -e 's/^port \([0-9]\+\)$/\1/p')" "$(ls -tr /home/|tail -n1)"

List sshd host keys

ls /etc/ssh/ssh_host_*_key.pub | xargs -n 1 ssh-keygen -l -f

Record terminal session

script --timing=NAME.time --append NAME.script
scriptreplay --maxdelay 1 NAME.time NAME.script

Fast and safe transit of scripts (or any data) via copy&paste

Use magic-wormhole

Pack

cat $SCRIPT|xz -9|base64 -w $((COLUMNS-1))
# alias: cat $SCRIPT | transit

Unpack

cat $PASTED_FILE|base64 -d|xz -d > $SCRIPT
# alias: cat $PASTED_FILE | transit-receive

Moving away git-dir

git init --separate-git-dir=/home/user/gitdir

Produces:

/home/user/work-dir/.git: gitdir: /home/user/git

/home/user/gitdir/config: worktree = /home/user/work-dir

Rescan the SCSI Bus to Add SCSI Devices

echo "- - -" > /sys/class/scsi_host/host0/scan

Flush Google public DNS cache

https://google-public-dns.appspot.com/cache

Free CDN for GitHub

https://staticaly.com/ https://cdn.staticaly.com/gh/USER/REPO/TAG/FILE

Whois servers

• whois -h whois.iana.org $TLD https://www.iana.org/domains/root/db
• dig _nicname._tcp.$TLD SRV https://tools.ietf.org/html/draft-sanz-whois-srv-01
• https://github.com/rfc1036/whois/blob/next/tld_serv_list

AS information

whois -h whois.radb.net -- "-i origin AS202053"

ROA validation, RPKI status

whois -h whois.bgpmon.net 94.237.81.0

Crontab format

.---------------- minute (0 - 59)
| .-------------- hour (0 - 23)
| |  .----------- day of month (1 - 31)
| |  | .--------- month (1 - 12)
| |  | | .------- day of week (0 - 6)
| |  | | |
* *  * * *  USER  COMMAND

crontab -e -u USER
.---------------- minute (0 - 59)
| .-------------- hour (0 - 23)
| |  .----------- day of month (1 - 31)
| |  | .--------- month (1 - 12)
| |  | | .------- day of week (0 - 6)
| |  | | |
* *  * * *  COMMAND

Cron scheduling with timezone

# Well before the actual execution time!
00 06  * * *  echo "/bin/ls -l" | at "$(date --date='TZ="Europe/Budapest" 10:30' "+\%H:\%M")" 2>/dev/null

Looking for a dedicated team member running your application or website?

Contact me: [email protected]

debian-server-tools's Issues

Add support for Python 3 byte handling in jwk_convert.py

Hi, Viktor!

Just tried to use your tool to convert a private_key.json from the latest version of Certbot to PEM format using Python 3.9 under Debian 11 Bullseye and found this error:

Traceback (most recent call last):
  File "/home/ansible/bin/jwk_convert.py", line 31, in <module>
    PKEY[k] = enc(v.encode())
  File "/home/ansible/bin/jwk_convert.py", line 26, in enc
    return '0x'+binascii.hexlify(base64.b64decode(data, b'-_')).upper()
TypeError: can only concatenate str (not "bytes") to str

I had converted the script to Python 3 using 2to3 but handling strings of bytes seems to need specific knowledge, which I don't have. I think the problem is here:

data += b'=' * missing_padding

I've tried to research this topic without success. Would you feel like upgrading the script to Python 3? I think that the need to convert JWK to PEM is going to be there for a while for all us Ansible users.

Incidentally, I'd love to create an Ansible Playbook to do the whole process, maybe through a module.

Thanks in advance and happy new year.

mega-get.py error

hello

python2 mega-get.py https://mega.co.nz/\#\!zp9lASoZ\!eyRrHr48saZCCMUzzMZMGMw6gD8kAnDTb2JsloNriRk
Traceback (most recent call last):
  File "mega-get.py", line 199, in <module>
    getfile(url_parts[1], url_parts[2])
  File "mega-get.py", line 152, in getfile
    dl_url = file['g']
TypeError: 'int' object has no attribute '**getitem**'

Printable lower ASCII charaters

Print 3 times 32 (96-1) ASCII characters.

for C in {2..7}{{0..9},{A..F}}; do printf "\\x${C}"; done | tr '\177' '\n' | fold -w 32

 !"#$%&'()*+,-./0123456789:;<=>?
@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_
`abcdefghijklmnopqrstuvwxyz{|}~

👉🏻 non-ascii-hu.sh

How to run the resize before Ubuntu server 22 LTS boot

Hi,

Thanks for sharing those useful scripts. I am trying to resize an VM server with a LV mounted root with no free space at the end.

Would anyone please share how to run this script before before Ubuntu server 22 boot up?

Script file: debian-setup/debian-resizefs.sh

I tried putting it in the /etc/init.d/ folder but It seems does not work. Thank you very much in advance.

Regards.

Add internet-measurement.com IPs to scanners

https://internet-measurement.com/#ips

Additional subnets for Stretchoid

# 104.236.188.0/24
# 107.170.196.0/24
# 107.170.236.0/24

Prepare for automation

OS image cleanup

• input values: dialog=whiptail save as bash variables (country, )
• input values: provider default values (hardware: net, disk+mounts; kernel: tz, ntp, cpufreq, rng, irqbalance, modules; users)
• virt-what + OS image check + sanitization (systemd?) + apt sources + dist-upgrade
• personal prefs: root, user (/etc/skel/ w/first-login.sh then rm + several different prefs)
• configure installed packages (prefer: debconf)
• isntall services + configure (Linux daemons, ?etckeeper, mail delivery methods, fail2ban, nscd, /root/dist-mod)
• (list of) custom shell scripts + cron jobs
• populate /root/server.yml for every installed component
• system-backup.sh (debconf, etc, /root, user data, service data)

Docker-izing Apache

• Main configs, mods-available/ (SSL certificates), conf-enabled/
• Virtualhost configs (sites-enabled/)
• Filesystem access to document root with proper permissions + read-only
• Answer on TCP port 80,443
• FastCGI UDS socket/TCP port for PHP-FPM
• Permanent log storage
• etc...

Add PDF support to save_all_attachments.py

Line 130 results in .xxx if not .txt if I read it right,
I changed it to

if part.get_content_type() == 'text/plain':
    ext = '.txt'
elif part.get_content_type() == 'application/pdf':
    ext = '.pdf'
else:
    ext = '.xxx'

to ensure to get .pdf extension if it is a PDF, I know this is not originally from you, but I just found it here. Thank you.