Intro
These are a collection of useful personal notes about various technical work that I have experienced or done and may be useful for others.
Over time information in these notes will become outdated. The point is not to keep an updated set of technical notes but to have snapshots of how things could actually be done (though if I do learn updated methods, I will update the notes)
Plenty of other interesting information is available in my dotfiles
Alpine Linux
Various notes/techniques/information about using Alpine Linux as a primary Linux distribution
Boot (EFI)
Using EFI in Alpine is supported but the documentation is not consistently available in the wiki where one would expect.
environment
Setup the necessary environment variables
export BOOTLOADER=grub
export USE_EFI=1
packages
Drop syslinux
and add grub/friends
apk del syslinux
apk add grub grub-efi efibootmgr
Updated: 2022-02-09
Boot (Arguments)
Information that expands upon the documentation for Alpine's initramfs command line options.
ssh_key
- installs
openssh
- enables/starts
sshd
- takes the string literal from
ssh_key
and puts that value into/root/.ssh/authorized_keys
apkovl
APK overlay file to download and apply onto the system (this is a URL). Placing the
file using the device:fstype:path
method does not appear to work.
These are generally considered "Alpine local backups" and managed via lbu
if seeking more information.
alpine_repo
URL to use as the repository in the system
ip
In order to get an IP one can use =dhcp
OR define everything important
to get a network connection during init:
172.16.0.200:none:172.16.0.1:255.255.255.0:myhostname::none:1.1.1.1
^ ip address to request
^ server ip
^ gateway
^ netmask
^ system hostname
^ autoconf is N/A
^ DNS
Updated: 2021-10-21
Package Cache
Utilizing nginx
to provide an Alpine Linux package cache for apk files. The idea
is, on a network, to use nginx to provide a common repository of cached packages
that any machine has requested that can then be re-used by subsequent requests.
Below is an example configuration file (that would then be "included" into an nginx.conf
)
/etc/nginx/apk.conf
---
server_names_hash_bucket_size 128;
server
{
listen 9999;
root /srv/http/apk/;
autoindex on;
# Requests for package db, signature files and files db should redirect upstream without caching
location ~ \.tar\.gz$ {
proxy_pass http://mirrors$request_uri;
}
# Requests for actual packages should be served directly from cache if available.
# If not available, retrieve and save the package from an upstream mirror.
location ~ \.apk$ {
try_files $uri @pkg_mirror;
}
# Retrieve package from upstream mirrors and cache for future requests
location @pkg_mirror {
proxy_store on;
proxy_redirect off;
proxy_store_access user:rw group:rw all:r;
proxy_next_upstream error timeout http_404;
proxy_pass http://mirrors$request_uri;
}
}
# Upstream mirrors
# - Configure as many backend mirrors as you want in the blocks below
# - Servers are used in a round-robin fashion by nginx
# - Add "backup" if you want to only use the mirror upon failure of the other mirrors
# - Use separate mirror server blocks to be able to use mirrors that have different paths to the package repos
upstream mirrors {
server 127.0.0.1:8001;
}
# the proxy_pass directive should look like this
# proxy_pass http://mirror.domain.example/path/to/repo$request_uri;
#
# Notice that $request_uri replaces the /$repo/os/$arch part of
# the mirror address. See more examples below.
server
{
listen 127.0.0.1:8001;
location / {
proxy_pass http://dl-cdn.alpinelinux.org$request_uri;
}
}
Which can then be referenced by an Alpine install
vim /etc/apk/repositories
---
http://<host>:9999/alpine/v3.14/main
http://<host>:9999/alpine/v3.14/community
Updated: 2021-09-07
Installation (Raspberry Pi 4)
Assuming the boot media is created (sd card) then the following should, generally, get alpine up and running
- Partition disks (as preferred) - leaving the boot partition alone
- Use
ext4
BUT one will have to manually editsetup-disk
to allow the fs for the Pi - Make sure to
mount
the root partition to/mnt
- And then make sure to
mount -o remount,rw /media/<boot device>
and thenmount --bind /media/<boot device> /mnt/boot
- Finally run
setup-disk -m sys /mnt
This should get everything installed, after first boot one should edit fstab and make /boot/boot a bind mount to boot itself (FAT32 doesn't allow symlinking in this case)
Updated: 2021-10-21
Swap
To enable swap in alpine (for manual disk setup):
during install
mkswap /dev/<device>
swapon /dev/<device>
after install, booted into machine
vim /etc/fstab
---
UUID=<diskuuid> swap swap defaults 0 0
enable swap
rc-update add swap
Updated: 2021-10-21
Hardware
Various notes about configuring/setting up/using various hardware
IR Transceiver
This section is mainly about capturing IR signals from a Bryant IR-based remote (for a ductless system) to be able to replicate and playback the messages via software (allowing for home/lan integrations). This method would work for other ductless systems from other manufacturers though some changes may be required. Other USB transceivers may also work but that would likely require various changes to get lirc working properly.
Context
The reason all of this is even necessary is that lirc
by itself has a number of tools all around dealing
with IR reception/transmission BUT irw
, irrecord
, and irsend
only understand the "remotes" (or codes) that the
underlying lirc
configs understand. Most of these are for things like TV remotes (e.g. power on, volume up) and not for
anything quite like a ductless AC remote. In order to talk to such a different device one has to first get the
raw data from the remote in question and then "teach" lirc how to speak those codes (via configuration).
Hardware
- Alpine Linux (3.13), lirc 0.10.1-r0
- x86-64 server (also testing on a pi4 with Alpine 3.13)
- Irdroid USB IR Transceiver
- Bryant ductless system (Models: 619PAQXXXBBMA, 619PEQXXXBBMA)
Also works in Alpine 3.14
Setup
- USB transceiver plugged in
- lirc installed
lsusb
reports the Irdroid device
Capturing
We need to capture the raw device inputs, to do that:
mode2 -d /dev/ttyACM0 -H irtoy > output
Press a single button on the remote and then CTRL+C
There is an extraneous spacing output that will probably be the final output line, this is not part of the code we need so remove it
sed -i '$ d' output
Now make sure we only capture the code outputs (and none of the other extra mode2
outputs)
cat output | grep '^(spac|pulse)' | cut -d " " -f 2 | tr '\n' ' ' > code
This will have captured a single button press. It's important to understand that these remotes maintain the state of the system and therefore a "power on" press at 72 degrees will show up differently than a "power on" at 74 degrees (basically many commands must be captured if a lot of settings are wanted)
Config
First define the remote type and that we're using raw codes to communicate
vim bryant.conf
---
# ACSTOP (74 degrees)
# ACSTART (74 degrees)
begin remote
name BRYANT
flags RAW_CODES
eps 30
aeps 100
ptrail 0
repeat 0 0
gap 40991
begin raw_codes
Next specify the name of the command to include
vim bryant.conf
---
name ACSTOP
Finally the raw output
cat code >> bryant.conf
For any additional codes one must just create a name <NAME>
and then the output (see the full example below)
Close-out the remote configuration
end raw_codes
end remote
lirc
Now make sure to run lircd
with the configuration
ircd -H irtoy -d /dev/ttyACM0 bryant.conf
and then send commands!
irsend SEND_ONCE BRYANT ACSTART
Example
# ACSTOP (74 degrees)
# ACSTART (74 degrees)
begin remote
name BRYANT
flags RAW_CODES
eps 30
aeps 100
ptrail 0
repeat 0 0
gap 40991
begin raw_codes
name ACSTOP
4415 4394 554 1578 554 511 554 1578 554 511 554 511 554 511 554 511 554 1578 554 511 554 511 554 1578 554 511 554 511 554 511 554 511 554 511 554 511 554 1578 554 1578 554 511 554 1578 554 1578 554 511 554 511 554 1578 554 1578 554 1578 554 1578 554 1578 554 1578 554 1578 554 1578 554 1578 554 1578 554 1578 554 1578 554 1578 554 1578 554 1578 554 1578 554 1578 554 1578 554 511 554 511 554 511 554 511 554 1578 554 511 554 5183 4415 4394 554 511 554 1578 554 511 554 1578 554 1578 554 1578 554 1578 554 511 554 1578 554 1578 554 511 554 1578 554 1578 554 1578 554 1578 554 1578 554 1578 554 511 554 511 554 1578 554 511 554 511 554 1578 554 1578 554 511 554 511 554 511 554 511 554 511 554 511 554 511 554 511 554 511 554 511 554 511 554 511 554 511 554 511 554 511 554 511 554 511 554 511 554 1578 554 1578 554 1578 554 1578 554 511 554 1578 554
name ACSTART
4415 4394 554 1578 554 511 554 1578 554 511 554 511 554 511 554 511 554 1578 554 1578 554 511 554 1578 554 511 554 511 554 511 554 511 554 511 554 511 554 1578 554 1578 554 511 554 1578 554 1578 554 511 554 511 554 1578 554 1578 554 1578 554 1578 554 1578 554 1578 554 1578 554 1578 554 1578 554 1578 554 1578 554 1578 554 1578 554 1578 554 1578 554 1578 554 511 554 1578 554 511 554 511 554 511 554 511 554 1578 554 511 554 5183 4415 4394 554 511 554 1578 554 511 554 1578 554 1578 554 1578 554 1578 554 511 554 511 554 1578 554 511 554 1578 554 1578 554 1578 554 1578 554 1578 554 1578 554 511 554 511 554 1578 554 511 554 511 554 1578 554 1578 554 511 554 511 554 511 554 511 554 511 554 511 554 511 554 511 554 511 554 511 554 511 554 511 554 511 554 511 554 511 554 511 554 1578 554 511 554 1578 554 1578 554 1578 554 1578 554 511 554 1578 554
end raw_codes
end remote
Updated: 2021-09-07
Kinesis Advantage 2
Using a kinesis advantage 2 (on Linux) is not any different than any other operating system. A few useful notes, though, about working with it.
Export
To "export" the current configuration:
- Go into Power User Mode (progm+shift+esc)
- Enable vdrive (progm+F1)
mount -t vfat /dev/sda /mnt/vdrive
(or whichever device gets mapped, notice the lack of the partition)- Poke around in the mounted area, mainly in the
active/
directory - Make sure to
umount /mnt/vdrive
- Close Power User Mode (same key press as 1)
Updated: 2022-02-27
QMK
There is a lot of documentation (by QMK itself) about how to use their tooling to flash systems. The problem one runs into is trying to do this from Linux directly (therefore no QMK Toolbox).
QMK is an extensive project but much of the documentation/tooling is so generic that finding out how to "do things by hand" requires a bit of digging (looking through QMK Toolbox to extract commands to manually flash a keyboard). It's possible to fully compile the firmware and flash a keyboard on Linux from the command line.
keebio/iris/rev6
For example in order to flash a keebio iris (rev6) from (Alpine) Linux:
- Install:
dfu-programmer
,qmk-cli
,gcc-avr
, andavr-libc
- Expect
qmk setup
to complain about all the missing tools for other keyboards that don't matter for this ONE keyboard - Run
qmk json2c the.json > keyboards/keebio/iris/keymaps/enckse/keymap.c
if the online configurator was used and the json was downloaded - Run
qmk compile -kb keebio/iris/rev6 -km enckse
Now, at this point, it's probably easier to run dfu-programmer
against each half of the keyboard (remember to set it for flashing):
sudo dfu-programmer ATmega32U4 erase --force
sudo dfu-programmer ATmega32U4 flash --force enckse.hex
sudo dfu-programmer ATmega32U4 reset
Of course different keyboards will require different, specific tools but QMK itself is definitely trying to cover a LOT more than a single keyboard needs day-to-day
Updated: 2022-02-25
Linux
Information that is general to "Linux" and not a specific distribution
Certificates (acme.sh)
Remember,
acme.sh
is not an EFF/Let's Encrypt project so using it may come with some side-effects
acme.sh
can be used as a replacement to something like certbot
. One should
install acme.sh
via the distribution package management system.
data
By default acme.sh
will store/manage data in the user's home folder
under .acme.sh
(e.g. /home/myuser/.acme.sh
) but this data should only ever
be touched by acme.sh
and not used/touched by the user.
account
To associate an email with the account for acme.sh
operations
acme.sh --update-account --email <your email address>
issue/renew
To issue (--issue
) or renew (--renew
) utilizing multiple domains and via
a webroot.
acme.sh --issue -d my.example.com -d my.other.example.com -w /path/to/webroot
deploy
One can then actually get the useful certificate files for something like nginx
via:
acme.sh --install-cert -d my.example.com -d my.other.example.com \
--key-file /some/path/to/my.example.com/or/other/key.pem \
--cert-file /some/path/to/my.example.com/or/other/cert.pem \
--fullchain-file /some/path/to/my.example.com/or/other/fullchain.pem
Which map to nginx
directives:
acme.sh CLI | nginx config |
---|---|
--key-file | ssl_certificate_key |
--cert-file | ssl_trusted_certificate |
--fullchain-file | ssl_certificate |
Updated: 2021-10-21
Bind Mounts
A very useful tool in the Linux toolbox is a "bind mount" which can be used, for example, to mount a directory from one location to another (e.g. no symlinks, allows for more options like read only).
Example
For example to bind mount a "new /var" over the old:
mount --bind /mnt/data/var /var
These can also be placed into /etc/fstab
Updated: 2021-08-24
Cloud Init
Notes about using cloud-init to bootstrap a small system (mostly virtualized in the following use cases)
networking
It can be easier to set a static IP via kernel parameters (e.g. ip=
then it
is to find the documentation for cloud-init networking which will tell you to
use ip=
or some other arcane method)
post-boot
a cloud-init system can end up no longer using ip=
in some cases (e.g.
Fedora cloud images) which means creating an /etc/sysconfig/network-scripts/ifcfg-Wired_connection_1
may be required to handle networking with contents like (for Fedora)
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
IPADDR=192.168.64.2
PREFIX=24
GATEWAY=192.168.64.1
DNS1=192.168.1.1
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
NAME="Wired connection 1"
DEVICE=enp0s1
ONBOOT=yes
AUTOCONNECT_PRIORITY=-999
vmlinuz/initramfs
These can be pulled out of most ISOs though the ISO needs to have been built
with the virtio
module if it is going to be used for virtualization (e.g.
passing a kernel/initram to a process to start as a VM)
disks
It is easier to use a "cloud ready" image that is a disk image that can be
booted directly (and not booting an installation media/iso). Then one can set
the kernel parameter root=/dev/path2
as necessary to boot the rootfs object properly.
Updated: 2021-10-09
LXC
nesting
To enable nesting (containers in an LXC container) one wants to include
the /usr/share/lxc/config/nesting.conf
in a machine's configuration. Though if,
for example, something like an "apparmor" setting is in the config and apparmor
is not installed it may require taking the values from the nesting config and
placing them directly into the machine's config (mainly want to make sure things
like /proc
or /sys
or /dev
are setup properly as more than read-only)
This does have security implications
filesystems
lxc can do more than directory
-based containers (including image files via
loop, btrfs, lvm, etc.) during creation.
Updated: 2021-09-08
Memory
Dirty Memory
View status of dirty memory (e.g. sync
is running before umount
)
watch -n 1 grep -e Dirty: -e Writeback: /proc/meminfo
Updated: 2022-03-18
Traffic Control (tc)
Any useful notes about using tc
, one can use bmon
to interactively review the impact of playing with tc
basic traffic shaping
An example, on Alpine Linux running as a router, of using ifb
with tc
to try and shape some traffic (in both directions)
#!/bin/bash
# the internal "lan" interface LAN devices are "attached" to
LAN="lan0"
# the ifb device to create/use
IFB="ifb0"
# make sure the ifb device exists
ip link add $IFB type ifb >/dev/null 2>&1
ip link set $IFB up
# using ifb, via lan, to handle interface redirection
tc qdisc add dev $LAN handle ffff: ingress
tc filter add dev $LAN parent ffff: protocol ip u32 match u32 0 0 action mirred egress redirect dev ifb0
# using testing numbers
tc qdisc add dev $LAN root handle 1: htb default 10
tc class add dev $LAN parent 1: classid 1:1 htb rate 900mbit
tc class add dev $LAN parent 1:1 classid 1:10 htb rate 850mbit
tc class add dev $LAN parent 1:1 classid 1:20 htb rate 500mbit
tc class add dev $LAN parent 1:1 classid 1:30 htb rate 250mbit
# ideally these would be different as upload != download
tc qdisc add dev $IFB root handle 1: htb default 10
tc class add dev $IFB parent 1: classid 1:1 htb rate 900mbit
tc class add dev $IFB parent 1:1 classid 1:10 htb rate 850mbit
tc class add dev $IFB parent 1:1 classid 1:20 htb rate 500mbit
tc class add dev $IFB parent 1:1 classid 1:30 htb rate 250mbit
# filtering by IP/subnets
tc filter add dev $LAN parent 1:0 protocol ip prio 1 u32 match ip src 192.168.1.10/32 flowid 1:20
tc filter add dev $IFB parent 1:0 protocol ip prio 1 u32 match ip src 10.0.0.0/24 flowid 1:30
Updated: 2022-02-28
Virtualization (virsh)
Notes about using virsh
and related commands
Installation
Simple installation (using a vnc "display")
virt-install -n myvm \
--memory 16384 \
--cpu host \
--cdrom /path/to/rhel8.4.iso \
--disk size=50 \
--graphics vnc,port=5901,listen=0.0.0.0,password=myvncpass \
--network default \
--vcpus 2 \
--os-variant rhel8.4
or rebuild from a disk image with all the same settings, by changing:
virt-install -n myvm \
... \
# size=50 becomes...
--disk /path/to/existing/disk.qcow2 \
... \
# remove --cdrom /path/to/rhel8.4.iso
--import \
...
Which is helpful if the qemu XML file gets removed/deleted
once configuration/install is done then console=ttyS0,115200
can be
added to the kernel parameters and one can attach to the machine console
via virsh console myvm
attaching
disks
A block device can be directly attached
virsh attach-disk --domain myvm /dev/sda1 vdb --config --type disk
usb device
Need the code for the usb (e.g. lsusb -v
) to place into the following file
vim usbdevice.xml
---
<hostdev mode='subsystem' type='usb' managed='yes'>
<source>
<vendor id='0x1234'/>
<product id='0x5678'/>
</source>
</hostdev>
Which can be attached to a machine
virsh attach-device myvm --file usbdevice.xml --config --persistent
The detach-device
command can work to detach the usb device too
Updated: 2021-10-21
macOS
Various notes/information about using/working in macOS
Built In
vnc
macOS can open vnc connections via open
open vnc://myvncserver:5900
Updated: 2021-10-21
Disks
iso handling
alpine (rpi4) bootable disks
Partition the target
diskutil partitionDisk <target disk> MBR "FAT32" ALP 2GB "Free Space" SYS R
and then make sure to set it for booting
sudo fdisk -e <target disk>
> f 1
> w
> exit
Unpack the tar payload in the /Volumes/ALP
directory, and
vim usercfg.txt
---
enable_uart=1
gpu_mem=32
disable_overscan=1
cloud ready
To create a cloud-init ready iso on macOS, place "user-data" and "meta-data" in a configs/
directory (or any name)
hdiutil makehybrid -o init.iso -joliet -iso -default-volume-name cidata configs/
(make sure to specify -joliet -iso
because otherwise macOS will try to use -hfs
which many systems will not have installed/ready)
mount/unmount
an iso can attached or detached via hdiutil
hdiutil attach <file.iso> -mountpoint /Volumes/mymount
# and then
hdiutil detach /Volumes/mymount
Updated: 2021-10-21
Software
Various notes about doing small projects in "just software".
gpg
Passwords
Change existing password
gpg --passwd <id/email>
Updated: 2021-12-13
Image to TOTP Token
Given a relatively "clean" QR code image (without the underlying base32 code), one should be able to extract via decoding, e.g.:
package main
import (
"bytes"
"flag"
"fmt"
"image"
_ "image/jpeg"
_ "image/png"
"os"
"github.com/liyue201/goqr"
)
func main() {
file := flag.String("image", "qrcode.png", "image to decode")
flag.Parse()
path := *file
fmt.Printf("recognize file: %v\n", path)
imgdata, err := os.ReadFile(path)
if err != nil {
fmt.Printf("%v\n", err)
return
}
img, _, err := image.Decode(bytes.NewReader(imgdata))
if err != nil {
fmt.Printf("image.Decode error: %v\n", err)
return
}
qrCodes, err := goqr.Recognize(img)
if err != nil {
fmt.Printf("Recognize failed: %v\n", err)
return
}
for _, qrCode := range qrCodes {
fmt.Printf("qrCode text: %s\n", qrCode.Payload)
}
}
Updated: 2021-09-07
PowerShell
Files
Empty
Generate empty files of size
fsutil file createnew filename.txt <filesize>
Updated: 2021-12-13
ssh
Profile
noprofile
Connect without profile
ssh -t <host> bash --norc --noprofile
Host Keys
Scanning
scan keys for git forwarding/relaying/mirroring/etc.
ssh-keyscan -t rsa github.com >> ~/.ssh/known_hosts
Updated: 2021-12-13
vim
Replace Mode (Startup)
If vim
starts in replace mode:
.vimrc
---
nnoremap R <Esc> " Disable replace mode
Updated: 2021-12-13